More random little things

I continue to be scattered; dipping my toes into a little bit here and there. Actually, that’s a little harsh – its more like, we got some interruptions, I got behind in various things, so I’ve mostly been trying to catch up on life, leaving only a bit of time for fun, creative endeavors.

But, I must create/fun, or I get grumpy. So, the symptoms of my creativity:

I created a full color 3D sandstone print for my nephew for his birthday. Instead of Shapeways, I used Sculpteo.. and was very impressed with their service. I haven’t received the print yet, but I hope its good.

In City:Skylines (no recent picture available), I finally let a city grow up to get metro stations. I’ve found that geographical arrangement is secondary to having a good pipeline for traffic flow. Ie, you can do almost any arragement as long as there’s a (elevated to stay out of the way) highway leading back and forth between I<->O/S/C<->R. Next up if I get back to it is Cargo / Freight / Ships / etc.

I’m watching an awesome lecture series on Human Behavioral Biology from Stanford. I started at the end (thanks to a reddit post that led me there), and now I’m watching the whole thing. Most wow for me so far: Game Theory and how that was discovered to exist in real species, and the differences between Tournament and ___ species, and how (unaware) humans have some tournamenty-ness driving their behaviors.

At work, very recently, I get to embed Lua in a business app! (Lua is the scripting language used by World of Warcraft, for example). The idea is I have a very dynamic set of data – that I need to map to a very fixed structure. Rather than write brittle code (change a field, change code), or very complicated code (to describe how things map into a multi-dimensional structure), I instead host a script and let the script do the mapping – and the user can change the script. Set up stuff before, clean up stuff after, done. Currently in the test console app stage. (click on image to zoom in to see sample code).

I’ve been watching the ISS HD feed more often lately. I keep trying to get an overhead view of KY during the daytime, but I keep missing it. My next opportunity is 6:50pm tonight as I write this. Satflare.com/isshd provides a map right next to the live feed, and an overlay of features – pretty awesome. Be aware that at night, the cameras don’t pick up any city illumination so it looks very dark.

I ordered a replacement wire sheath for my Solidoodle 2. It arrived, but I haven’t had a chance to install it yet. I don’t have anything specific that I want to print.. so its not a priority. I do have a pattern growing in my head, having to do with collecting data from waze about driving habits and then making a 3D version of driving habits.

I’m halfway through a Football Kicker/Punter Coach-shopping video. This is actually for pay. Another hour left or so. This is my next priority; getting finished this weekend, its bugging me.

Oh! I also got to learn Photoshop in an evening, to make an advertisement for my mom’s business. Thanks to LFPL’s Lynda.com integration, I was able to watch the Photoshop Essentials course for free – and it got me enough mastery within an hour to be able to:

Use layers
Free-form crop to correct a picture of a sign
Color correction (my video experience helped here a lot)
using clip and mask layers to limit blurring to portions of the screen to make text more visible
using clip and mask layers to apply some selective de-brightening of overly bright elements

Very proud of how quickly I picked it up, but its also thanks to good coursework available.

I am a Jack of All Trades, master of none, but Apprentice or Journeyman at several.

Setting up a Private Key Double-Bind Retention System

It sounds really fancy, but its not.

The other day, I got to my office, and found that my fob was missing from my keychain. I traced my steps back, and found it on the ground next to my car – it had fallen entirely of my S-biner. Not cool.

So, this is my solution:

I have transferred all keys to very solid rings – not the little itty bitty loops that I’ve had 2 of them unwind on me.

Then, I’m attaching each ring to the other rings in a big circle of beeners. So if one fails, I have a backup. RAID? Nay, i say, RABID – Redudant Array of Beeners Implemented Dually. (I made that up) (Its okay for me to laugh at my own joke)

Angry Birds, btw, is my office key.

The Gray fob is the Anytime Fitness fob; and the White fob is a Tile. Interesting app, Tile. Another post, perhaps.

FortressCraft Evolved: Tier 1 Tour

I am continuing to thoroughly enjoy FortressCraft Evolved, an early-access game on Steam. Just about every week, I wait in anticipation of the stuff that DJ Arcas is putting into the game. This last time, it was an auto-excavator to automate creating large vertical shafts and a conveyor belt builder.

The final vision of the game is something where there is a definite resource-gathering, base-building, automating story, combined with a Tower-Defense game as the local Fauna try to dispatch your base (and you).

I’ve probably worked on 4 or 5 worlds now. Every time, I learn of something I could do better, and then I have the desire to start over and be more “perfect”. 68 hours into the game now. And I only just (last game) got down to Tier 2 technology, and there are three Tiers so far that I could play with.

Here’s a tour of my latest world, with my preferred way (so far) of doing things. This is at Tier 1 level, probably 3-4 hours into the game.

Central Power Hub and Smelter

Back

Front

I’ve ended up putting a coal storage hopper under the Central Power Hub (CPH) (the thing with the laser shooting out of it). The CPH draws coal from it to burn to create energy.
I then put a Pyrothermic Generator (PTG) next to the hopper.
Both the PTG and the CPH burn coal to generate energy, and charge up the battery (big block with a sphere in it). It seems like between the two of them, they can generate energy at about 10 units per second? (I’m averaging more like 7, but I’m not running it at max capacity yet).
This provides enough energy to run the smelter (front picture, “insert more ore”), as well as send enough power down the single laser to the rest of the base (only ore extractors right now).
The smelter takes in raw ore from the hopper below, and deposits the smelted bars into the hopper on the right (Front view). If I need to leave for a while, the hopper on the right gets a conveyor leading to another hopper so I can keep collecting while I’m AFK without the system backing up.
A single conveyor belt brings the ores in. This is enough for 4 types of ore, at the first upgraded level of drill motor (which means it generates 4 ore every N seconds instead of 2).

At first, I though “meh, why build an upgraded drill motor when I can just build a 2nd extractor for cheaper”. But, when I got to the 4x and 8x drill motors, I understood the wisdom of investing early.

Vertical Access Shaft

The Roof	Looking Down
Looking Up	Important Floor (Looking down, mid-fall)

My first worlds were wide more than they were tall.. at first. What I found when digging down is, you very often hit an ore vein on the way down. So now, I pretty much try to dig down rather than out, in search of ore.
I rely on the grappling hook (F) to get me back up. I use two torches on the roof to signify “which way is out” (its easy to get turned around) (See the “Looking Up” picture in more detail)
Now that we have falling damage (just this last week!), I have to build in ledges along the way to jump down to. Note that once I have a battery at a level, I can jump and take the fall damage (energy loss) and then recharge from the battery. I always put a torch under the ledge.
My guess is in the future, the grappling hook fire distance will be decreased and I’ll have to build room for lifts.
I line one side of the shaft with power, and the other side with conveyors. Right now, I’m only at tier 1, so a single laser is enough going down, and a single conveyor is enough coming up. So far, the most I’ve needed is 3 lasers and 2 conveyors, but I haven’t gotten very far.
Any level always has a “lip” around it, for safe landing purposes.

Floor / Level Exit from Shaft

I send power down into a battery, and then send it right back down the shaft again in the same vertical line.
Note that if ores block the laser’s path, sometimes I manually dig through the ore, or else I’ll shift the shaft to the left or right to accommodate.
At higher power levels, I set up the lasers next to each other, and likewise a receiving bank of batteries. Because the batteries share with each other, it evens out the power flow at that level.
I try to dig my conveyors one deep from the floor so I can walk over them with impunity. I have to route them to the other side of the vertical shaft to keep them out of the way of the lasers.
I started to build lasers so that they were above my height, however, I’ve given up on that now. Its just easier at ground height.

Coal Generation and PTG’s

I’ve found that for up to 5-6 PTG’s, I can have a single extractor generating coal onto a conveyor belt that then leads to the PTG’s.
Long before I get to my 2nd or 3rd PTG, the coal generation has exceeded storage space and there’s plenty of coal for all the PTG’s.
It takes about 3 PTG’s to fuel the max capacity of 2 unaltered T1 lasers. (30 vs 32). Until you build a second smelter, or an autoexcavator, you don’t even approach the halfway point of a single laser.
In this case, rather than send power to the coal extractor via laser, I just added a PTG right there. This was the second vein of coal (having exhausted the 10-coal courtesy seam that the easy level gives you), and it was only 13-15 conveyors away from the base.

That’s all I have time to share at the moment.

I wanted to make an actual log of the first few hours of gameplay – like “what task was I trying to achieve” and how quickly I could achieve it – however, I have run out of time, so that level of documentation will have to wait. What I can say is that the tasks changed very quickly, every 5-10 minutes. Besides, this is version 1.14e, which is far from the finished version, so the task list will likely change a lot in the future.

I am still super-excited about this game, and I eagerly await the next drop (bombs!!!??) and the eventual mayhem when mobs become antagonistic. (I hate camo-spiders already. They scare the goobers out of me. sneaking like that. Worse than creepers)

Fortresscraft Evolved

I have been playing a game called “Fortresscraft Evolved”, an early access (developer still working on it) game that my coworker Mike mentioned at lunch the other day. I really enjoyed Minecraft, so I decided to give it a try.

It has given me an entirely new build order to investigate. And, there’s a lot of “routing” to think about. I’m loving it. (Note, I’m specifically playing survival mode)

There’s a ton of tutorials out there. I’m not going to do a tutorial. But I am going to mention a bunch of things that I learned about, to calculate (for myself) an optimal build order. Granted, “optimal” really means “suited to my liking”, and granted, the furthest I’ve gotten in the game is to automate coal retrieval, but, this is where my thoughts are at.

Here were my most recent questions:

Extractor Priority

You start off with 3 extractors, but there are 4 initial ores to think about – Coal, Tin, Copper, Iron. Which one to ignore?

At first, this question was complicated by me not knowing about super dig. (link, scroll down to “The Extractor”). Given superdig, where you convert 1 energy for 1 dig (run out of energy fast) for a x% possible resource gain, I can quickly get about 40-50 ores out of a vein with the power that my suit holds.

This means that ore extractors are more for having a “passive” gathering strategy – power dumped in now, yields ores later, that you can go pick up – and in the long term a more efficient one (more ore gets salvaged, less destroyed)

To determine what I want, I choose my end goal: I want to

automate getting coal from the coal vein to the furnace to generate power. (2 storage hoppers, and 40-50 conveyor belts)
put a storage hopper by each extractor (tin, copper, iron = 3 storage hoppers)
build an extra extractor so we have an extractor on each vein (1 extractor)

I could also assign a bonus goal of automating delivery of tin, copper, iron to the smelter – I think for that I’d need 200-300 conveyor belts, and +2 storage hoppers (one for holding the ore, one for holding the bars). I’ll also need a bunch more torches. And some suit upgrades. Here’s the spreadsheet I put together:

So, it looks like Coal, Tin and Iron should should get your initial extractors; and then Copper when you build one for it later. This is all prior to the search for Lithium, which I’ll make a separate spreadsheet for later, if I get that far.

* Note: this list is already out of date. Watching the author play, turns out that PTG’s are better at power generation than CPH’s, so I should really add a PTG to the above list.

Ore Discovery

Unlike Minecraft, this game has an Ore pinger/scanner. Problem (for now) is that it either pings (a) Unknown ores, or (b) Known Ores. Once you discover an ore, to find a second vein of that kind of ore becomes problematical – too many positives. The solution suggested online was, when searching for a new ore, ONLY research it after you have found a LARGE vein. Note that you will probably find things in this order:

a) Snow, Rock, Dirt, Leaves, Trees, Stone
b) Coal, Tin, Copper, Rubble
c) Deep Stone (heh. Nice one, DJ – pisses me off every time)
d) Iron.

Vertical Navigation / Getting to the Ores

My first attempt, I started building the standard “stairs” down to get to an ore. It took a while. I did not know about the grappling hook. Which might get nerfed in the future.

Second attempt was to build an elevator shaft, but have a ledge at the top, so that I could grapple my way out of the shaft. I would strategically place lights so that if I’m facing a light, that means it’s the way out. However, I was being pretty at things, and that took a while.

Third attempt, I used super dig (ctrl-right) and zoom (Z) to my advantage

Do not dig straight down unless there was a ceiling above me that I could grapple back up to. Instead, try to dig at an angle.
After locating an sizable vein of an ore, either super-dig, or hold down the dig key, and just dig towards that ore, not being pretty about it. The idea is to be able to run/fall down to the ore.
Build a “wall” at the entrance of the hole to grapple back up to.
On subsequent visits to the ore, clean out annoying blocks and add in some lights on the way.
Make it pretty when doing conveyor belts.
Use a hill as a natural backdrop to grapple back up to.

It also appears that as you are falling, you can still control your fall using WASD. Watching the main developer DjArcas play on twitch, he builds these large chevron beams (probably using build-to-me Ctrl-Left) that he somehow jumps down onto (without missing). (If you watch the episode from 10/25/2014, at about 2h:30m or so, he’s building things / starting this world out)

If you are lucky enough to find a large cavern, dude, You’re in luck.

Ok, enough for this blog post. I am enjoying the game, its got a nice build order story and some nice layout challenges for laying out conveyors and power and stuff. Recommend playing, I’ve gotten a good 10 hours of play out of it so far (and I think its $9 or so).

[S|V]andalizing some Flip-Flops

I has modified some Fleep-Flawps.

My wife thinks I’ve lost my mind. She refuses to be seen with me, or let me be seen by others, wearing them. That’s okay, I can live with that. In-house only.

Points being:

That is elastic shoelace cord .. without damaging the shoelace (no cutting!). They were removed from another pair of dressy brown sneaker-like things – replaced with black elastic to make them blend in a bit better. (Once again, a stylistic difference between my grown up and my inner juvenile)
They don’t flip or flop anymore either.
It holds the ~~flip flop~~ sandal snug up against my inter-toe’s. I can even run in these things.
I don’t have to squish my toes to hold on to it anymore. I can let my feet spread apart. This is functionally very good for me. (Here’s why toe-hold is not so good)
It’s a zero-drop solution – which is what I’m used to. Try finding sandals which have no drop. Just try. We spent the day at DSW, Off Broadway Shoes, and Quest Outdoors, and not a single fudging pair has zero drop. (Keens has a pair that comes close, but they didn’t have it in my size) [Drop == the front is lower than the rear, or a “heel”. I cannot handle heels anymore, my feet have grown back to barefoot-comfortable state]
The backs stuck out too far – so I measured it, and cut it off, with a pair of scissors. Yet another reason why my wife won’t let me be seen in these.

I think that if somebody came up with a simple velcro loop thing, and some good marketing, they could make a possible killing with it. If you do this, I just want a few pairs, thanks. Maybe I’ll learn how to sew. Hmm.

(I love my life, and my wife, btw)

A Day at the Office: Sql Loops, Dapper Query Multiple

My wife pulled me out of the office today, to take her to the State Faire. However, right now, she’s meeting with her coffee girls. So: I can put some stuff that’s been brewing in my mind into a blog post.

Working with somebody I had not worked with before, I am learning a lot of interesting things:

SQL Loops

We’re working on getting data from hundreds of sites, pooling it, aggregating useful information, and then reporting it. The size of the pooled data is around 25 million rows, and will grow by several million a year.

When we change how the aggregation works (new math, new metrics), we have to delete all the aggregated data and re-aggregate. Due to foreign keys and such, truncating the tables is not an option (from a sproc, at least). A straight delete took forever and gave us no feedback.

My coworker used this construct to provide a way to measure delete progress:

Version 1:

while exists(select top 1 TrooperId from Trooper where TrooperId > @TrooperID)
begin
  select @TrooperID = Min(TrooperId) from Trooper where TrooperId > @TrooperID
  DELETE FROM [aggregated].[Statistics] where TrooperId = @TrooperID
end

I have never done a while statement in SQL before. Very elegant. It got around most of the problem of trying to delete too much without any feedback.

Even better, the next time I looked at the source, I found something else:

Version 2:

WHILE (1=1)
BEGIN
    DELETE TOP ( 10000 ) FROM [aggregated].[Statistics]
    IF (@@ROWCOUNT = 0)
         BREAK
    --WAITFOR DELAY '00:00:02'
END

Even simpler!

A Case for Simpler Dependency Injection

I’m used to code like this:

private readonly DbConnectionProvider _dbConnectionProvider;
private readonly ITrooperGetQuery _TrooperGetQuery;

public TrooperAccessGetQuery(DbConnectionProvider dbConnectionProvider, ITrooperGetQuery TrooperGetQuery)
{
	_dbConnectionProvider = dbConnectionProvider;
	_TrooperGetQuery = TrooperGetQuery;
}

Any dependencies your code has on other components, get “injected” into your constructor. Usually used along with a Dependency Injection library; to call this code, you would do something like:

var query = IoC.Get<ITrooperGetAccessQuery>();

Or, alternately, would in turn take a IITrooperAccessGetQuery parameter in their constructor, and rely on the person calling them to make the decision…

.. bubbling up all the way till you get to some magical code in the app that ties everything together. Somewhere else. This has always annoyed me.

With this approach, seems to be that a) constructors get really large and unwieldy, and b) “what’s going on” is spread out between where DI is set up and where it is consumed(*)

(*) Some would argue that’s the whole point of DI.

Anyhow, I found this style code which needed access to the TrooperAccessGetQuery:

private ITrooperAccessGetQuery _tagq;
private ITrooperAccessGetQuery TrooperAccessGetQuery
{ get { return _tagq ?? (_tagq = new TrooperAccessGetQuery()); } }

What this says: “Hey, my system is simple, I don’t need a master system putting everything together. My components know what components they normally depend on, but you have a chance to override. If you call the default constructor, I’ll give you a component with all default components.”

Thus, I added a default constructor to my component:

public TrooperAccessGetQuery() 
    : this(new DbConnectionProvider(), new TrooperGetQuery()) { }

And life proceeded. Its pretty, and avoids ugly constructors, and arcane app bootstrap code, and having to choose a dependency injector.

Dapper Query Multiple

I had not seen this construct before. Useful:

public static Counts Get()
{
	const string sql = @"
select  count_big(1)
from    imported.TrooperStatistics with(nolock)
where   is_summarized=0

select  count_big(1)
from    imported.TrooperStatistics with(nolock)

select  count_big(1)
from    aggregated.StarDateHash with(nolock)
where   is_aggregated_total=0

select  count_big(1)
from    aggregated.StarDateHash with(nolock)
where   is_aggregated_time_interval=0";

	var connectionProvider = ObjectFactory.GetInstance<DbConnectionProvider>();

	using (var connection = connectionProvider.GetOpenConnection())
	using (var multiple = connection.QueryMultiple(sql))
	{
		return
			new Counts
			{
				Total = multiple.Read<long>().First(),
				NotSummarized = multiple.Read<long>().First(),
				NotTotaled = multiple.Read<long>().First(),
				NotCalculatedTimeInterval = multiple.Read<long>().First(),
			};
	}
}

I learn something new every week. Thanks, JM!</P?

Ubuntu, Headless Mediasmart Server, Software Raid, Samba

I am now several days into this experiment. Its not working quite as I had hoped, but it is working. So here’s a roadmap/dump, with links along the way:

Headless Server Install

I had a HP Mediasmart EX-485 Home Server that went unresponsive. Did not appear to be a drive failure, the O/S started hanging. After rescuing everything, I had this pile of 4 hard drives sitting around – 750G, 1TB, 1.5TB, 2TB – and I was wondering what I could do with them. Well, build a server of course! Hey.. I have this hardware sitting around.. it used to be a Mediasmart EX-485 Home Server… But it doesn’t have a display, or keyboard, or mouse. There are places you can order some add-on hardware to do this, but it would cost me money.

I researched a couple of options, the winner was: to hook up the hard drive (I chose 750G) to another computer, install Ubuntu on it, and then additionally install sshd, and only then then transfer it to the mediasmart chassis. Luckily, most driver support is built in to the linux kernel, so switching hardware around is not a problem.

Then I downlod Putty on my windows desktop machine, and use it to connect to the server (formerly named MGZWHS, now named diskarray).

Adding in Drives and Making a Pool

I booted the server up, and checked what drives were available (sudo lsblk), and it only showed me /dev/sda and its partitions. As an experiment, with the server up and running, I hot-plugged in another drive, and looked again.. sure enough, there was a /dev/sdb present now.

I plugged in all the drives, then went and did some research. This lead me to LVM (logical volume manager), and there were a ton of examples out there, but all of them seemed to use hard drives of identical sizes.

At first, I thought I needed to repartition the drives like this guy did – so that I could pair up equal sized partitions, and then stripe across the partitions – But once I got into the fun of it, it became much simpler.

create PV’s for each disk
create 1 VG covering all the PV’s
create 1 LV with –m 1 (mirror level 1) on the VG. This command went crazy and did a bunch of stuff in selecting the PE’s to use for the different legs, and the mirror log …
create an ext4 fs file system on the LV

The –m 1 “peeks” into the physical volumes and ensures that any piece of data is backed up to 2 separate physical volumes – and as my physical volumes are all disks, they’re on different disks.

Surprise, though – it took about 2-3 days for the mirroring to catch up between the drives. Screen dumps available here: http://unix.stackexchange.com/questions/147982/lvm-is-this-mirrored-is-copy-this-slow

Creating a Samba Share

I then approximately followed these directions to create a samba share. However, its not quite right yet – the permissions of created files / the permissions of other files, its not quite matching. However, on my windows machine I can definitely see the files on my server, and its fast.

NB: You can see the .sync files and the lost+found of ext3fs

Syncing Via BtSync // stuck

I then followed these directions to instal btsync, and then attempted to sync my 1.2TB of data from my windows box to diskarray. It got part of the way there, 385MB .. creating all the files as user “diskarray” (herein lies my samba file sharing problem) – however, its gotten stuck. Windows btsync knows it needs to send 1.1G of stuff to diskarray .. they both come online .. and nothing gets sent. There are ways to debug this – I know how to turn on debug in windows, but have not yet followed the directions for linux – and eventually, I hope to fix it.

// todo: Simulating Drive Failures

I did attempt a drive failure earlier, but that was before the copy was 100% done – so, it was, shall we say, a little hard to recover from. Later on, I plan on taking one of those drives out while its running and see how I can recover. Maybe even upgrade the 1G to a 2G, if I ever take apart the external hard drive that I used to have on the TIVO. What should happen is the lv would degrade from “mirrored” to “normal”, and the unmirrored partitions would become unused. We shall see.

Figuring out a Sync strategy

At first I was going to draw this all out – I did, with VISIO, but the drawing isn’t quite right pretty enough. It does reflect the solution I’m going towards.

I have the following computers, with the following uses:

“Big” Computer – my main workhorse desktop at home. It does a lot of Video Editing, Audio Mixing, etc.

Currently has USB 3TB drive #1 mounted

Surface #1 – my most portable computer. Usually just edits things in the cloud, occasionally gets an offload of an SD card while I’m on the road.
Laptop #1 – My “I would program on this laptop” laptop, which I mostly use for Remote Desktop into my work computer
Laptop #2 – this is the laptop that controls the 3D printer. It has a lot of 3D printer tools on it.
Work Computer – for work stuff. Not my computer

Currently has USB 3TB drive #2 mounted

I pretty much use the pattern of:

C:\2014\<project Name>\ is where I put all files for a project.
I keep the same project name between computers if I move the project around. For example, when I was doing book design for Dad’s book, that started on Laptop #1, then moved to “Big” Computer
I consider “iTunes” to be a project, as well as my local instance of my Dropbox folder. Unfortunately, these are still in 2013.

My needs:

When I’m working in C:\2014, it needs to be fast and stable.
When I’m working in C:\2014, it eventually needs to get backed up to the big backup
Not all C:\2014 projects should be on all computers. In fact, almost NONE of them should be on the Surface, it only has a smallish SSD. Same deal with Laptop#2, that has 10G free or so after the O/S.
The Big Backup should be offsite-level backed up.

Limitations

A computer cannot btsync with itself (yet)
A R/O folder (like the phone backup destination) cannot be inside a R/W folder.

Options

I thought about something like this:

BigComputer 3TB \BtSync1 syncs to Work 3TB \BtSync1
I create a sub-sync of WORK:\BtSync1\users\sunny\2014 to BigComputer:c:\2014

It would work, but it would be a bit ugly. Lots of hops going offsite and then back home to get things backed up.

Winner?

I believe I’ve decided on the following:

The large 3TB drives maintain their own sync pool.
The local folders (C:\2014) MIGHT maintain their own sync pool, on a per-project basis. For example: 3dModels between my big computer and the 3d printer laptop.
Every project should end up at Sunny’s Big Computer
I’ll use Robocopy on a schedule to bridge the final gap from Sunny’s big computer c:\2014 to USB:\BTSync1\users\Sunny\2014
When I bring another machine online at home which can hold large drives, I’ll add it to the big sync pool as well to have a local (faster) backup. (Offsite can lag behind by days when we’re talking video files)

For things like backing up Mom’s important stuff, I’d probably create a USB:\BTSyncMom folder and have that be its own pool, not make it sub- or super- to any other sync pool. Or, continue to use Crashplan there.

Something I Don’t Yet Have

Windows Home Server gave me the ability to restore a machine to a specific state, covering all the installed software on a machine. I could do that using a copy of Ghost, or some other Hiren tools. I don’t yet have a plan for that. On the other hand .. reformatting a machine these days is “no big deal” for me, with all the data being out in the cloud somehow anyway.

Fun with Bittorrent Sync

BitTorrent Sync can be downloaded here: http://www.bittorrent.com/sync

Large Folders

I now have two external hard drives, one at home, one at work, synced with each other:

I first xcopy’ed one external drive to the other. This took about 20 arns.
I then hooked one drive up, and pointed BitTorrent at it. It started indexing.
I took the other drive to work, pointed Bittorrent at it there with the same shared folder secret, and it too started indexing.
The two instances of BitTorrent spent quite some time chatting with each other over the network: “I have this file!” “I do too!” .. getting to the point where they agreed that they were in sync with each other. This chatting and indexing phase took perhaps 2 days.
They are now in sync. They still “chat” at each other every now and then, but the network traffic is minimal. (I think)

Robustness

I then played with the robustness of the sync. First, I renamed a folder on one – did it transmit it as a rename? Why, yes it does:

I then turned BitTorrent Sync off on one of the sides (but not the other), and did some more renaming. I got a mixed bag of results:

What it looks like to me is, if BtSync is listening to a folder, it remembers actions taken and can catch up other clients to those actions; however if BtSync is turned off, it takes its new indexing data as “new stuff that happened” (ie, it doesn’t know there was a rename) and thus deletes and adds files as necessary. In the end, the two repositories are in sync, and the deleted files are copied over to the .SyncArchive folder.

iPhone Backup

I turned on the “synch my photos from my iPhone” feature. It creates a bitTorrent sync source, with only a readonly secret for others to consume.

If I delete a photo on my computer, it is NOT resynced from the phone.
If I delete a photo on my phone, it is NOT deleted from the computer
The files are not date/time stamped like they are with the Dropbox export.
It only synchronizes when I open the BitTorrent Sync app on the phone. There is no option for background refresh. (+1)

So far so good, I like it.

Nested Folders

The idea is this:

D:\BtSync1 is synched from A to B
On the same computer, can I additionally sync D:\BtSync1\Users\sunny\2014\3dmodels from A to computer C as C:\2014\3dmodels?

This way, I can keep specific projects (in this case, 3d printing stuff) synched between two computers, while having the data synced to the offsite backup as well?

Answer: Yes, as long as the parent (D:\BtSync1) and Child (+Users\Sunny\2014\3dmodels) are both Read-Write secrets. Ie, I could not place my phone backup folder in BtSync1, but I can do what I want above.

Testing it out:

Deleted file “LaundryBasketHandleV1.stl” on A (not shown by name in the log file below).

Deleted on B, and C

Deleted file “LaundryBasketHandleV1_fixed.stl” on C (“MOLLY-L2013”)

deleted on A and B

Created a new folder “coldwellbankercoaster” with a bunch of files on B (“SUNNYDESKTOP”)

copied to A and C.

I like it so far. Dude, I would pay $100 (one time, multi-install) or $15 (per computer) for this software. And for right now, its free!

I am brewing a plan for total world domination. My world, anyway.

Rescuing Files from Windows Home Server V1

My Home Server died a curious death. I didn’t loose any drives.. but the operating system started hanging. No more file copies, no more logging into the console, no more responsing to MSTSC. I had run this for close to 8 years, I think? Its hard to remember how far back. Sometime around when I got married, that we bought it, with Christmas Money. It has saved us 3 or 4 times with computer backups.

First job: Make sure I had a current backup of everything that was on the server. Could I get to the raw files? I popped a drive out, attached it to my Desktop PC using a SATA-to-USB .. and..

Access Denied! I had elevate my security level to gain access to the drive. Not a problem; I found Beyond Compare, and ran it as Administrator.

The other part of the equation was where to save the files. I had a 8 month old backup of the home server on an external hard disk that I brought back home from the office.

This is what Beyond Compare found for me:

The files on the WHS drive are under x:\DE\shares.
DE stands for drive extender.
The DE folder is normally marked hidden.
There’s actually another partition on the root of the home server, that has the folder structure for everything, and has soft-links to these files across drives.
The folder on the right, G:\2013-12-29_16… is how Home server backs up files to an external hard drive.
Everything is done via NTFS (or maybe it was FAT_32 for some of it). Very simple, straight forward stuff. I love it!
I put Beyond Compare in “Left Newer and Orphans” mode – and it found stuff on the drive that was not yet in the (much larger) backup.
Beyond Compare did a stellar job of copying the files.

Next up: BitTorrent Sync.