BitTorrent Sync can be downloaded here: http://www.bittorrent.com/sync
Large Folders
I now have two external hard drives, one at home, one at work, synced with each other:
- I first xcopy’ed one external drive to the other. This took about 20 arns.
- I then hooked one drive up, and pointed BitTorrent at it. It started indexing.
- I took the other drive to work, pointed Bittorrent at it there with the same shared folder secret, and it too started indexing.
- The two instances of BitTorrent spent quite some time chatting with each other over the network: “I have this file!” “I do too!” .. getting to the point where they agreed that they were in sync with each other. This chatting and indexing phase took perhaps 2 days.
- They are now in sync. They still “chat” at each other every now and then, but the network traffic is minimal. (I think)
Robustness
I then played with the robustness of the sync. First, I renamed a folder on one – did it transmit it as a rename? Why, yes it does:
I then turned BitTorrent Sync off on one of the sides (but not the other), and did some more renaming. I got a mixed bag of results:
What it looks like to me is, if BtSync is listening to a folder, it remembers actions taken and can catch up other clients to those actions; however if BtSync is turned off, it takes its new indexing data as “new stuff that happened” (ie, it doesn’t know there was a rename) and thus deletes and adds files as necessary. In the end, the two repositories are in sync, and the deleted files are copied over to the .SyncArchive folder.
iPhone Backup
I turned on the “synch my photos from my iPhone” feature. It creates a bitTorrent sync source, with only a readonly secret for others to consume.
- If I delete a photo on my computer, it is NOT resynced from the phone.
- If I delete a photo on my phone, it is NOT deleted from the computer
- The files are not date/time stamped like they are with the Dropbox export.
- It only synchronizes when I open the BitTorrent Sync app on the phone. There is no option for background refresh. (+1)
So far so good, I like it.
Nested Folders
The idea is this:
- D:\BtSync1 is synched from A to B
- On the same computer, can I additionally sync D:\BtSync1\Users\sunny\2014\3dmodels from A to computer C as C:\2014\3dmodels?
This way, I can keep specific projects (in this case, 3d printing stuff) synched between two computers, while having the data synced to the offsite backup as well?
Answer: Yes, as long as the parent (D:\BtSync1) and Child (+Users\Sunny\2014\3dmodels) are both Read-Write secrets. Ie, I could not place my phone backup folder in BtSync1, but I can do what I want above.
Testing it out:
- Deleted file “LaundryBasketHandleV1.stl” on A (not shown by name in the log file below).
- Deleted on B, and C
- Deleted file “LaundryBasketHandleV1_fixed.stl” on C (“MOLLY-L2013”)
- deleted on A and B
- Created a new folder “coldwellbankercoaster” with a bunch of files on B (“SUNNYDESKTOP”)
- copied to A and C.
I like it so far. Dude, I would pay $100 (one time, multi-install) or $15 (per computer) for this software. And for right now, its free!
I am brewing a plan for total world domination. My world, anyway.
How about $40 yearly 😀
I’m glad you commented!
Yeah, so after I wrote the above blog post.. things started to go weird. It started syncing the same sub-folder over and over .. got stuck in a loop .. chewing up a lot of bandwidth. I aborted the bittorrent sync experiment.
Since they went paid, I’ve been meaning to re-do the experiment, but I haven’t created the time to do it. I hope it works, right now I have no current off-site backup of my home linux array.
$40 a year, no way. (I like to buy rather than rent). But I bought the pro version even after being a heavy AeroFS user, because BTSync has improved over time. The free option is still useful, but if you have 10 folders and 5 devices, you have to set up all 10 manually on all 5 devices, or 50 times.. Also selective sync is nice for laptops without a lot of storage.
I set up the following:
1. Always on Mac Mini Server with 2TB Disk and public IP and set to sync all new folders by default (Selective Sync off).
2. My Macbook Air. (Which I pretty much use as a desktop) Set to sync all by default with selective sync off.
3. My Macbook Retina – Set to selective sync only since it has limited space.
4. My PC laptop – Set to Selective Sync only.
5. My phone (Android).
6. One more AeroFS server set up in the cloud using Linux on DigitalOcean.
Notes:
1. This is an awesome way to upload stuff from your phone
2. This is a great way to share stuff with friends
3. This is a reasonable way to sync stuff between computers. I has conflict issues, etc. with older versions, but not so much recently.
4. I don’t like that it doesn’t run as a daemon in OS. (i.e. you have to log in in order for it to work.)
5. I don’t like the subscription model they want for professional use, so if I decide to use for professional use, I would move back to AeroFS, which is at least free for the first 20 users.
6. Why does the web interface only work in Linux?
7. I set up a linux “droplet” fairly easily, but it would be great if they had a dedicated minimal VM you could download for server purposes.
8. It would be great is they supported a server version with deduplication and/or encryption. (So you could have a server on a hosting site without trusting the host).
Thank you for the suggestion Noa!
What I’m hearing you say is that btsync is working well for you at a 2TB size … and that possibly might want to retry it. Thank you!
Hi, yes it works well overall. There are still some wrinkles, however:
1. Even if your drive is full, it will still try to keep syncing until your drive has no space left. When that happens, strange things happen – both with btsync and with other apps. I noticed it on one PC because safari kept crashing.
2. When adding folders from someone else onto your own devices, they will be added to *all* your devices that are set to full sync if you are using Sync Pro, and using the default folder name (which you may not even know). And.. #1 can happen. As an example, a friend shared with me his video folder, and I added it on my phone. Little known to me, my laptop at home started downloading everything from his folder (many hundreds of gigabytes!) and the folder was named “public”. In this sense, using Pro can be like a downgrade! Generally you want your own folders to be replicated to all your devices by default – but not other people’s folders!
3. Sometimes cross-platform sync is not perfect – but that can’t be helped, as there are differences between the platforms that no tool can solve.
4. If you might want to share out something separate, make it into a separate folder from the start. For example, I had a folder like “people” that I used to keep files related to other people and then inside that, folders like “sue”, “bob”, etc. When I wanted to share the “bob” folder with Bob, it turned out to be a pain. (I could share the sub-folder separately, but that is a bit buggy – for example, the subfolder started syncing to my other machines separately!)
5. The selective sync feature really is awesome in that it allows you to literally click a file you don’t really have and open it a few seconds or moments later. But, I recommend it mainly for folders you will be accessing in read-only. If you want to move files in that folder, etc., you will run into strange issues sometimes if you perform the action on a machine that has only selective sync.
That said, BTSync is still a great tool. It can manage a large number of large files, and it is good for sharing files with other people as well. It has improved greatly since the first time I used it, and I am sure it will continue to improve.