Followers

Monday, September 15, 2008

How Dropbox ended my search for seamless sync on Linux

By Ryan Paul

A few months ago, our own Jon Stokes bemoaned the frustration of managing and accessing data strewn across a multitude of personal computing devices. His description of the challenges of syncing and saving resonated with many of us on the Ars staff and with quite a few readers, too. As you can see by looking at the comments that some of you posted in response to Jon's synchronization conundrum, many of us roll our own imperfect solutions and yearn for something better.

For me, that "something better" has arrived in the form of Dropbox, a cross-platform sync tool. But I didn't discover it until I had tried everything else first.

Needlessly arcane solutions

As a Linux user, I have some pretty nice tools baked into the OS to help me get the job done. My approach is to store everything on my 4TB home NAS, which I can mount on my laptop via sshfs for secure remote access. It's a very seamless solution that works great when I have decent Internet connectivity, but it falls flat on its face when I can't connect or when my connection is unreliable. I had to augment this approach by using synchronization tools for critical files so that I still have access to important data when the tubes are clogged.

The sweet penguin juice roaring through my veins commanded me to adopt a needlessly arcane solution, so I started putting all of my article drafts into a private Bazaar repository on my personal web server. Real men use distributed version control systems for everything, right? It was unnecessarily excessive and got cumbersome quickly. I decided to scale back and try rsync, but it still wasn't transparent enough for my liking.

Then the penguin juice really kicked in. I started writing a wacky Python script that would use inotify to detect filesystem changes and automatically perform synchronization under certain conditions. During my futile struggle to get that hack into a usable condition, Ars Linux community member Whiprush saved me from myself and introduced me to Dropbox, a turnkey synchronization solution that gave me almost everything I wanted.

I began pursuing the path to computing zen by hotboxing my home office dropboxing my files.

Hands on with Dropbox

Dropbox is a cloud storage service with really smooth native platform integration on Linux, Windows, and Mac OS X. The Dropbox client software will automatically keep files synchronized between multiple computers and the user's Dropbox web storage space. It detects when files are modified on the local filesystem and will immediately upload the changes. The web service then propagates those changes to all other computers on which the user is running the Dropbox software.

The synchronization experience with Dropbox is impressively seamless and requires no user intervention. Another big win for users is that it works flawlessly across operating systems and provides the same level of fluidity on all three platforms. After suffering with rsync in Cygwin on Windows, I found Dropbox's excellent multiplatform support refreshing.

Dropbox shows a small icon in the user's system tray or notification area to indicate the current status of the service. It will display a blue arrow icon when it is in the process of synchronizing and a green checkmark icon when syncing is complete. When data is propagated over from the user's other systems, the icon will pop up notification bubbles that say which files were modified.

Dropbox synchronization is so fast that it feels almost instantaneous. After initial synchronization, it will send only the changes made to the files rather than sending the entire file every time. Dropbox also keeps each set of changes, which makes it possible for users to access previous versions of their files.

The client software also ties itself into the file manager on all three operating systems. Files and folders that are managed by Dropbox are adorned with checkmark and arrow icons to indicate their synchronization status. When a file is changed and is being synchronized by Dropbox, the icon overlay changes accordingly until synchronization is complete. Dropbox also adds a special submenu to the right-click context menu that provides access to some of the service's special features, such as revisions.

In addition to serving as a synchronization tool, Dropbox can also be used as a quick and easy way to share files with other people. Files that are placed in a user's Public folder can be accessed by other people who are given the public URL. This URL can be obtained through the Dropbox web interface or can be pushed directly to the clipboard by selecting the "Copy public link" item from the Dropbox right-click context menu.

The Dropbox web interface is minimalist, but it has a few really useful features. The main view is a list of recent events that will show which files have been added and removed. It also has a very fast file management interface that can be used to browse the files and perform some basic file operations. You can move, rename, and copy files, view and restore deleted files, and see file revisions. It will also give you a gallery view of folders that contain images. Another really nice feature is support for downloading any folder as a compressed zip archive.

During our testing process, we put the service through some rough challenges to see how it handles poor connectivity and change collisions. When the connection drops, Dropbox will wait until the connection is restored before attempting to synchronize again. To simulate a collision, I disconnected my laptop from the Internet and made a few modifications to some files in my Dropbox folder. Then I reconnected the laptop after I changed those same files on my desktop computer.

Dropbox handled the collision by creating copies of the changed files on the laptop and giving them special names to indicate they were conflicted files. Then it overwrote the originals with the latest version of those files from the cloud. The filenames of the conflicted files include the name of the computer on which they were modified and the date on which the modification was made. Dropbox can't merge different versions of modified files, but it makes it easy to see and manage conflicts.

Looking at the implementation

On Linux, the parts of the Dropbox client software visible to the end user are implemented as an open source plugin for GNOME's Nautilus file manager. The plugin is, however, just a thin shell and it doesn't handle any of the synchronization logic or communicate directly with the Dropbox web service.

The heavy lifting is done by the proprietary dropboxd daemon, which runs in the background and interacts with the Dropbox web service. Files are stored on Amazon S3, and each user gets 2GB of space. The Nautilus plugin automatically downloads the daemon and its dependencies and installs them in the ~/.dropbox-dist directory. The plugin uses the daemon as a gateway for communicating with the Dropbox web service. How it does this is not yet documented, but it is fairly easy to figure out by looking at the code.

According to the Dropbox developers, there are no plans to open the synchronization protocol, but they will document the protocol the plugin uses to communicate with the daemon. Third-party developers could use this, for instance, to create a KDE file manager plugin.

Dropbox is a real winner. The company jumped into a software space that is saturated with mediocre offerings and has delivered a better user experience and more cohesive platform integration. The Dropbox software does have some limitations right now that will make it a bit less palatable for enterprise adoption, though. There is currently no way to self-host Dropbox storage, so companies that need to manage their own data will have to wait. The Dropbox FAQ indicates that a self-hosted version could arrive in the future.

Original here

No comments: