Web users increasingly store and juggle their files across multiple locations sharing between web services like Facebook, Google Docs, company file shares and their personal laptop. This is a problem for both users and developers. For users, there does not exist an easy way to manage their distributed data. For developers, distributed data sources make it difficult to separate application code from user data storage. This distribution of data across multiple “clouds” makes data interoperability across different data sources difficult for both users and developers. One reason for this difficulty is that each system has a distinct UI and a disparate set of protocol for access. For example, web services like Google Docs, Box.net, and Dropbox may be accessed through different RESTful APIs, FTP requires the FTP or SFTP protocol, and data on individual computers may have several different means of remote access such as SSH or SMB (Windows File Share).
In November 2008, I created and built Upkast (www.upkast.com), a system for integrating and accessing multiple sources of data through a centralized protocol. First, I created an abstraction layer on top of the protocols for different distributed data sources by mapping them to a set of standard file system APIs. Then I created a centralized protocol for accessing data from these sources. I also created an interfacing layer that allows for new data providers to integrate with Upkast by implementing a set of standard APIs. With Upkast, interoperability of data between any set of different sources can be accomplished with a single centralized API action, regardless of the underlying system. To solve users’ interoperability and management problems, I created web-based desktop-like interface on top of Upkast. In this interface, a user’s different sources of data simply appear as drives or folders of the same file system. With this interface, interoperability, such as moving files and folders from one data source to another can be done with simple drag and drop actions as if the files and folders are on the same file system.
The Upkast system also has many broader applications for developers. By decoupling data provider and application logic, applications no longer need to worry about integrating with multiple data systems. For example, a word processing or photo editing application can be built on top of Upkast that can process documents and files from any source. Similarly, a search tool can be built so that file search can be performed across all of a user’s sources of data.