Garzik's various Linux projects
Menu:
cld: Coarse locking daemon and namespace service
Please visit Project Hail for updated information.
Inspired by Chubby, this software provides a cache-coherent reliable distributed filesystem for small files. Its primary uses include consensus service (election of a master, with fail-over, also known as lock service), reliable name space, and reliable [small] configuration file storage.
Notable attributes of this service:
- API presented to developers is that of a filesystem. This renders the API easily understandable to most developers.
- Consensus service, as a service. Coarse locking (locks held for hours, days, or longer) may be leveraged to provide master election and fail-over services for other distributed services built on top of this service. Centralized coarse locking frees other distributed services from having to implement the same high-availability fail-over code, over and over again.
- Offloaded consensus. For services built on top of this consensus service, consensus may be achieved even if less than half of the cluster is available. Consensus depends more on lock service high availability, than purely on number of downstream service cluster nodes.
- Cache coherence in a network filesystem. If you are willing to stall writes in order to process cache invalidations, then cached reads can proceed rapidly (in the common case) and without network activity, because the cache is known to be valid.
Status: Initial server and client coding complete. Debugging has just begun. Several key design elements not yet implemented: replication, client caching, switch from db4 to simpler database.
Download source code from the git repo (or browse)
Related: Chubby paper (PDF), Paxos made live (PDF)