Garzik's various Linux projects
Menu:
storaged: single-node data storage service
Please visit Project Hail for similar projects.
This HTTP REST-based service is a very simple PUT/GET/DELETE data storage service. It is intended to be used as a low-level piece of large-scale data storage infrastructure.
The HTTP REST-based protocol is obviously ASCII-based. This is useful during development, but will be converted to a binary protocol once the API has stabilized.
The service provides operations on stored data ("objects"), and the containers used for organizing objects ("volumes").
A volume, sometimes also known as a cluster or partition, is an independent object namespace. Two objects stored on separate volumes may have the same [volume-] unique identifier, and have same or different contents.
An object is simply an anonymous blob of any length. When you PUT (upload) an object, you receive in return two items:
- an identifier, guaranteed to be unique within that volume
- a hash value (currently SHA-1), that may be used for optional end-to-end integrity checking, or in If-Match and If-None-Match headers.
From this basic interface (ancillary operations excluded):
-
PUT $URL/$VOLUME/$COOKIE
<- (len, data)
-> (sha-1 hash) -
GET $URL/$VOLUME/$COOKIE
-> (sha-1 hash, len, data) - DELETE $URL/$VOLUME/$COOKIE
you can build cluster-wide redundant storage services, exporting data storage capability from any device supported by your OS: directory full o' files (current storage backend implemented), one-big-file, NFS file, block device, SCSI OSD, ... all via a single, standard, easy to use, device-agnostic interface.
Status: early beta, all basic interfaces work; single-threaded; only
one storage backend implemented
Git repo: link (browse)