Description
The interfaces in go-datastore have been needing updating for a while. This came up in #28 so i decided to start the conversation up. Lets collect all these ideas together here so we can move forward with this.
Interface additions i've been looking for involve streaming interfaces:
type StreamingDatastore interface {
// GetR returns a reader to read the value directly from the source (sans buffering)
GetR(k ds.Key) (io.Reader, error)
// PutW returns a new put writer that can be used to stream a value
// into the datastore
PutW() (WritePutter, error)
}
// WritePutter is used to stream values into the datastore. Since the caller may not
// know the key they want to store this value as until the write is complete (content
// addressing) the key is set after the write is complete
type WritePutter interface{
io.Writer
FinishWithKey(k ds.Key) error
}Also, for queries and maybe 'gets' in general, it would be nice to have a readonly view interface. Many storage backends (bolt in particular) allow you to read and operate on the data within the datastore without making a copy (i.e. using the datastores memory) within a closure. This would be really useful for things like 'load this protobuf object from disk' where we don't need our own copy of the bytes.
// A datastore implementing ReadClosurer allows passing a closure to operate
// on a readonly view of the data in question
type ReadClosurer interface {
GetFunc(k ds.Key, func(i interface{}) error) error
}Alternatively, this could be done with an output parameter and interfaces:
type OutParamer interface{
GetOut(k ds.Key, out DSUnmarshaler) error
}
type DSUnmarshaler interface {
DSUnmarshal(val []byte) error
}This would behave similarly to the json.Unmarshaler interfaces.