Defining the interface
In the last post I decided that I would be using Thrift to define the service interface, exceptions and data types for my kev/value store. Thrift was invented by Facebook (full disclosure: they are my employer at the time of this writing) and open sourced via the Apache Foundation, then internally improved and open-sourced again as fbthrift on Github. Note I am using Apache Thrift for this project, though I would recommend if you want to use Thrift in production in C++ specifically that you give fbthrift a look first.
For documentation on the Thrift IDL as well as some best practice type guidance Thrift: The Missing Guide is what I point people towards in and outside of Facebook. I will intentionally be iterating the thrift IDL in conjunction with the service code, so I will be making use of things like optional fields to support backwards compatibility with clients.
There is no right or wrong process to define a thrift interface, personally I tend to either have a firm idea of where I want some aspect to start and so I write that and then fill out the other bits as required to support the part I think I have a clear picture of. For a key/value store the fairly obvious starting point is the service API.
[code language=”text”] service KVServer {bool set_key(1: KVObject kv) throws (1: ServiceException service_exception);
string get_val(1: string key) throws (1: KeyNotFound key_exception, 2: ServiceException service_exception);
KVObject get_obj(1: string key) throws (1:KeyNotFound key_exception, 2: ServiceException service_exception);
void del_key(1: string key);
}
[/code]
At the core here are four methods with slightly awkward names thanks to set being a reserved word, and my personal feeling that its more readable to be slightly awkward but consistent. This interface has some intentionally sub-optimal choices, like del_key returning void and promising not to throw which effectively offers a contract that deletes can never fail. There are also some custom types here, rather than make this post extremely long I’m showing the finished form of a few rounds of iterating on how complex I wanted my exceptions to start and how far I wanted to take my key/value abstraction.
A quick look at these types:
[code language=”text”] exception KeyNotFound {1: string key
}
[/code]
This is a pretty basic exception for when a get operation fails due to a missing key.
[code language=”text”] exception ServiceException {1: string what,
2: bool retryable, // Should clients consider this a hard failure
}
[/code]
With the ServiceException we have slightly more going on, and possibly a bad idea. I have a hunch I will quickly find retryable and non-retryable error types and being able to tell my clients when to go away for good seems useful. Not having to keep things in sync seems like a win, but I’m prepared to pitch this overboard quickly if I don’t find a good usecase fast.
Thrift does not support inheritance so I am hedging a bit by calling this a service exception and leaving myself room to define more specialized exceptions if it turns out that stringly typed isn’t enough.
[code language=”text”] struct KVObject {1: string key,
2: string value,
}
[/code]
I first started out with an interface that took string arguments and returned string values, and really there is nothing inherently wrong with that. I quickly realized I wanted to implement a primitive call like get_obj and thus I needed a struct to represent a k/v pair which could be returned as a unit from a single call. The reason I know I want this is that I will want to eventually look at prefix scans, and its slightly frustrating to get the results of a scan as a list of values without knowing their keys. Note that in Thrift also strings are encoding agnostic and thus can be a text or binary string.
The remainder of the thrift file which has the rust namespace declaration and some comments can be found on github: https://github.com/nharring/kv_server/blob/315ad3fc7360b00132a7b550623ac44c2abbeb6a/kv_server.thrift