Simple Distributed in-memory key/value store. GoKV provides high availability and fault tolerance which makes it suitable large-scale applications system without sacrificing performance and reliability. With GoKV, you can instantly create a fast, scalable, distributed system across a cluster of computers.
go get github.com/tochemey/gokv
Go-KV is designed to distribute key/value pair in a cluster of computers using push-pull anti-entropy method to replicate nodes' state across the cluster.
When a data entry is changed on a node the full state of that entry is replicated to other nodes.
This approach makes Go-KV eventually consistent. However, at some point in time the cluster will be in complete synchronised state. For frequent state synchronisation
one can set the syncInterval
value to a low value. The downside of a low value is that it will increase network traffic.
- Built-in client to interact with the cluster via the following apis:
Put
: create key/value pair that is eventually distributed in the cluster of nodes. Thekey
is a string and thevalue
is a byte array. One can set an expiry to the key.PutProto
: to create a key/value pair where the value is a protocol buffer messagePutString
: to create a key/value pair where the value is a stringPutAny
: to create a key/value pair with a givenCodec
to encode the value type.Get
: retrieves the value of a givenkey
from the cluster of nodes. This can return a false negative meaning that the key may exist but at the time of checking it is having yet to be replicated in the cluster.GetProto
: retrieves a protocol buffer message for a givenkey
. This requiresPutProto
orPut
to be used to set the value.GetString
: retrieves a string value for a givenkey
. This requiresPutString
orPut
to be used to set the value.GetAny
: retrieves any value type for a givenkey
. This requiresPutAny
to be used to set the value.List
: retrieves the list of key/value pairs in the cluster at a point in timeExists
: check the existence of a givenkey
in the cluster. This can return a false negative meaning that the key may exist but at the time of checking it is having yet to be replicated in the cluster.Delete
: delete a givenkey
from the cluster. Node only deletes the key they own
- Built-in janitor to remove expired entries. One can set the janitor execution interval. Bearing in mind of the eventual consistency of the Go-KV, one need to set that interval taking into consideration the
syncInterval
- Discovery API to implement custom nodes discovery provider. See: Discovery
- Data encryption using the
cookie
and the set ofsecrets
via the Config - Configuration can be customized. See Config
- Comes bundled with some discovery providers that can help you hit the ground running:
- kubernetes api integration is fully functional
- nats integration is fully functional
- static is fully functional and for demo purpose
- dns is fully functional
- Distributed cache
There is an example on how to use it with NATs here
To use the nats discovery provider one needs to provide the following:
Server
: the NATS Server addressSubject
: the NATS subject to useTimeout
: the nodes discovery timeoutMaxJoinAttempts
: the maximum number of attempts to connect an existing NATs server. Defaults to5
ReconnectWait
: the time to backoff after attempting a reconnect to a server that we were already connected to previously. Default to2 seconds
DiscoveryPort
: the discovery port of the running nodeHost
: the host address of the running node
This provider performs nodes discovery based upon the domain name provided. This is very useful when doing local development using docker.
To use the DNS discovery provider one needs to provide the following:
DomainName
: the domain nameIPv6
: it states whether to lookup for IPv6 addresses.
This provider performs nodes discovery based upon the list of static hosts addresses.
The address of each host is the form of host:port
where port
is the discovery port.
To get the kubernetes discovery working as expected, the following need to be set in the manifest files:
Namespace
: the kubernetes namespaceDiscoveryPortName
: the discovery port namePortName
: the client port name. This port is used by the built-in cluster client for the various operations on the key/value pair distributed storePodLabels
: the POD labels
Make sure to provide the right RBAC settings to be able to access the pods.