Skip to content
/ seppen Public

Simple distributed in-memory key-value store

License

Notifications You must be signed in to change notification settings

eiri/seppen

Repository files navigation

Seppen (雪片)

common tests

Simple distributed in-memory key-value store

What is it

Yet another distributed in-memory key value store with REST interface build on top of cowboy and ets.

Motivation

This is essentially a pet project built to study a best way to deploy a distributed erlang application in Nomad.

I'm toying with an idea of writing Nomad's erlang task driver and need some piece of software simple and familiar enough that its idiosyncrasies wouldn't stand in a way. This one day projet should be exactly this

Architecture

Storage

Storage is gen_server owning a private ets set table. Entries stored indexed by hmac of payload which acts as uid, meaning that the same payload with different keys stored only once. Keys, pointing on values' hmacs, stored in a separate index table. To simplify config erlang cookie used as a key for hmac.

Sharding

Placement

Sharding done with consistent hasing where each erlang node holds a shard range. The placement decided by first byte of value's hmac, so there might be max 256 shards. Ranges defined explicitly by nodes names provided in format {name}-{range beginning inclusive}-{range ending inclusive}@hostname. If beginning of the range large than ending, the node will hold two ranges flipped over 0.

Configuration examples

The explicit definition of a shard allows flexible configuration for number of shards and copies. For example:

  • Single shard, single copy: seppen-0-255@hostname
  • Three shards, single copy: seppen-0-84@hostname, seppen-85-169@hostname, seppen-170-255@hostname
  • Three shards, two copies: seppen-0-169@hostname, seppen-85-255@hostname, seppen-170-84@hostname

Limitation

Seppend been a toy project done mostly to study deployment of erlang apps in Nomad doesn't deal with nodes failures, handovers, brain splits and the rest of the complex distributed data stuff. This maybe get tackled later, during looking at handeling relup and Nomad cluster degradation.

Interface

Interface is a cookie-cutter cowboy REST web-server. The web server on each node is stateless, load-balancing done by external means (haproxy or traefik)

API

OpenAPI specs

Build and test

$ make test
rebar3 eunit
===> Verifying dependencies...
===> Compiling seppen
===> Performing EUnit tests...
.......
Finished in 0.160 seconds
7 tests, 0 failures
rebar3 ct --name [email protected] --setcookie legNe7mIRCiMEy5X --suite seppen_SUITE
===> Verifying dependencies...
===> Compiling seppen
===> Running Common Test suites...
%%% seppen_SUITE: ..................
All 18 tests passed.

Distribute testing:

$ make dist
rebar3 ct --name [email protected] --setcookie legNe7mIRCiMEy5X --suite seppen_dist_SUITE --readable true -v
===> Verifying dependencies...
===> Compiling seppen
===> Running Common Test suites...
...

TEST INFO: 1 test(s), 6 case(s) in 1 suite(s)

Testing lib.seppen.seppen_dist_SUITE: Starting test, 6 test cases
%%% seppen_dist_SUITE ==> test_put: OK
%%% seppen_dist_SUITE ==> test_get: OK
%%% seppen_dist_SUITE ==> test_get_if_none_match: OK
%%% seppen_dist_SUITE ==> test_get_list: OK
%%% seppen_dist_SUITE ==> test_delete: OK
%%% seppen_dist_SUITE ==> test_get_empty_list: OK
Testing lib.seppen.seppen_dist_SUITE: TEST COMPLETE, 6 ok, 0 failed of 6 test cases
...

Distributed testing suite starts two nodes '[email protected]', '[email protected]' each holding a range and confirms proper distribution of test data across the nodes. Been common tests there are a chance for testing to leave behind slave nodes on test crash, this could be cleaned up by make dist_clean

Deploy to Nomad

TBD

Name

Seppen (雪片) means "snowflake" in Japanese and also can be written as 切片, meaning "segment". The name hints on ephemeral nature of in-memory store and its distributed "sharded" architecture.

License

MIT

About

Simple distributed in-memory key-value store

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published