akka-exchange

Copyright © 2015-2016 BoldRadius Solutions

A demonstration multi-node Clustered Akka application including Spray, Sharding, Data Duplication (aka Replication), and Akka TCP / Streams.

This document outlines the design and usage of the project.

• System Design
  • Architecture Overview
• Vagrant / Docker Setup

System Design

The design of the system is to look sort of like a real live online trading exchange. Users are able to trade via both a Web interface; for larger customers such as banks a TCP/IP trade protocol is also provided.

Two types of security are tradeable:

• Stocks
  • Identifier: Ticker Symbol
  • Exchange Type: i.e., NYSE, NASDAQ, et. al.
• Bonds
  • Identifier: CUSIP
  • Bond Type: i.e., Municipal, Corporate, Treasury, etc

Architecture

The overall architecture is outlined roughly in the following diagram. Specific detail is found within.

The system utilizes Akka clustering, and is broken into several components. Each of the cluster nodes, uses several plugins which demonstrate various Akka Clustering features.

• Frontend Node(s): The Rest API [akka-http / seed node(s)]
• Trade Ticker Node: Replicated Feed of completed trades/current prices. Retrieves Trade notifications from the Event Bus. Saves the data via Akka Persistence.
• Trade Engine Node: A Cluster Singleton which proxies the transactions for requested trades. Makes sure there's only ever one instance so that it can gate trades on a single item.
• Trading Users DB Node(s): Sharded Database of active users and their portfolios (all the bids and offers they have on particular stocks & bonds). Utilizes Sharding & Persistence.
• Securities DB Node (s): Database of known Securities, i.e. Stocks & Bonds. Utilizes Data Distribution (aka Replication).
• Network Trade API Node(s): Network API for Trading, using a Binary Protocol. Utilizes Akka IO and Akka Streams for the sample client.

Notes

• TODO: Consider using the FSM DSL for Trade Engine. Does it work well with Clustering and Persistence?
• TODO: AtLeastOnceDelivery to ensure a crash can recover a trade? Trade timeouts probably won't tolerate this but need an excuse to demo it.
• TODO: Should we find an excuse for Cluster Client

Frontend

The frontend of the system is a node running akka-http¹ , exposing a REST API for web users to trade, browse offerings, etc. This node (or nodes, if you choose to run multiple to load balance, etc) also acts as seed(s) for the cluster. With the default run scripts, the first frontend node will start on port 2551.

1_{(note that performance is still not optimised for akka-http, but will be in future releases. Alternately we could use the highly performant Spray, upon which akka-http is based.)}

Trade Ticker

A persistent node of Trade Data. This provides a Feed of completed trades/current bids & asks (offers).

Utilizes Akka Sharding to spread out data & reduce single node resource usage. Additionally, uses Akka Persistence to ensure tolerance to crash/failure/actor migration without losing Trade data.

Trade Engine

A Cluster Singleton which proxies the transactions for requested trades. Makes sure there's only ever one instance so that it can gate trades on a single item.

Also determines if the system is up & available (beyond 'appropriate nodes up') by checking that all required components are in a 'tradeable' state.

Trading Users

Sharded Database of active users and their portfolios. The Portfolio represents all of a User's stock/bond bids & offers active in the system. Sharded for resource balancing, and Persistent, so it is tolerant to crash/shutdown/actor migration.

Utilizes Akka Sharding to spread out data & reduce single node resource usage. Additionally, uses Akka Persistence to ensure tolerance to crash/failure/actor migration without losing User data.

• Each Trading User has a sub-actor for their portfolio, with a subactor for each security type. So, each user will have a child Portfolio Actor, each of which has child actors for Stock Portfolio and Bond Portfolio. (TODO: maybe should be a trader has a StockPortfolio and BondPortfolio?)

Securities

Ephemeral (but Replicated) Database of known Securities, i.e. Stocks & Bonds. This is all of the Securities active in the system, with information on every bid offer, and trade that has occurred. This data is initialized cleanly from the Trader DB each time the node restarts, to ensure it is fresh at all times.

Utilizes Akka's Data Distribution module for Replication, so that there's a "Primary" instance of the Securities DB actor, with additional nodes acting as "Secondaries" consuming data. If the Primary node crashes/is shut down, the Secondary node takes over with the replicated data.

Network Trade API

Network based Trade System (TCP/IP with custom Binary Protocol) for emulating non-web users, such as large banks or high frequency traders.

This utilizes Akka I/O to handle the TCP/IP and serializing/deserializing of the messages. Interfaces to the system through the same Actor messaging as the REST API.

Vagrant and Docker Setup

In addition to being a demonstration of Akka Clustering, this project serves as a Docker demonstration. For now, the Docker configuration has been focused on running within a Vagrant host, but each module/node of the project has a full Docker config that can standalone.

Setting Up Vagrant

This Vagrant is setup to initialize a VirtualBox Linux host running Docker, in which each container is setup. The configuration for the VirtualBox host is in Vagrantfile.host; you could easily change that to spin up say, an AWS Server instead of VirtualBox. The actual Docker hosts are all configured in the primary Vagrantfile.

Best behavior of the Vagrant setup will require some Vagrant plugins:

vagrant plugin install vagrant-timezone # Set a timezone other than UTC on the host, for simplifying log reading

Parallel execution of vagrant (The default) causes some fun race conditions due to container linking. VAGRANT_NO_PARALLEL=yes vagrant up seems to be the best way to bring up all of the configured nodes.