RedisClient

⚠️ 🚧

This project is abandoned.

It was a prototype and the excuse to learn how Redis works. After reading about MULTI, WATCH and the benefits of EVAL in the monothreaded Redis, the "procedures" seemed like a good idea, but later on I discovered how Redis Cluster works, and then became more clear that Lua scripts in Redis were mean to be short and concise macros rather than big pieces of code. A procedure using keys that are stored in different cluster leaders would fail. Also, although I implemented a connection multiplexer for both commands and subscriptions, the idea of having multiple connections in which I could schedule blocking commands turned out to be messy and limited.

All in all, it was good fun and a learning experience.

⚠️ 🚧

Status: ALPHA

An experimental .NET Redis client that uses a special syntax for easing LUA script invocation named "procedures". The interface is based on templated strings, allowing to execute custom defined server side procedures as regular Redis commands. Executions are done through "channels", in essence virtual connections that provide seamless access to Redis through three different connection pools.

• Templated strings interface (more about parameter binding).
• Transparent connection management (more about connection management).
• Basic output binding (more about output binding).
• Server-side scripting through procedures (more about procedures).
• A debugging tool for procedures is available in this repository.
• Support for asynchronous, synchronous and "fire and forget" operations.
• Redis Cluster: Not supported at the moment, but is the next thing in my roadmap.

Why procedures?

• They have all the benefits of regular LUA scripting in Redis, like atomicity and avoiding multiple round trips to the server, since a procedure is just a way to wrap regular LUA scripts.
• Multiple procedures can be defined in a same text file since they are limited by proc and endproc boundaries.
• RedisClient handles the procedure deployment.
• Procedures are invoked by name rather than using EVAL or EVALSHA.
• Instead passing parameter values in KEYS and ARGV arrays, and having to hardcore the index location of the data in those arrays, named parameters are used.
• Array parameters of arbitrary length are supported, being the length of those arrays defined at parameter value binding time.
• Procedures in RedisClient use the same parameter binding capabilities than for regular Redis commands.
• Procedure results can be inspected as any other Redis command.
• Learn more about procedures.

Simple procedure example

Imagine a catalog application, with products/services defined as different hashes in the Redis database, where each hash contains the properties of each product, like name, url, stock, description, picture url, etc... Also you have different zsets containing the keys of all products sorted by a specific properties or just grouped by categories. Since there may be a big amount of products, pagination is needed to avoid blowing up the server response with too much data.

How is this pagination achieved without server side scripting? First querying the zset with the desired range to obtain the list of hash keys that need to be retrieved, and then retrieving each key (usually) one by one.

This can be expedited and simplified with server-side scripting. For example, you can use a procedure to get the list of products directly, without extra round trips:

proc ZPaginate(zset, page, itemsPerPage)
	
	local start  = page * itemsPerPage
	local stop = start + itemsPerPage - 1
	local items = redis.call('ZREVRANGE', zset, start, stop)

	local result = {}

	for index, key in ipairs(items) do
	    result[index] = redis.call('HGETALL', key)
	end

	return result
endproc

Using the templated string syntax you can invoke this procedure easily:

// Execute procedure
var result = channel.Execute("ZPaginate @key @page @items", 
                              new { key = "products:bydate",  page=3, items=10 });

// Expand result of the first line as a collection of results
var hashes = result[0].AsResults();

// Bind each hash to an object
// Where <Product> is a class with properties that match the hash keys.
var products = hashes.Select(h => h.AsObjectCollation<Product>()).ToArray();

Server side scripting has multiple advantages, like preventing multiple round trips to the Redis instance or atomicity. Continue reading about procedures

Use in a web application

Also in this repository you will find SimpleQA, a proof of concept of a Q&A application using RedisClient.

Getting started

• Installing.
• Setting it up.
• Binding parameters.
• Getting results.
• Subscribing to channels.
• Executing procedures.

Installing

The alpha version is available in NuGet.

PM> Install-Package vtortola.RedisClient -Pre

Setting it up

The API has two fundamental parts:

• RedisClient class handles the connection management. Usually you have one instance across all your AppDomain (or two instances if you have master/slave). It is a thread safe object, that usually is cached for the extend of your application lifetime.

_client = new RedisClient(endpoint))
await _client.ConnectAsync(CancellationToken.None).ConfigureAwait(false);

• IRedisChannel interface is used to execute commands. Channels are short lived, cheap to create, non-thread safe objects that represent virtual connections to Redis. Channels provide seamless operation for commanding and subscribing.

using (var channel = _client.CreateChannel())
{
    await channel.ExecuteAsync("incr mykey")
                 .ConfigureAwait(false);
}

     It is possible to execute multiple statements per command, splitting them with line breaks. Statements are pipelined to the same connection (but still they may be interpolated with other commands by Redis, use MULTI if you want to avoid it).

using (var channel = _client.CreateChannel())
{
    await channel.ExecuteAsync(@"
                  incr mykey
                  decr otherkey
                  subscribe topic")
                  .ConfigureAwait(false);
}

Read more about available options.

Read more about connection management.

Binding parameters

Although is possible to use RedisClient composing strings dynamically, it is unrecommended. Providing command templates will increase the performance since then the command execution plan can be cached.

Parameter binding works passing an object which properties will be bind to the command parameters, identified by a starting '@'. Only integral types, String, DateTime and their IEnumerable<> are supported. Commands should always start by a Redis command or a procedure alias.

using (var channel = _client.CreateChannel())
{
    await channel.ExecuteAsync(@"
                  incr @counterKey
                  set currentDateKey @date",
                  new { counterKey = "mycounter", date = DateTime.Now })
                  .ConfigureAwait(false);
}

Collections are added to the command as sequences. For example, it is possible to add multiple items with SADD:

using (var channel = _client.CreateChannel())
{
    await channel.ExecuteAsync("sadd @setKey @data",
                  new { setKey = "myset", data = new [] { "a", "b", "c" } })
                 .ConfigureAwait(false);
}

Object's properties, IEnumerable<Tuple<,>> and IEnumerable<KeyValuePair<,>> can be sequenced with the Parameter helper. This is handy for example saving objects as hashes:

using (var channel = _client.CreateChannel())
{
    await channel.ExecuteAsync("hmset myObject @data",
                  new { data = Parameter.SequenceProperties(myObjectInstance))
                 .ConfigureAwait(false);
}

Read more about parameter binding.

Getting results

A command execution result implements IRedisResults, which allows to inspect the return in every single statement of the command through a IRedisResultInspector per statement.

• Each statement correlates to a position in the IRedisResults items. First statement is item 0, and so on.
• .RedisType: indicates the result type.
• If the result is an error, accessing the statement result will throw a RedisClientCommandException with the details of the Redis error. It is possible to get the exception without throwing it using .GetException().
• .GetXXX methods: will try to read the value as XXX type, and will throw an RedisClientCastException if the data is not in the expected type.
• .AsXXX methods: will try to read the value as XXX type, or parse it as XXX (there is no .GetDouble() because Redis only returns string, integer or error, but there is a .AsDouble().
• .AsResults() method: will expand a single result as another collection of IRedisResultInspector. This is useful when a LUA script is returning an array of other Redis types.
• .AsObjectCollation<T>() allows to bind the result to an object by parsing a sequence of key-value pairs, and bind it to the object properties. For example member1 value1 member2 value2 will be bound as { member1 = "value1", member2 = "value2" }.
• .AsDictionaryCollation<TKey, TValue>() allows to bind the result to an object by parsing a sequence of key-value pairs as KeyValuePair<>.

using (var channel = _client.CreateChannel())
{
       var results = await channel.ExecuteAsync(@"
                                   incr mycounter
                                   hgetall @customer",
                                   new { customer = "customer:" + customerId} )
                                   .ConfigureAwait(false);
       var value = results[0].GetInteger();
       var obj = results[1].AsObjectCollation<Customer>();
}

Read more about getting results.

Subscribing to channels

IRedisChannel exposes a NotificationHandler property that can be used to get or set a handler for messages received by this channel. The handler will receive RedisNotification objects containing the message data.

using (var channel = Client.CreateChannel())
{
    channel.NotificationHandler = msg => Console.WriteLine(msg.Content); // will print 'whatever'
    channel.Execute("psubscribe h?llo");
    channel.Execute("publish hello whatever");
}

Note: You may feel tempted to put the SUBSCRIBE and PUBLISH statements in the same command, however it won't work because they will be executed in parallel in subscriber and commander connections respectively. Although technically possible to do, I considered this a very unlikely scenario, so in alas of better performace parallel execution is used.

Subscriptions are automatically cleared on IRedisChannel.Dispose(), so make sure you always dispose your channels.

Read more about subscribing to topics.

Executing procedures

Rather than executing LUA scripts directly, they need to be wrapped in what is called a procedure:

proc Sum(a, b)
    return a + b
endproc

Procedures are loaded in the configuration, and they are automatically deployed to Redis when connecting the first time. Multiple procedures can be uploaded from the same reader.

var options = new RedisClientOptions()
options
  .Procedures
  .Load(new StringReader(@"
     proc Sum(a, b)
       return a + b
     endproc"));

Then they can be invoked as normal Redis commands:

using (var channel = _client.CreateChannel())
{
    var result = await channel.ExecuteAsync("Sum 1 2")
                              .ConfigureAwait(false);
    var value = result[0].GetInteger();
}

Procedures accepts single and collection parameters:

• parameterName will expect a single value'.
• parameterName[] will expect one or more parameters. They are LUA arrays.

Also, parameters can be passed as keys to the script (important for clustering) using the $ prefix, either in single or collection parameters. The parameter (or parameters) will be passed in KEYS rather than in ARGV.

Quick example:

-- sums the content of a and stored the content
-- into the key specified in <asum>
proc sumAndStore($asum, a[])
   local function sum(t)
       local sum = 0
       for i=1, table.getn(t), 1 
       do 
          sum = sum + t[i]
       end
       return sum
   end
   local result = sum(a)
   return redis.call('set', asum, result)
endproc

Invoking:

using (var channel = _client.CreateChannel())
{
    var result = await channel.ExecuteAsync(@"
                               sumAndStore @key @values",
                               new { key = "mysum", values = new [] { 1, 2, 3} }
                               ).ConfigureAwait(false);
    result[0].AssertOK();
}

This will store the value 6 as string in the key mysum and will return OK. The value mysum is passed in KEYS rather than in ARGV.

Read more about procedures.

Read more about procedures management.

Read more about available options.