About the new INFO command, Diskstore and Cluster ETA

Monday, 24 January 11
With this blog post I'll try to keep you posted on the progresses of Redis, the recent changes to the INFO output, and the ETA for diskstore and cluster.

New INFO output

During the week end I managed to improve the INFO output that was rapidly becoming a mess ;) INFO is a very interesting command for Redis users, but there are a few issues that need to be addressed:

To address these problems what I did was:

All this will NOT be merged into 2.2, but is already part of Redis unstable.

An example of the new output:

redis> info
# Server

# Clients connected_clients:1 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0

# Memory used_memory:931360 used_memory_human:909.12K used_memory_rss:1085440 mem_fragmentation_ratio:1.17 use_tcmalloc:0

... more sections ...

It is possible to ask for specific sections. The following output shows one of the most interesting new sections:

redis> info commandstats
# Commandstats

(against my MBA 11", so typical servers will have smaller values)

I hope this will help us debugging better user problems, as for instance big set intersections or calls to KEYS will be easily detected. Note that in fast computers the 1 microsecond resolution provided by gettimeofday() may not be enough, but it's not a big problem, this is our equivalent of the MySQL "slow log". As long as slow operations are logged, this is fine. For all the rest there is the count of calls that is good to understand what fast operations are stressing the server.

As you can see the new format is almost backward compatible with the old one. There is just to handle empty lines and to filter lines starting with "#", that are now considered comments.

Diskstore and Cluster ETA

At the end of 2010 I was actively hacking again on Redis Cluster almost full time. Now the project is a bit delayed to be more aggressive developing diskstore brach.

Why, you may ask?

Because our logic was as follow:

Then we discovered that the Virtual Memory implementation is not as good as we want it to be. So I started working at diskstore, that is a much simpler project, already in alpha stage and working, in a public repository.

In a few weeks I'll start against with Redis Cluster, that will be Redis 3.0, while Redis 2.4 will provide a stable implementation of Diskstore.

In the meamtime Pieter Noordhuis is doing a awesome work in the area of client libraries. We were focusing too much on the server side, but clients are also a very important part, and a few clients can really benefit from some serious optimization Pieter is working on.

A final note. It's almost one year at this point that I and Pieter are working at Redis thanks to VMware. I want to say thank you to this awesome company that made all you saw about Redis in the latest year possible.

Edit: so what's the ETA? More or less 2 months for beta quality (RC) diskstore, and at least six months for beta quality cluster. This are a bit optimistic forecasts, but after all in a few weeks we'll not have to work too much at 2.2 as it will ship stable, so there will be more time for the unstable branches.
Posted at 10:15:15 | permalink | 4 comments | print