Comments for post An update on the Memcached/Redis benchmark
Vivek Munagala writes: Awesome post. It helped me understand many concepts of redis.
Kim, different instances of redis do not share the memory. So if you set a key in one instance, it will not be seen from the 2nd instance. And I dont understand why you need 16 times the memory if you have 16 instances. Obviously you need to distribute your keys among all the instances instead of replicating. As simple as a checksum of key can help you in distributing.
Kim writes: I have a question about synchronization. Are the two instances of redis synchronized, so that if you do a SET on one process, the other process is updated as well? If not, that is an argument for a threaded model where you ensure consistency.
Memory requirements are also another concern. If you want to use your 16-core CPU, you would need 16 times the memory with separate processes.
Adam Malter writes: Hi - thanks for the great data.
We are a shop contemplating Redis for persistent k/v stores and this causes me a bit of concern at the app server level.
For example, we currently have 4 memcache boxes, probably on dual quad core instances. If we were to supplement this with a similar Redis cluster; 8 instances per machine times 4 current machines - so 24 connections per app server.
Now, our app servers are java and using the Spy memcache client, we found that best performance was achieved when we had 2 or 3 connections per server.
Would this mean that with our starting cluster you might advise to have 50-75 connections (which translate to threads on the java side)
This connection scaling worries me, even if the instance to app server ratio is 1:1. With a growing cluster, growing number of cores per box, well, things get exponential quickly...
I am very new to evaluating Redis and might have some stuff backwards or might be over-worrying on connection count. Could you speak to what configurations are recommended.
Jeo writes: All this is especially true given virtual machines.
dean writes: Good stuff. Can't wait for Redis Cluster!
Thanks for your devotion to this cool project!
Louie writes: Great work. I appreciate the quick responses to community discussions and benchmarks. It really demonstrates your commitment to the project which in turn makes it easier for all of us to adopt Redis.
anon writes: Can you point to a guide showing how to implement redis most-efficiently on a 16-core machine ?
Also, do you know of any similar benchmarks against other key-value storage facilities ?
Like Luxio, tokyo cabinet by itself or through kumofs, Tx, or even the new BlitzDB merged into Drizzle not long ago ?
Also, the redis own benchmark is quite simple. Can it be updated with more complex tasks, inserting different random values into different random places, mixing PUTs and GETs, varying inserted data size, and basically checking both the average case and the worst case?
Links to projects:
Tx (a bit off-topic, but as you showed text search, can compare with that): http://code.google.com/p/tx-trie/
Luxio: http://luxio.sourceforge.net/
Tokyo cabinet: http://fallabs.com/tokyocabinet/
Kumofs: http://kumofs.sourceforge.net/
Flare: http://labs.gree.jp/Top/OpenSource/Flare-en.html
Kai: http://sourceforge.net/apps/mediawiki/kai/index.php?title=Main_Page
home