Redis for win32 and the Microsoft patch

Redis for win32 and the Microsoft patch Friday, 09 December 11

A few days ago Microsoft released a patch to compile Redis under win32. The team working on this project used the already existing win32/win64 port as a reference, and used the libuv library that powers the node.js project.

Yesterday the story hit Hacker News as I discovered a few hours later (I was away from the computer since December 8th is an holiday here in Italy), and as usually when you mix Microsoft, Open Source, and a news site, the result is some friction inside comments.

I decided to write this blog post to clarify my opinion on the matter, and it is funny how this blog post will delay another that I've already written and I was ready to publish today, called "We are programmers, we need a revolution", that in some way is related to what I want for the Redis future... but you'll see that post in a few days if you are interested. Now back to the win32 patch.

How good the patch is

What the Microsoft team working at the patch did was to port Redis to libuv that is, mainly, a library for evented programming based on libev, but cross platform. Actually libuv is ending as a container of many useful programming tools to interact with the operating system that are usually different between POSIX and WIN32. It was developed for the node.js project in order to, eventually, contain every difference between POSIX and WIN32 inside a unique library.

Persistence was not properly addressed yet, but apparently the next version of the patch should handle it better. However currently the persistence is basically unusable since it blocks the main thread while Redis is saving a snapshot. There are also intermitting problems passing the test, it is not clear to the authors of the patch if it is due to the testing engine itself or to actual issues in the patch.

But in short: the patch is exactly as functional as it was the native win32/win64 port that was already provided by dmajkic: a port good enough to develop under Windows without the need of running Redis under a virtualized Linux install (not a big effort btw, in my opinion), but that was not good enough to use Redis in production under win32 systems.

It is worth to note that the native port operated by dmajkic and not using libuv has the remarkable advantage of just adding the minimal set of changes in order to port Redis to win32, it implements a new win32 backend in the event library we use, ae.c, that proved to be a very stable and performant component in our stack in the latest two years. So it was a lot more compact, and I see this as an advantage.

Patches or pull requests?

Microsoft was criticized for not sending a pull request, but a patch... I think here the point is that they provided some code: send a pull request, send a patch, or an email, it's the same and IMHO there is very little point in this formal things. But I think that in this specific instance the patch into a gist was the right way to contribute, since the patch was huge and since in the past I stated many times that I don't want to add win32 support directly in the Redis main project, but I'll favor the creation of a satellite "Redis-win32" project that is separated from the main project.

Also note that in our contributing guidelines we state that it is better to talk with me or Pieter before going forward with the development of significant code, after all I say no many times, so why wasting efforts? But Microsoft did not informed me simply because this was a project they wanted to do anyway I guess, so sending a patch is appropriate even more. However I was informed by email about the fact a patch would be published in 24 hours, and I appreciated it.

In short: Microsoft behavior as an OSS contributor in this case is fine from my point of view.

Why I'll not accept this patch

I don't think Redis running under win32 is a very important feature. It is cool to have a win32 port that can be used for testing, as we had before, and as we have in a different implementation thanks to the Microsoft patch, so developers using Windows can easily test Redis and develop their projects. But what is the point in providing a production quality win32 port?

I think that Linux completely won as a platform to deploy software, and even if you want to run your code under win32 systems what's wrong about installing Linux boxes to run Redis? For instance Stack Overflow runs their systems in a mix of Windows and Linux boxes, they have no troubles into using Linux to run Redis.

Instead handling a win32 port directly in the main project means to delay everything else for the little gain of having, eventually, a production ready win32 port of Redis. In Redis we use a lot subtle things about the operating system, from copy on write to the time needed to fork a process, to the way operating systems overcommit memory. If we add a new platform, in the future, exploiting the OS to do the best for our users will get harder and harder. It is completely not the case.

However I like the idea of a win32 port as a separated project, with a different set of developers, and not officially supported by the main project. That is just added value, and can provide a more reasonable port for development, or even for production at some point, without impacting the main project: so fork the code, and have fun. I'll help if needed, ask me questions, let's collaborate on general ideas. I'll also put a page about the win32 port in the redis.io site so that users will be aware of the port.

Let's merge just for libuv?

In the latest days I also heard that is a good idea to switch to libuv in general, win32 port or not. I beg to differ.

To start the node.js project is using libuv since they are interested in multi platform code able to run under Win32 and POSIX. Otherwise libuv from the point of view of what Redis uses of an evented library does not offer new interesting things (we just use file events and timers with our ae.c library). If there is some interesting abstraction that we'll need to use in the future, like streams, we'll implement it in ae.c, but as far as I can tell we will NOT have that need.

Also, I've an argument that for me is truly important:

$ wc -l ae*.[ch]
     397 ae.c
     118 ae.h
      94 ae_epoll.c
      96 ae_kqueue.c
      72 ae_select.c
     777 total

What the above means is: ability to resolve any possible bug with our events or timers in no time, instead of trying to understand a much bigger multi platform code.

I avoid dependencies, but when dependencies are needed I don't have problems with them. Redis includes a full Lua interpreter and the jemalloc allocator. it was idiotic to provide my implementation of a programming language, or to rewrite an allocator that works as well as jemalloc works. When dependencies provide a lot of added value it is worth adding them. Instead when you need to switch to something bigger and more complex without any gain, why to do it?

Ah, and about the gain being some kind of feature only exciting for we code nerds and having zero effects on how a system works, please read my next article in a few days, we are programmers and we need a revolution.

home