30 Jan 2012

Thoughts on Socket.IO

I have been hacking away with Socket.IO for the past few months. My initial goal was to explore how Node.js/Socket.IO scales when you need to support a largish audience, when compared to my previous solutions involving Erlang/OTP and a handmade COMET library.

I have built these COMET based streaming solutions by hand a few years ago. Back then, there was no HTML5 or websockets, and most off-the-shelf COMET “solutions” were bloated Java-based solutions. We had to roll out our own client side COMET library, because there were no standalone or decoupled open source alternatives. And, boy was it hard, especially when it came to supporting support IE (as always). I mention some of the elaborate client side techniques involved on these slides. The guys at LearnBoost have not only come up with a completely transparent solution that degrades gracefully all the way down to IE 5.5, but have also decoupled the client and server side components. There are third-party server-side bindings for plenty of other languages too. Having said that, the Erlang implementation (and quite a few other languages as well) seems to be supporting only version 0.6.x of Socket.IO. It looks like it will remain that way for a while, as Socket.IO v0.8 is almost a complete rewrite and breaks backward compatibility.

I wrote a simple application using Node.js and Socket.IO that covered most of the basic “features” you would expect in a typical chat/pubsub system. This involves stuff like: detecting online/offline status of users, sending messages to a single user, broadcasting messages to everyone in a group/room, handling users who use multiple tabs etc. The code is up on GitHub.

If you’re not already familiar with Socket.IO, I urge you to check it out, because I couldn’t believe that I ended up doing all of that in just about 100 lines of code – client and server-side combined! I have always loved JavaScript, but the absolute joy of sharing the same paradigm and code conventions across both the client-side and server-side is just amazing.

Socket.IO provides a lot of things you need out of the box. Consider for example, sending a message to a group of people. Socket.IO allows you to make a user join a channel/room and once you do that, it also allows you to send messages to only that specific channel:

// make the user's socket join a room called "foo"
// send message only to "foo"
sio.sockets.in('foo').emit('new_message', {'message': 'hello', 'from': 'bar'});

Want to send a message to everyone else, except the current user (say, when “Jack” comes online)?

socket.broadcast.emit('user.online', {'username': 'jack'});

To do these by hand, it would involve writing boiler-plate code that will deal with storing the user sockets in a map, iterating over it etc. which Socket.IO handles for you.

What about scaling?

Of course, we all need planet scale, don’t we? The single threaded nature of Node.js does limit the default Socket.IO set-up, which in my crude benchmarks seemed to be a few thousand moderately active connections. If you are interested in seeing actual benchmark numbers, Drew Harry has an excellent write-up on that.

To get around the limitation imposed by a single core, you can load balance multiple node processes with node-http-proxy or HA Proxy. Unfortunately, we can’t use Nginx for load-balancing as the current stable version does not support reverse-proxying of HTTP/1.1 connections used by Websockets.

Alternatively, you can use Redis pubsub to allow you to scale past a single process. But even that has some limitations due to the current chatty nature of Socket.IO. I’m sure that this is already something that the LearnBoost guys are looking to address.

Another idea which I had which I have not explored fully is to use Node’s cluster module and its message passing API to synchronize between multiple Socket.IO processes.

It should be noted here that 3-4k concurrent users is still more than sufficient for a lot of web applications, and when you are looking to support hundreds of thousands of users, you are looking at a different problem space altogether. The approach to take also depends on the specific needs of your application. Socket.IO with Node.js offers a ridiculously simple way to get off the blocks for building web applications with soft real-time elements, but I wish it was as simple to scale it to multiple cores or machines. For example, in our Erlang/OTP solution, we had the ability to scale past multiple machines as Erlang handled that layer of abstraction for us very well. So, going forward, I really wish Node would allow robust process synchronization and message passing. Perhaps the current work being done on isolates would allow Node applications to scale past a single core and perhaps even machine.

If you have something interesting for me to work on in Node, shoot me an email.

11 people have responded to this post.

Leave a Reply