http://stackoverflow.com/questions/9107384/server-scalability-html-5-websockets-vs-comet

There is lots of talk about compatibility, support and issues with WebSocket which is why most people probably won’t use it directly, but instead use one of the many products or open source projects that wrap up WebSocket and provide some fallback mechanisms for the environments where WebSocket will not work. Since there is an API on the client side it is easy to make these wrapper APIs conform, therefore it is easy to swap it out for other products or remove completely if there is ever a time where WebSocket is ubiquitous.

The problem is, on the server side, there is no API, just a protocol specification.

Existing server side APIs

Years ago there was CGI, a defined interface for writing dynamic content from a web server, this was used by shell scripts, perl scripts, compiled binaries, and these things were fairly portable – at least the portability was in your hands, for example, if you used perl you had to make sure you had perl in your environment. Next came more advanced APIs, Java Servlets, Microsoft’s ISAPI. Servlets, more so than ISAPI maybe, meant you could write Java based server side web applications that could run on any web server or application server that implements the Servlet API.

WebSocket on the server

As simple as WebSocket is to use on the client side, the lack of support for it and therefore the necessary fallback mechanisms prevent it from being usable on it’s own. This leads it to only really being useful within specialist products such as Comet servers which have their own APIs on client and server (Some may expose a WebSocket like API on the client). In these scenarios you haven’t really opened up any new possibilities, Comet servers have been doing this without WebSocket for years, WebSocket just brings a new transport mechanism which in some areas is better, but ultimately the user won’t notice.

Writing a WebSocket application ties you to a particular server side implementation of WebSocket. There is no API to write to other than proprietary ones, and since the whole paradigm of WebSocket (bidirectional communication) does not fit in with traditional web request/response based APIs, such as Servlets, you cannot really write a WebSocket application using existing server APIs. From early discussions on the WebSocket mailing list, it was clear the authors had the goal to make implementing WebSocket so simple that this would not be an issue, as you could just simply implement WebSocket each time you wrote a WebSocket application. To me this seems a bit naive – people have to fit in with existing infrastructure and since WebSocket is not ubiquitous you need fallback mechanisms which are not trivial to implement from scratch each time.

10 Add clients that subscribe to X subjects.
20 Once subscriptions are done, measure the latency of all messages for 30 seconds
30 GOTO 10

This means we are measuring the steady state of updates, while no other activity is going on.

We have soak tests that do a whole lot more than that, but we’re talking about benchmarks here.

It is easy to come up with more real world scenarios. For example, clients unsubscribing and subscribing to other data, clients sending messages into the server, and using different features of the platform. The purpose of a benchmark is to have measurable attributes, for the standard benchmarks this is clear – latency, cpu, numbers of users etc. It is also easy to understand – ‘Liberator can support 100,000 clients receiving 1 message/sec at under 10ms latency’ is is something you can compare to other products (with a few more details like message size clarified).

The problem with the real world scenarios is they are less comparable, less meaningful – ‘Liberator can support X clients subscribing and unsubscribing every minute, while sending messages to the server every 30 seconds…’ – it becomes fairly arbitrary. The standard benchmarks are the kind of numbers you can ask a customer for – how many clients will you have? how fast does your data update? With the real world examples, try asking questions and see what you get – how often will people switch views (change subscriptions)? how often will users trade? You will get answers that are vague at best.

Nevertheless, I have been doing some benchmarks along these lines to see how various commonly used features impact the performance of Liberator and the platform.

Transformer is a server that sits behind Liberator and allows modules to be implemented to transform the data coming from sources. So I added Transformer into the chain in the benchmark environment. Tests include using each of the three types of module, C, Java and LUA. The modules themselves do very little apart from passing the message through, so it is really only testing the bridge between languages. Results are good, with very little latency added at all.

Trading. Clients have to send messages to Liberator to trade, so I added some scenarios that include client to server messages, being echoed back and round trip latencies recorded, as well as looking at the usual update latency of other data.

Containers. Liberator has the concept of a container. A meta object that can be subscribed to with a single command. A container is a list of references to other data, which you get subscribed to by the server. This is very convenient for long lists of data in grids, such as a list of bonds. It is also possible to only subscribe to a portion of a container, allowing the client to only receive what is visible on screen and not anything scrolled off screen. Items can be added and removed from containers with the clients subscriptions updated accordingly. This is how many clients subscribe to the majority of what is on their screen, so scenarios where subscriptions are made this way are quite important.

I will probably publish some actual graphs soon, but final results are still being run.

Althought the focus for Comet is on server to client push, most Comet solutions provide an API to allow client to server push within the Comet environment – rather than forcing you to send client to server messages over HTTP directly.

The ‘Web’ way

The problem is many people are still stuck in the ‘web’ way of doing things and they only come to Comet when they need server push.

The problem with this is it leads to a circular architecture. Somewhere on the server side lives some application logic, sending messages and receiving messages, and probably doing something to them in between. Before the web, or if the web isn’t a requirement, people would use communication APIs that allow you to send and receive – from low level socket APIs to higher level messaging – but you send and receive using the same API.

With some web applications, messages are sent to an Application server using standard web technology, this is then passed on to another server process which may feed into the back (or side?) of the Comet server, which sends messages using Comet to clients. The circular architecture.

The Comet way

If you are using a Comet solution, make full use of it. A single API on the client, and a single API on the server.

What benefit do you actually get?

I think I have covered the benefits of using a single API, but there is more. Liberator, and some other Comet servers, provides a security API to control access to the system and granular access to specific data. When using the Comet server for data in both directions this aspect of controlling data access is contained in one place. With a circular architecture, subscribing to data needs to be controlled by the Comet server, but sending data needs to be controlled by the application server. It’s quite clear that this situation is not a good thing.

What about efficiency?

At the moment, most Comet solutions will be using new HTTP requests for each client to server message – so technically it isn’t much different to the circular architecture (as far as the network is concerned). What about the future? I have blogged about WebSocket before, but if/when WebSocket does become more widespread using Comet servers for bidirectional messages will be more efficient, since client to server messages do not have the overhead of HTTP requests.

So take a step into the future, it isn’t all about posting requests to application servers and all the constraints of the request/response stateless nature of the web. Comet gives you a much more natural way to communicate!

Some of my blogs relate to Liberator, but I don’t often talk about it directly, explain the different versions or describe what you can do with it.

Caplin Liberator is a commercial product that we sell, usually as part of a larger framework called Caplin Xaqua. Liberator was first developed in 2000 and has been used in many large deployments since then.

A few years ago we decided to release Liberator FE, this is a free edition of Caplin Liberator.

So what can you do with Liberator FE?

The short answer is ‘pretty much everything you can do with the commercial version’, but what is that and what are the differences?

Liberator FE comes with two APIs. Most people familiar with Comet will understand the idea of a JavaScript library that runs in the browser talking to a Comet server and exchanging information – Liberator of course provides this, it is called StreamLink for Browsers. Not all Comet servers provide server side integration APIs though. For some the model is ‘everyone is a client’, so even if something is conceptually a backend component it might still use the same API as the clients. Liberator has the concept of DataSources, as the name suggests these components provide the data. Clients can also provide data, but it is treated differently. Liberator FE provides a Java DataSource API.

So you might take the Java DataSource API and write an application that feeds data into Liberator that is then available for clients. The client application would be a web page using the StreamLink for Browsers JavaScript API to subscribe to the data that the DataSource is publishing. It will receive data in real time, as soon as it is published, and clients can also publish data back to the server – it is fully bidirectional.

What kind of applications can you develop? How long is a piece of string? You can use your imagination.. Caplin’s main focus is financial trading applications, but anything that could benefit from real time data or highly interactive experiences can be developed. Sports scores, chat and other social media applications, real time monitoring, games, whatever you can think of! We’d love to hear of other ideas.

So what can’t you do?

Liberator FE is a full blown version of Caplin Liberator, performance, scalability etc, it is all the same. However, you don’t get other language APIs, for example, .Net, Silverlight, C/C++. You don’t get two further integration APIs, for handling realtime permissioning and monitoring.

You also cannot use Liberator FE for a commercial application – it is against the license.

Liberator FE has been downloaded by a lot of people and we hope this is a good way for people to get familiar with Liberator and Comet technology – whether it is for a non-commercial project or as a way of easily evaluating Liberator for a commercial project.

So if you want to try out Comet, if you are interested in some of the things I blog about and want to try it out, Liberator FE is a good place to start! Let us know how you get on.

Here are the links:

London Ajax User Group: Comet Panel – 13/07/2010 from Skills Matter on Vimeo.

Video/Slides: http://skillsmatter.com/podcast/ajax-ria/comet-panel/zx-486

Photos: http://flickriver.com/photos/dylans/sets/72157624493006656/

Now you can see what a bunch of Comet vendor geeks look like!

Thanks again to Dylan and Skillsmatter for hosting the event.

Thanks to Dylan for organising the event, and thanks to the other panelists for taking part. It was good to finally meet Alessandro from Lightstreamer, and its good to see the various open projects talking about their technology too. A couple of guys from Nirvana were in the audience and we had a good chat down the pub afterwards.

Everyone on the panel seemed to have similar views on WebSocket, although some have implemented it and some have not.

I’ll post a link to the video and slides when they are available soon.