Here are the links:

London Ajax User Group: Comet Panel – 13/07/2010 from Skills Matter on Vimeo.

Video/Slides: http://skillsmatter.com/podcast/ajax-ria/comet-panel/zx-486

Photos: http://flickriver.com/photos/dylans/sets/72157624493006656/

Now you can see what a bunch of Comet vendor geeks look like!

Thanks again to Dylan and Skillsmatter for hosting the event.

Early on in CometDaily’s lifetime I wrote a piece on the Evolution of Comet at Caplin which covers how Caplin’s core technology has evolved since the start in 1997. The article was written in 2007, so I thought I would say how things have moved on since then.

The article finishes off talking about Caplin Trader our Ajax trading front end framework. This is still a large focus for Caplin, but we have also expanded out. There is a lot of extra functionality on the backend that was developed along with Caplin Trader – higher level integration to Trading and Permissions for example. Although Caplin Trader allows you to host other RIA technologies within it, we also wanted to more openly support other client side technologies in their own right. So we worked on these APIs to create Caplin Xaqua which is the full stack of our software, from backend integration APIs, through Liberator (and Comet) out to our StreamLink client APIs, which we expanded to include .Net, Silverlight and Flex.

Coming back to Comet, with the new client APIs and also new browsers and browser versions, we improved our coverage of Comet techniques to ensure the best possible connection is made for each scenario.

And the future? Well I have blogged about HTML 5 WebSockets more than once and to reiterate, it is a good new tool for Comet server implementers (rather than people developing web applications themselves) and i’m sure when the market share for browsers supporting WebSocket grows we will be adding the capability to Liberator and Streamlink.

There has been lots of buzz and some really interesting bits coming out so far including Silverlight 4 RC, Internet Explorer 9 preview (with HTML5 and CSS3 etc.), Windows phone information, etc.

Scott Hanselman has posted a great roundup of the announcements and keynotes.

Rumour on the street has it that, finally, the investment banks are going to start abandoning IE6. Barclays are allegedly going to do an enterprise-wide rollout of IE8 in Q2 of this year (put back from Q3 last year). Even more surprising (and impressive) is a rumour that Standard Bank of South Africa are going to ditch Microsoft browsers completely and move to Chrome for their corporate standard!

The idea of WebSocket is sound – it should get past some of the limitations of the various Comet techniques used today on various real time websites. It gives you full duplex communication between a browser and a server – Comet fakes this somewhat, but in most cases it is sufficient.

Some people seem to think that WebSocket is going to mean no more Comet servers and no more frameworks for real time data. But as Greg points out, WebSocket just doesn’t give you enough.

I see three main issues with WebSocket, or rather its uptake by web developers.

1. Browser support

We still have to deal with IE6 at the moment and WebSocket is not supported yet in the current browsers (apart from Chrome). So when can a web developer realistically rely on WebSocket being available for the majority of his users? This is likely to remain an issue for a long time.

2. Proxy support

WebSocket through proxies are a fairly unknown quantity – WebSocket is designed to fail cleanly so falling back to another mechanism is possible. However we don’t really know how many proxies will work with WebSocket. This includes server side proxies, which are often overlooked and are usually out of the control of the server side developer.

3. Features

What I am referring to here are the low level features that Greg talks about. His blog demonstrates that you cannot just take the simple WebSocket API and write a real time application (other than a demo) without taking into account the kind of edge cases he describes. Without this a web application could appear unstable for many users.

Taking all this into account, what is WebSocket good for? Who should use it?

Conclusion

Application developers use frameworks that hide issues and allow them to concentrate on their application, not the infrastructure. Frameworks can handle the error cases and implement various fallback methods for when WebSocket doesn’t work or an old browser is being used – there are free and commercial frameworks offering this.

So in this sense while Websocket will not improve web applications directly, it is something that framework developers can use to improve their frameworks – consequently improving those web applications that use the frameworks.

Cricket is played all round the world most of the year round, which means there are often games happening while you are sitting at your desk at work – or maybe you don’t have Sky Sports and have to follow the scores online even when at home.

So what do most people do? You sit there working while one of your browser tabs is open on http://www.cricinfo.com updating you with the scores. There are other sites too, but from my experience CricInfo’s ball by ball approach gives me the most information.

This is fine when England play Bangladesh, or even for an Ashes match against the old foe Australia for most of the game. However, when it gets to a crucial part of the game – which happened a number of times in the classic 2005 Ashes series or when India play Pakistan (both having a huge number of supporters) – CricInfo and many other websites grind to a halt. In fact, due to CricInfo’s multi server caching setup sometimes the page reloads and has gone back in time!

That’s just not cricket!

Back to today’s game, India are playing South Africa in a 50 over game, two of the top sides in world cricket. Being English I didn’t have CricInfo open for this game, but when, on Twitter, I heard noises that Tendulkar was doing something a bit special I opened CricInfo and saw he was on 193, 1 run from the record and 7 runs from a milestone 200!

I clicked on ‘Full Scorecard’ to see more details of the innings, it took a while to load and when it came up it said he was on 139! I went back and no, he was on 196. The live page then updated and he was on 198, it updated again and he was on 196. It got to the point where he was on 199 needing just 1 run and about 2 overs to go and nothing happened. Cricket fans all over the world were madly clicking refresh, but every time it reloaded it was the same.

I went back to Twitter, searched on #cricket and all was revealed, probably within a couple of seconds of the actual event happening.

So what is the problem here? I wanted to see the rich, ball by ball, information on CricInfo, but the service could not cope. Twitter, and other similar services are designed to cope with this kind of load. CricInfo and other sports sites are not, but could they be?

The Real-time Web

There are certainly a lot of things they could do to help that don’t involve buying more servers and more bandwidth. CricInfo’s page reloads a 25k frame every 20 seconds – a ball is bowled roughly every 40 seconds in a cricket match. How much data on that page changes with each ball? 250 bytes maybe or a bit more at the end of each over, let’s be generous and call it 512 bytes, ie 1/50th of the whole page reloading. Can you see where I am going here?

The Real-time Web is a term banded about regarding lots of things these days, to me it means live data. That means you see new data when it happens, not the next time you check, not the next time the browser checks for you – that isn’t real-time.

It may be better than old style search engines that had to spend hours or days crawling the web to find new content before it was available in a search, but it still isn’t real-time until you are updated automatically.

When were you when Tendulkar scored 200? Unfortunately, I was a few minutes behind!

From a business point of view the conversation gets interesting when Kevin Bourne asks “do you not want to see a point where people want 100% of the choice but all out of the can with a library of applications that they can then tailor much more specifically to what they want to deliver to their clients?”

You can see the other highlight videos on the Caplin Systems You Tube Channel.

You can also download a full transcript of the event:
Buyers Market – Single Dealer Portals and Courting the Online Client

Back when Alex Russell coined the term Comet, he talked in terms of what it was possible for browsers of the time to do – holding open HTTP connections, but it’s obvious from his post that he was thinking of any of a family of techniques for pushing data to the web browser.

Here at Caplin, part of our Xaqua platform is a world leading comet server called Liberator. According to Joe, that part of our offering is about to be outdone by websockets. In actual fact, we’ve been waiting and hoping for websockets for a long time, not because they will replace comet, but because they will make the comet experience better. When we first played with a websocket implementation (Kaazing’s websocket server), we were excited about using them but since there had been no indication back then that any browser was going to support them natively, we held off.

Our streaming comet server, Liberator accepts connections from Flash, .Net, Java clients and native clients, but it can also use a plethora of different techniques to allow web browsers to connect, from applets with direct socket connections (hardly used at all now), to long polling, multipart-replace techniques, or simple long running XHRs or forever frames (many of these techniques are described here). There are others too – server sent events, etc. Each of these techniques has different benefits, different firewall traversal characteristics and many of them only work in a subset of the browsers we support. As with any well architected Comet server, different techniques can be plugged in, new ones can be created easily, and the best one for any situation can be selected automatically for establishing the connection.

Websockets will not change any of this. It will provide yet another substrate option for our Comet connection to run over, one that will have some benefits over many of the other techniques, and once we have implemented it, it will be automatically used where appropriate. The fact that it’s so far only supported by development versions of Chrome will mean that that will be less than 10% of the time, but we can hope that more browsers will follow suit.

Nor is providing a connection the full extent of any Comet server worth its salt. Liberator provides robust failover when the connection goes down, session management, and a full pub/sub engine and javascript API.

The key to Comet is that the normal interaction between browser and server is turned on its head; the browser gets told about events as and when they occur, rather than having to specifically ask (and this is a vital ingredient for the real time web). Comet makes no demands that we use a particular method to achieve this, and we have a huge armoury that we will continue expanding as browsers advance.

Far from heralding the death of Comet, websockets will make Comet faster, easier and simpler.