What is the “real-time web”?

Phil Leggetter — Mon, 20 Apr 2009 11:00:54 +0000

Are we really seeing the real-time web?

The phrase “real-time web” has been streaming its way around the Internet for a while now. It’s presently being used to describe information being available in search results as soon as it has been published by its author. Examples of this are Twitter or FriendFeed search.

As far as I’m aware it was Robert Scoble who made this phrase mainstream. Here’s an example from the start of February, but by this time Robert had been using the term for a while and a quick Google finds the term used as early as June 2008:

Let’s do a search for anyone who has written about the Canon 5D MK II but lets constrain that to posts that have at least one like and at least four comments. Here’s the search. Note that the post I wrote just one minute ago is already in the results page. This is the real-time web. – Robert Scoble (http://scobleizer.com/2009/02/09/is-the-real-time-web-a-threat-to-google-search/)

Whilst this is an example of something being almost instantly available from search does it really qualify as being “real-time”?

The first thing to do, as others have done, is understand what the term “real-time” means. Rather than going into this I recommend you have a read of the wikipedia definition of real-time and watch a Kaazing presentation on What is the Real-Time Web. There have been fewer attempts at defining “real-time web”. Even Wikipedia presently (19/04/2009) lacks a definition.

For me for something to be real-time web it should meet the following criteria:

Get up to the second information on a chosen subject.
Make an initial request for information to be delivered across the Internet. Subsequent updates to that information, or information stream, should be delivered as soon as possible without the need to make a additional requests.
Initial information and subsequent updates should be delivered in an acceptable amount of time.

The real-time web experience

If I go to my twitter homepage I don’t want to have to refresh the page, by clicking the refresh button, or hitting the F5 key, in order to see new messages from the people that I’m following. Twitter doesn’t really want users doing this either as it puts load on their server – but let’s leave this for another time.

Twitter

This lack of real-time updates on the Twitter site has lead to a surge in Twitter client applications such as TweetDeck and Thwirl. These clients simulate real-time by polling the Twitter API at intervals. I’d like to highlight the word “simulate” again and clarify what polling is. What the Twitter client applications are actually doing is making requests to Twitter at periodic intervals to see if the user has had any messages and if they have it displays them, if there are no new messages then it’s a wasted request.

TweetDeck

Twhirl

A much better model would be for Twitter to notify the client that new messages are available by passing the additional messages straight to the Twitter client.

Real-time web delivery time

What determines the amount of time that is acceptable between a message being published and it being received by the consumer (user) depends on the context of its use. For something such as instant messaging or a Twitter conversation the time between publication and consumption can be reasonably large, maybe 10 seconds. However, if the publisher is a trading system sending a price and the consumer is a trader 10 seconds is far too long. By the time the price reaches the trader it may be out of date.

Real-time technology

XMPP and SUP have been touted as the technology to deliver the real-time web but they appear to be server side solutions or require clients with knowledge of the specific technologies. For a web solution people start to turn to Comet server solutions. Caplin Systems – the real-time web company – have had the technology available to deliver real-time data for many years now so I wanted to understand why, only now, has the idea of a “real-time web” really taken off.

Caplin Systems

Caplin Systems have a high performance scalable real-time comet server called Liberator that can push updates to subscribing clients as soon as it receives information. The clients are:

Java clients: standalone application or applet
.NET clients such a windows desktop application or web page embedded SilverLight widget
JavaScript clients: any website such as http://twitter.com or http://friendfeed.com

Twitter and FriendFeed don’t use Liberator but they could easily do so. All that needs to be done to integrate Real-time Rich Internet Application (RTRIA ?) functionality to an existing web site is include a JavaScript library called SL4B (StreamLink for Browsers). The RTRIA functionality can then be added to an existing site in the same way that RIA functionality can be introduced to a website using other JavaScript libraries such as JQuery. However, in this case the RIA functionality that can be added is a true real-time web experience of data being pushed to the client as soon as it becomes available, without any need for the user to check to see if any new information is available.

Conclusion

My opinion is that the instant availability of information via search straight after it’s published doesn’t really justify the term “real-time web”. To be truely real-time the information needs to be pushed to the user. The true real-time web experience and the technology to achieve this is already here but at the moment few people are aware of it and therefore it’s not in mainstream use. The ideas behind Comet, the Liberator server and Liberator clients provide technology and infrastructure to push data to clients without the need for polling or a user driven information request. Scalability and side by side integration with existing systems are already possible which means there’s no reason why existing clients such as websites can’t start taking advantage of it.