I don’t think any of your points are wrong, but the blog was not meant to cover everything, it would need a whole series of blogs to do that.

I think from your list I would pick out failover, latency and authentication/entitlements (although I thought I covered this, reading it back it doesn’t make it clear I was talking about more than just SSO)

Finally on the subject of performance, in our experience clients pay a lot of attention to our vendor published results and many also want to run their own benchmarks (mainly because scenarios can differ so much) for which we provide a set of tools to do so with.

Nirvana ( http://www.my-channels.com/ ) supports Comet ( and the WebSocket protocol )for our Javascript browser based clients however we do so much more than Comet for all of our other supported Enterprise, Web and Mobile clients.

While the 9 questions Paul defines are relevant they are far from complete and although integration is mentioned in the prequel to the list it is subsequently glossed over. To establish a more complete list of discussion points to aid these types of technology decision one might also include:

- Support for clustering and failover approaches this enabling compliance with business contingency policies.
- Support for standards. Things like HTML5 WebSocket support, AMQP, JMS etc.
- Support for integration with 3rd party technologies. Not just back end but client side integration for end to end B2B type solutions
- Support for integration with existing enterprise management technologies. Measuring end to end latency is only possible if the underlying technologies used supports such functionality and expose it in administration and management API’s and tools.
- Security, not just SSO but an understanding of authentication and entitlements and how such functionality can be leveraged in client implementations
- Configuration management, both client and server. Snapshotting production environments, being able to roll back configurations and quickly deploy new instances.
- Support for different messaging paradigms. Not everything fits into a publish/subscribe model. Some interactions require point to point or queue based communications.
- Support for different deployment topologies, i.e broker and brokerless based servers, caching severs and server federation.

Finally on the subject of performance in our experience clients pay little attention to vendor published results and always understandably insist on establishing what is possible in their respective environments; typically via a vendor bake off. To date Nirvana has proven to be very successful in these real world test case scenarios and consequently is used today by some of the worlds leading SDP implementations.

Perhaps market share and implementation base should also be included in the questions to be asked !

Maybe some kind of test and hardness spec could be written and vendors submit their own tools do the vendor specific stuff.

StreamHub has been on my radar, but they haven’t really come up much yet.

I’d welcome more vendors to publish performance results, as mentioned we’re just running some new ones ourselves at the moment, as are Adobe.

I’d also be interested if Lab49 want to collaborate on running some tests, it can be time consuming though!