Discovering the Joy of DM-OPEN

The agencies involved in the demo, and many technology vendors, worked closely over the month or so prior to the event to iron out challenges that cropped up in what should have been a straight-forward integration and compatibility exercise. The demonstration relied on the FEMA Disaster Management OPEN (DM-OPEN) communications tools. DM-OPEN is not a pub/sub system, nor is it really even message-oriented middleware. It’s a database that senders update and receivers poll for messages. Programs request all messages sent in the past X milliseconds, so even once a message is consumed, it will still be there the next time you poll. Because of indices on the database, you can’t send the same document twice – every new message needs a unique timestamp and sender ID.

There’s also a problem with how it handles time zones, such that applications need to poll by a timestamp in the message, instead of “sent time” on the server. This in particular led to additional implementation problems. There were inconsistencies in how participants were writing timestamps, which made matching events a challenge. For example, one vendor used a 12 hour format while the rest were looking for 24 hour format, and another’s messages were being completely left behind because of syntax problems.

In the end, lots of high touch communications and coordination paid off. Normally you don’t want your demo to be called a train wreck, but in this case, the blaring sirens and blinking lights meant the scenario had come off without a hitch.

Hard and cumbersome doesn’t cut it for emergency response

It’s one thing to get a bunch of smart people on the phone from the vendor and agency community twice a week to iron out issues—real-world emergency response situations clearly don’t afford such luxuries. In an emergency, dozens of organizations each need a filtered view of events that are, by definition, unexpected. This demo process placed a bright spotlight on some infrastructure changes that would make the deployment of emergency management systems much easier:

• Creation of a publish/subscribe message queue for communications
• System for guaranteed in-sequence, once and only once, delivery of messages from one to many
• Eliminate the need to embed timestamps or other arbitrary content in messages
• Get rid of schema validation that’s so strict it causes messages to be rejected for non-conformance.
• Enable filtration and routing of messages by geospatial coordinates and polygon inclusion

Geospatial routing is the key to information flow

This last point is very important, and it’s a new concept in message routing. In an emergency people have a literally “life and death” need for information about a specific area of interest. They can’t be bothered with topics and queues, nor with messages that don’t apply to them. It’s powerful to decouple senders and receivers so the only thing they need to share is the geospatial coordinates of latitude and longitude, or inclusion in a polygon. With geospatial routing senders don’t have to figure out which recipients they are sending alerts to, they can just “fire and forget” alerts with the confidence that the right systems and people will get them.

There are still two main types of application architectures in the world – request/reply where data is looked up as needed from a known location, and event-driven where new information is proactively sent to relevant systems in real time. Each has its uses, but there is no question that emergency response would be best served by a federated, event-driven information bus that can forward information between agencies and other organizations as events are unfolding.

First Forbes magazine did a cover story on The New Masters of Wall Street, highlighting the hedge funds, prop traders and big banks that are turning HFT into big business.

Then Traders Magazine did a related story about how the exchanges are responding in their Race to Zero Latency. Clearly the obsession with execution latency is being driven by the need to keep up with these ultra-fast consumers of liquidity.

And you know HFT is nearing its hype peak because last week the Daily Show did a special report on the subject.

Unlike Paris Hilton though, HFT is not making money because it is hot, it is hot because it is making money. Generating buckets of profits is one thing that never goes out of style.

How obsessive? Architects building these systems measure latency in microseconds, and the best applications exhibit just tens of microseconds of end-to-end latency. Shaving microseconds is like dropping weight before your prize fight weigh-in—whatever it takes, get it down.

To help these latency obsessed traders develop even faster trading systems, Solace has extended its Unified Messaging API to include a shared-memory transport based on inter-process communication (IPC). This capability lets two applications share information using Solace’s API with less than 700 nanoseconds of average latency in a shared memory environment. Yes, I said nano — billionths of a second. Remember the famous Tabb Report on The Value of a Millisecond? There are a million nanoseconds in a millisecond. 700 nanoseconds is a scant seven-tenths of a microsecond.

To be clear, IPC is a highly-specialized technique that only certain systems can leverage because it occurs within the confines of a single server. For example, when the components of a high-frequency trading system (feed handler, algo, risk assessment, order execution) have been consolidated onto a high-powered multi-core server within a collocation facility. Today these applications run on many machines and share data using low latency messaging (like Solace’s). Shared memory transport among applications running on a single server eliminates the few microseconds associated with network hops and additional time lags associated with copying memory around between applications. And since IPC is now available as part of the same API customers already use for ultra low latency and other kinds of messaging, applications get the speed they need without giving up the familiar API or the flexibility to redeploy in a networked scenario as needed.

As always, we’re not publishing some mysterious single number with no detail on what it means. A white paper describing the environment and parameters of the tests is available for download on our website so customers can dig into the facts and even reproduce the results using their own systems and data. In fact, we did all the testing a quad-core 3GHz Intel Xeon E5450 server because not everyone has the latest Intel Nehalem.

HFT architects have generally been exempt from corporate technology standards because the stakes are so high they can justify whatever makes them faster. With Solace, HFT no longer needs to be an exception. The same messaging API that is speeding up back office and front office networked trading can be used to speed up collocated HFT trading as well.

Many people have JMS pigeon-holed as being just for these relatively simple, low-volume tasks because while it’s very easy to set up, its performance is relatively limited. You can always scale JMS by splitting the workload across dozens or hundreds of servers, but that introduces major architectural complexity, administrative nightmares and high TCO. That simple JMS deployment is no longer so simple.

Solace recently announced a new version of its JMS broker that has the horsepower to support enterprise-scale applications in a purpose-built appliance that’s easy to deploy and has a small footprint in the datacenter. Each Solace message router (just 4 rack units high) supports 100,000 persistent JMS messages per second, and 11M non-persistent JMS messages per second (for example 5.5M in and 5.5M out, or 3M in and 8M out in a pub/sub configuration). Best of all, each Solace message router can handle high volumes of persistent and non-persistent messages simultaneously.

Throughput of 11M messages per second allows JMS to be considered for some of the largest event networks in the world with a single fault-tolerant pair of Solace routers. This will dramatically reduce the effort it takes to architect and operate large JMS networks. This kind of volume is useful when you have a very high powered application, as described above, but also when you need to connect thousands of applications that each have a moderate amount of information to send. At these kinds of volumes, thousands of clients could each consume thousands of messages/second and still have headroom on the router.

It’s time for the workhorse to get some time on the track.

Solace signed a partnership with Sun shortly before the Oracle acquisition was announced in the spring. Sun’s technology assets and Solace’s high-performance JMS solution have generated interest with Sun customers since we began working together more than a year ago. Sun is a powerhouse in the data center of many of the country’s largest, most demanding companies, including most of the major banks. For our part, Solace is reshaping what has traditionally been software-based middleware into a hardware asset more like a network appliance than a server.

The combination of these two strengths is the basis of a new prototype appliance we demonstrated in the Sun booth at the SIBOS show in Hong Kong September 14-18. This new Sun financial network appliance combines the massive throughput and easy management of Solace’s messaging technology with Sun’s hardware and software to build a flexible solution for integration between back office banking systems and leading payment networks. Integration with payment systems is a notoriously complicated thing to get right, and the turnkey approach of this appliance can make connectivity faster, easier and more profitable for participants of banking inter-networks.

The appliance is based on the Sun Netra x4450 chassis, a powerful virtualization server that’s designed for telco-grade reliability in a low-power 4U form factor. It includes Solace’s JMS broker and middleware functionality with appliance-style turnkey administration. Finally, it includes Sun’s Secure Network Layer software as well as the Sun Integrator platform for easy integration of banking assets.

In related news, Solace announced a new version of our JMS protocol that’s used for this appliance, and is also available directly from Solace. It sets new standards for JMS performance with support for 100,000 persistent and 11 million non-persistent messages per second.

We look forward to a long and successful relationship with Sun (and eventually Oracle).

This has been a busy news week for us, as we released information about two exciting product developments and a project that we’ve been hard at work on with Sun Microsystems. For good measure, we were included in a fourth announcement issued by OASIS. Below is a summary of the four pieces of news. Watch this space, because we’ll be posting more information about all of this news over the next few days.

• The release of new benchmark data showing our JMS broker to be the world’s fastest.
• The addition of shared memory IPC transport to our Unified Messaging Platform API that brings performance into the world of nanoseconds.
• A announcement with our partner Sun to demonstrate an appliance for inter-bank connectivity.

Finally, OASIS (the Organization for the Advancement of Structured Information Standards) just announced that we’ll be participating in a joint demonstration at the upcoming Emergency Interoperability Summit in Baltimore.

Today in New York our CEO Craig Betts co-keynoted the High Performance on Wall Street conference with Andy Bechtolsheim and a representative of Barclays Capital. The theme was something we talk about a lot at Solace: the convergence of middleware and the network. The need to effectively route an increasingly massive volume of data at high speed is defining competitive advantage in more and more situations. The only way we’ll ever address this trend is by making the intelligent routing of messages a robust and readily-accessible service of the network, just like the routing of IP packets. We can only shove so many servers into the increasing number of holes in the dike before the water pours through.

Craig talked about the 4 key business drivers of IT innovation today (the need for speed, the need to save, the desire to be more environmentally sensitive and the need to stay productive through M&A activity) and how they all lead down this path of convergence. Mr. Bechtolsheim dove a little deeper into the specifics of network architecture and operations to illustrate the point that raw computing power isn’t the way to crack this nut. Finally, Barclays Capitals’ head of middleware talked about the business value of consistent, predictable performance which drove them to select a (ok, our…) hardware-based middleware platform.

From business drivers, to nuts and bolts, to the bottom line, the presentation made it clear that the migration from old-fashioned software-based infrastructure to hardware is a fundamental corner we must turn.

Here at Solace, we entered this market earlier this year with products and solutions for scalable geospatial routing and alerting. This is hot property in a variety of markets including government, logistics and social networking.

Late last week the NY Times printed an article on how Goldman and others are using what they’re calling “high-frequency trading” to get ahead of the market and generate profits.

This is not new news to folks on Wall St, nor is it illegal. As opposed to “front running” scandals of the past where trading firms would use inside information to profit from coming news, all of these advantages are achieved after information has been made public. They just get to see it before everyone else by having faster infrastructure, even if it is only a few milliseconds, or microseconds faster. This is like an astronomer getting a better view of planetary objects by having a better telescope. You dramatically increase your likelihood of discovering something good up there if you get a telescope as good as the US government scientists.

Recently, Ralph Frankel, Solace’s CTO of financial services, had an article published on this topic in Securities Industry News. If you’re interested in this topic, it goes into much more depth about how latency arbitrage in high-frequency trading actually works.