Hi, I am Gregor Hohpe, co-author of the book Enterprise Integration Patterns. I work on and write about asynchronous messaging systems, distributed architectures, and all sorts of enterprise
computing and architecture topics.
Find my posts on IT strategy, enterprise architecture, and digital transformation at ArchitectElevator.com.
Two weeks ago I presented a keynote at the DEBS (Distributed Event-based Systems) conference. The emphasis of the conference was on academic work in the field but event-based systems are rapidly gaining traction in commercial applications as CEP (Complex Event Processing) is becoming a household name like SOA (and equally difficult to pin down). All this talk about events reminded me that applications on the Web are becoming increasingly event driven.
How many times have you visited a website just to check the status of a resource and to make a decision based on it? Do I have enough money left in my checking account? Has that check cleared yet? Did this camera I am looking to buy go on sale? Is anyone selling the used car I am looking for? While an HTML and HTTP-based Internet is inherently tied to a synchronous request-response interaction, the user model often more closely resembles an ECA (Event-Condition-Action) interaction. A specific event (e.g., the price of the digital camera drops below $300) is applied to one or more conditions (bank account balance is > $1000 and significant other approves) and results in an action (buy camera). Sadly, most Web page users have to implement the event-driven aspect of this interaction through polling, i.e. by repeatedly opening a Web page, retrieving the data, and comparing it to the conditions. This is obviously tedious and cries for a more efficient solution.
Many years ago, the availability of information was the biggest challenge. With the advent of the Internet finding the right information was the challenge. Now that this problem has been mostly solved (my employer had a certain hand in that) accessing and processing information has become the biggest challenge. Having an event-based system to perform preprocessing for us and alert us only when certain conditions match allows us to process more information in a more efficient manner.
The late 90ies saw the rapid rise of a push-based Internet but things have gotten fairly quiet in that segment (a search for "Marimba" nowadays seems to yield more musical sites than Internet-related topics). I believe that the most established event model for the Web today takes the form of feed readers. Feed readers retrieve feed data, checking periodically for new items in the feed. Originally meant for syndication of news items and blogs, these readers have one important property: they perform the polling in the background and alert the user to new items. This works especially well in reader sidebars, as supplied for example by Google Desktop or other pluggable gadget frameworks. Such sidebars display feed entries in a floating bar on the desktop so that the user can easily track the arrival of new items while working on other applications.
For these types of alerts to be useful, the user has to have the ability to specify alert criteria or rules. Increasingly, feeds accept query parameters to perform such functions. For example, Craigslist allows me to receive new entries (i.e. alerts), when someone posts the type of item I am looking for, based on price and keywords. For example, I can be alerted to any Audi S4 Avant for sale in my area with a subscription to http://www.craigslist.com/cgi-bin/search?areaID=1&subAreaID=0&query=s4+avant&catAbbreviation=car&minAsk=min&maxAsk=max&format=rss. Displaying the resulting feed in Google sidebar puts the burden of implementing an event-based system onto the RSS client, not me.
One can also compare these types of subscriptions to long-running database queries that update the result set whenever new data arrives. Such analogies remind us that we can often view the same system from a different perspective. Is an RSS feed more like a database query or more like an event stream? I believe it is both, it depends on how you use it.
As always, many useful applications on the Internet are somewhat complicated by security issues. While a feed into my checking account or credit card activity would be surely convenient, serious security concerns are likely to make this a wish rather than reality. But many banks have recognized user's desire for events and provide one-off event notification mechanisms in lieu of RSS. For example, my bank alerts me via e-mail or SMS when a deposit is made (yeah!) or a check cashed (boo!). Once again it turns out that asynchronous messaging is a great way to deliver events. So the idea of using e-mail and SMS as a transport seems natural. As ubiquitous as these protocols are, though, the integration opportunities are more limited than with HTTP and RSS.
Interestingly, users receiving these event streams have to deal with some of the same issues that can make building event-based systems challenging. For example, events can arrive out of order. Or event correlation may be difficult. For example, I may receive an event telling me that my credit card payment posted. Another event alerts me to the debit to my checking account. Without sufficient context it is not obvious that these events are related because they originate from different sources. Maybe banks should allow me to specify a correlation identifier to make this easier so I can do some CEP on my financial transactions? Lastly, event-driven applications have to deal with events as they come in, often requiring the recipient to regain enough context to deal with the event. For example, when I process my e-mail stream I often have to remind myself what we were talking about by looking at the history of the conversation or a related Web site.
This rambling has been all work and no play, and experience shows that this makes us a dull boy (and makes us want to kill our family with an axe). To stay clear of that scenario I prepared a few running examples that show how to build event streams on the Web. Stay tuned.
In the meantime, if you want some event-based notification of content changes on my site, I have two feeds for you, one feed for my conference talks and one feed for my ramblings.
Gregor is a cloud architect and author. He is a frequent speaker on asynchronous messaging,
IT strategy, and cloud.