Enterprise Integration Patterns: Gregor's Ramblings

This blog chronicles my journey with asynchronous and distributed system design over two decades, covering integration, messaging, and conversation patterns, and patterns.

Find my posts on IT strategy, enterprise architecture, and digital transformation at ArchitectElevator.com.

RSS Feed Twitter LinkedIn

Queues are key elements of any asynchronous system because they can invert control flow. Their ability to re-shape traffic enables high-throughput systems that behave gracefully under heavy load. But all that magic doesn't come for free: to function well, queues require flow control Read more »

When describing messaging systems, it's natural to focus on data flow. However, control flow is what gives those systems their critical runtime properties like resilience or low latency. It's time to make control flow a first-class consideration. Read more »

Event-driven architectures (EDAs) are frequently pitched as loosely coupled, when compared to other forms of integration. Coupling is a multi-faceted and nuanced property, so as architects we should have a closer look. Read more »

Coupling is integration's magic word. Despite entire books being written on the subject, it's still widely recited as a "loose coupling is good" mantra. Coupling deserves better, so let's clarify the dimensions and nuances of coupling. Read more »

Do serverless solutions lock you in? Let's find out by porting / rebuilding the Serverless Loan Broker on top of Google's Cloud Platform (GCP). Read more »

Infrastructure as Code (IaC) brings repeatability into the provisioning and deployment of cloud applications. However, the vocabulary used by most automation tools describes cloud platform resources as compared to the application's intent. As serverless applications take us further away from the infrastructure, we should also find better abstractions to express our automation. Perhaps, Enterprise Integration Patterns give us a head start? Read more »

Infrastructure as Code (IaC) has a whole new meaning for serverless applications. Rather than provision resources (the serverless frameworks do that for us), automation determines the system composition and configuration. As expected, quite a few options are available from simple command line to programming frameworks. Time to have a look. Read more »

Part 3 of the mini-series on implementing the EIP Loan Broker as a serverless solution with AWS Step Functions uses a Publish-Subscribe Channel and a stand-alone Aggregator to request and process loan quotes. Read more »

In part 2 of this blog series I implement the Loan Broker Example using a Recipient List pattern, implemented in DynamoDB, Step Functions, and Lambda. Read more »

A lot has happened since we implemented the Loan Broker Example in EIP: we have the cloud, serverless computing, machine learning, service meshes and all sorts of other bells and whistles. Nevertheless, the integration patterns have passed the test of time as we shall see by implementing the original loan broker example with AWS Lambda and AWS Step Functions. Read more »

After examining which patterns are embedded in Google Cloud Pub/Sub in an earlier post, I implemented a few common patterns on top of Google Cloud Functions, Google's serverless implementation. It's amazing how easy it has become to get messaging patterns running in the cloud with only a minimum of wrapper code. Read more »

Enterprise Integration Patterns are timeless. That's why our book keeps selling well 20 years after its publication in 2003. However, the code examples haven't aged quite as well. I therefore updated several code examples to use modern tech like GoLang, Kafka, RabbitMQ, AWS, Azure Cloud Functions, and Google Cloud Pub/Sub. Read more »

About two years ago, I revived the ramblings from a four-year hibernation. Upon resurrection, the ramblings started to take a broader scope, including not only messaging, conversations, and patterns, but also communication and IT architecture topics. The new topics were a natural reflection of my new role as Chief Architect in a massive IT transformation and after a little while I decided that they deserve their own channel, so I wrote a book about it. Read more »

OOP is one of the conferences that originated in the "OO" era of the early to mid-nineties. While most of these events have shed the double "O" -- OOPSLA has become SPLASH and JAOO has become GOTO, OOP stayed true to its name and remains one of the premier German developer conferences. This year OOP celebrated its 25th anniversary. Here are my impressions. Read more »

The digital world is all about scalability: millions of web sites, billions of hits per month, more data, more tweets, more images uploaded. To make this work, architects have learned a ton about scaling systems. With everything around us scaling to never-before-seen throughput, the limiting element in all this is bound to be us, the human users. And the organizations we work in. One may wonder, then, why scaling and optimizing throughput in organizations is considered a very different field usually completely ignored by the IT architects. I feel that many of the scalability and performance enhancement approaches experienced IT architects know can be just as well used to scale organizations. If a coffee shop can teach us about maximizing a system’s throughput, maybe our knowledge of IT systems design can help improve the performance of organizations! Read more »

When you hitchhike through the galaxy, a towel will generally do. To ride the you need to be a bit better equipped. Riding the elevator up and down is exciting and provides tremendous value to the organization. But it is also demanding as you need to adjust to different audiences, backgrounds and dynamics, becoming a technology shapeshifter of sorts. To help with this transformation, I recommended a few books for each major section of the corporate high-rise. Read more »

As I recently observed, Corporate IT tends to be afraid of code: code is where all the pesky bugs come from, which have to be fixed by quirky, expensive, and unreliable developers or external consultants. This attitude plays to the favor of vendors who peddle products that "only" require configuration, as opposed to coding. Sadly, most configuration is really just programming in a poorly designed, rather constrained language without decent tooling or useful documentation. Being afraid of code and being unfamiliar with the modern development lifecycle is a dangerous proposition in a world where everything is becoming software-defined. Read more »

Corporate IT lives among zombies: old systems that are half alive and have everyone in fear of going anywhere near them. They are also tough to kill completely. Worse yet, they eat IT staff's brains. It's like Shaun of the Dead minus the funny parts. Read more »

I recently wrote about architects and the role they play in large organizations. Still, the question often remains what an architect should be doing besides "riding the elevator". Let's try another analogy: movie characters. Read more »

IT loves virtualizing stuff, following the old rule that in computer science every problem can be solved by just one more level of indirection. Cloud computing is based on virtualization of compute resources - you don't need to know which machine your application is actually running on and you can get new ones with the click of a button. Before cloud was a buzzword, though, VMware and others have virtualized machines at the operating system levels. Recently (in terms of buzz, not technology), containers bring another level of light-weight virtualization of resources. And let's not forget the Java Virtual Machine, which also claims a level of virtualization. What are we supposed to do with all these levels of virtualization? Read more »

Part of my job is to review system architectures. So I frequently ask teams to show me "their architecture", but almost as frequently I don't consider what I receive an architecture document. The inevitable question "what do you expect?" is also not so easy for me to answer: despite many formal definitions it's not immediately clear what architecture is or whether a document really depicts an architecture. One important test I apply is whether the document or the description contains any non-trivial decisions and the rationale behind them. Read more »

Architects can often be found commenting or complaining that many things in IT are the same old stuff in new packaging, created by marketing departments who were in need of a new buzzword. For example, aren't microservices really just SOA done right while SOA itself stands for "Same Old Architecture"? And the whole reactive movement seems to have re-discovered callbacks. Maybe there is some truth in this as at the recent SATURN conferences I found myself saying more often than I had expected "I blogged about this". So let's see which of my past ramblings are still relevant these days. Read more »

I have not blogged about events in a while, but SATURN 2015 has been an amazing event that's well worth rambling about. Read more »

Discussing the Google cloud Pub/Sub system in the last rambling reminded me that the Publish-subscribe Channel pattern makes for a good example of the subtle but important difference between Messaging Patterns and Conversation Patterns. This comparison is timely as I picked up documenting Conversation Patterns again. Read more »

Google released the beta version of their publish-subscribe API just a few weeks ago. I show how to build a very simple demo app using the Java API and map the functionality to integration patterns to illustrate the design choices the team made. Read more »

As indicated a good while ago I spent some time thinking about patterns that instead of following a message through multiple systems, looks at the message exchange over time between a (mostly) fixed set of systems. I call these message exchanges "conversations". Sadly my writing efforts on Conversation Patterns fell very much asleep around 2008, partly due to my fear that with the rise of REST, stateful conversations between systems would step into the background as most integration problems are now solved with a simple POST or GET. A presentation and conversation at Frank Leymann's IaaS PhD Seminar enlightened me that this is not at all true: systems following the REST architectural style very much engage in conversations. Read more »

We were tempted multiple times to extend the EIP icon language, but always felt that simplicity should win over precision. However, seeing the visual vocabulary that my former Google colleague Ivan Gevirtz created, I am convinced that combining the two is useful. Read more »

Architects frequently play a critical role as connecting and communicating element between multiple parties. Especially in large organizations such communication is an important factor: too many parties speak different languages to convey different, and often conflicting, objectives. Many layers of management only exacerbate the problem as communicating to corporate leadership resembles the "telephone game" where a circle of kids relays a message one-by-one just to realize at the end that the message has completely changed along the way. Read more »

Defining what a software or IT architect is or does is no less challenging than defining software architecture itself. The SEI maintains a list of software architecture definitions, yet I have not seen a corresponding list for software architects. One could try to skirt the issue by declaring that software architects are the people who make software architecture. However, I think that this approach would be missing some important aspects of being an architect. Read more »

One advantage of working for a relatively large organization is that I get to do a little more writing again. Not because I have spare time, but because in times of rapid change, communication across a large audience is critically important. It turns out, short papers are a good way to achieve that and my brief, but accurate technical position / decision papers have become a trademark of my architecture group (don't worry, we architects produce more than just paper). Correspondingly, one of my daily struggles is to convince others to write as well and to coach them to become better writers. In a sense, this forces me to transport what I know about writing from purely procedural knowledge, i.e. I know how to do it, to declarative knowledge, i.e. being able to understand why it is done and to describe to others how to do it. While this rambling's title is a pun on the popular books "Japanese for busy People", it intentionally implies an ambiguity that we are both writing for a busy audience, but are busy authors as well. Read more »

At a recent presentation, Martin Fowler introduced himself as a guy "who is good at explaining things". While this certainly has a touch of British Understatement™, it also highlights a critically important but rare skill in IT. As technology invades all parts of personal and professional lives, causing companies to absolutely depend on software and systems, decision makers often can no longer keep track of technical details. Because architects live at the intersection of technology and business, it is their responsibility to explain complex topics and highlight the ramifications of technical decisions. Read more »

Drawing a picture turns out to be a useful system design technique. Read more »

I was recently asked to help a team decide whether they should use messaging. Of course, I have not forgotten what I learned during many years in consulting: the consultant always answers "it depends." A good consultant can tell you what the answer depends on. And a truly great consultant can convince you that you are asking the wrong question in the first place. Read more »

My New Year's resolution was to write more, so here my thoughts on how to actually make that happen. In a sense it's a plan for myself to be more productive, but hopefully the ideas also work for other folks. Ironically, my work on EIP II has been stalled for a good while, so you're getting advice on how to be a prolific writer from someone who has not written much in 2 years. Read on at your own risk. Read more »

The beginning of a New Year is the time to reflect on the past and make resolutions for the future. It's become my tradition to kick off the year with some reflection on EIP, so here we go. Read more »

I joined JavaOne this year as a panelist on Cloud Computing. Here my belated impressions on this year's JavaOne. Read more »

By now everyone must have heard about Google Wave, the communication and collaboration platform announced at Google I/O. Google just announced that they are ramping up towards 20,000 developers on the current sandbox and will extend to 100,000 users by end of September. This will give many more people to opportunity to experiment with Wave. Read more »

I guess I am the last person to blog about Google I/O. By now, everyone should have read about Wave and the free Android phone (with 30 day SIM!). I am just now getting to write this up on my flight back to Japan, so I’ll give you my personal impressions and viewpoints, if you are still interested. Read more »

I have been speaking more frequently about cloud computing in recent days. As SOA is becoming a daily reality, I needed to advance to new, still slightly nebulous topic. What could be a better fit than cloud computing? All jokes aside, cloud computing provides for some interesting discussions and one should safely assume that Google has a word or two to say about it. Read more »

Blogging about design patterns seems about as original as blogging about the Java (TM) Programming Language, except for the missing (TM). However, as I just attended a workshop on software service engineering, I realized once again that people from different fields have very different notions about the concept and usage of patterns. Many workshop attendees raised questions about formalizing patterns, tooling, etc. I touched on this subject a tiny bit in a post from long time ago, but I figured that the topic really deserves more attention, especially since to me a pattern is so much more than the "proven solution to a recurring problem within a specific context." Read more »

The end of the year is always the time to reflect on the past happenings. My friends in Japan often send a New Year's card with 12 pictures, each showing the significant event during the month. I am not sure something significant related to Enterprise Integration Patterns happened each month, but I think it's still nice to reflect a little bit on the history and current state of the patterns. After all, the patterns have matured from a draft paper to being adopted as the lingua franca for a number of open source messaging projects. Read more »

Since Google's Developer Day, I have been promising to speak more regularly about Google products. This is a bit of a balancing act for me, as I want to avoid the slippery slope of becoming a corporate sock puppet. I also don't want to neglect my interest in connected systems, SOA, and asynchronous messaging. With the Internet turning into the largest connected system, the gap between Web-based technologies and connected systems design has become significantly smaller, allowing me to combine interests and Google developer products more naturally. Of course, the fact that Google is rolling out new developer products and APIs around the clock, does not hurt either. Read more »

I am just returning from a trip to Montreal and Keystone, CO for the OOPSLA and Colorado Software Summit conferences. I spoke on SOA Patterns, workshopped my conversation pattern paper, and gave six talks on event-driven architectures and building mashups using Yahoo! Pipes and Google Mashup Editor. Phew! Read more »

What’s wrong with this picture? I crash a stylish party at a pricey venue (SF Moma). I have no business being there, nor do I have the required invitation. I nonchalantly talk my way in, and hit the crowd with two fellow geeks. Within a few seconds an attractive blonde with a low cut dress approaches us and engages us in a conversation. We grab a few free drinks, indulge on the tirelessly circulating horse d’oeuvres, and talk to the blonde. Read more »

I attended the Enterprise Mashup Summit last Friday. It was a small-ish event, with about 50 people attending. We saw about 10 presentations, mostly by vendors plus an open forum. None of the presentations were too sales-ey, which gave the event a nice workshop feel. Given the limited size, it would have been nice to have more time for interaction. Here is a quick rundown of my impressions. Read more »

The Internet is cruel. So you finally made the leap from EAI stone age to become a hip mashup developer. Now it turns out you are once again behind the curve, Unless you've written your won Facebook app and scored a million or so installs you are just a worm in the dust of the global data superhighway. Join the club. Read more »

As I reported from Mashup Camp, an increasing number of vendors play in the mashup space. I am obviously not the only one who noticed. Dion Hincliffe recently discussed 17 different mashup tools. Maybe I have an MBA hidden inside me (or 10 years of consulting left some marks), but I was temped to conduct my own market segmentation. Gartner would be proud of me. Just be aware that my analysis is not meant to be comprehensive and all statements carry a 0.5 probability of being right :-) Read more »

Mashups pull data from different sources, aggregate and transform the data to be used in different contexts. EAI solutions pull data from different sources, aggregate and transform the data to be used in different contexts. Huh? Read more »

I attended the first day of Mashup Camp today. The event took place at the Computer History Museum, which is actually walking distance from my office. Ironically, the only day in the year when something actually happens within walking distance in Silicon Valley it had to rain. Oh well. Read more »

Last time I claimed that users like events. This time I want to show how I fulfilled my personal desire for events off the Web. I built two solutions to alert me to new book reviews on Amazon, one using a Python script, the other using Yahoo! Pipes. Read more »

Two weeks ago I presented a keynote at the DEBS (Distributed Event-based Systems) conference. The emphasis of the conference was on academic work in the field but event-based systems are rapidly gaining traction in commercial applications as CEP (Complex Event Processing) is becoming a household name like SOA (equally difficult to pin down). All this talk about events reminded me that applications on the Web are becoming increasingly event driven. Read more »

When graphical user interfaces became popular thrifty people had the idea that they could generate them automatically. Well, that bubble burst pretty quickly. When people say nothing is new with Web services, they are right in this case. Once again people want to generate the interface automatically. Read more »

Last week Google held a developer day in 10 cities around the world. Here are my take-aways. Read more »

You can only get away with joking about transactions for so long before the gods of transactions bring you to justice. So is was only a matter of time before Eric Newcomer would catch me waxing about coffee and transactional integrity... Read more »

Beginning of this year I submitted an article to a renowned software publication who is running a special issue on software patterns. Sadly, my article did not make it into the magazine but I decided to publish an expanded version on my site. Hey, I got to make it worth your time following my ramblings! Am I bitter that my article was rejected? Not at all, but if I ever get a hold of these #^%#$& -bleep- of a -bleep- I'll... Read more »

JavaOne is one of the few geek conferences that take place in San Francisco. The content can be hit or miss but the proximity and the good parties always make it worth stopping by. I also had to fulfil my duty to recruit the last smart software engineer in the Bay Area to Google. Read more »

I have been pretty quiet recently, but I finally have some results to show. The current issue of IEEE Internet Computing (May/June 2007) contains my guest column for "Toward Integration". Also, I authored one chapter in the upcoming book "SOA Expertenwissen" (in German). Read more »

Even though I have been working for the (self-declared) non-evil empire for a while now, I still have a lot of ties to the Microsoft community, even beyond drinking Weissbier with Christian Weyer. I am particularly proud that I have been able to maintain my MVP status, Microsoft's Most Valuable Professional award. Here my (brief) report from the MVP Summit in Seattle & Redmond. Read more »

Erik Doernenburg and I have been presenting our talk on Software Validation a few times now and have received very positive feedback. In the talk we mention that beyond visualization you can use similar techniques to perform checks and validations against a system. However, none of our demos actually included an example. So I went out to correct that and added validations to my messaging visualizer. Read more »

I am just riding the train back from Aarhus (the second largest city in Denmark in case your geography skills are as weak as mine were before my first trip here) to Copenhagen. Many speakers consistently rave about JAOO. The speaker roster, the logistics, the attendees - everything just seems to be very high quality and run very smoothly. This year I was nominated to be track host for the SOA track entitled "SOA - What's Left to Say?" Read more »

This year I got to go to fooCamp! What is fooCamp? It is sort of like Burning Man for techno geeks. But just as Burning Man has gone pretty high tech fooCampers started to have more fun by burning things. It's difficult to put the experience into words... Read more »

I confess. I am a groupie. When Tim O'Reilly invited me to be part of a workshop on Web 2.0 patterns I was quite excited. First, because I was going to meet the exalted one in person. Second, because I would have a chance to learn a lot about Web 2.0 as I am pretty much starting from zero. Read more »

I just returned from a trip to Germany to attend a workshop on The Role of Business Processes in Service Oriented Architecture. Besides a guaranteed top score in buzzword bingo the workshop provided a unique opportunity to connect thought leaders from academia and industry over the course of a week. The attendee roster sported the usual industry heavyweights, i.e., Microsoft, IBM, SAP, making me proud to add Google to the list. Read more »

Hanging out with my intellectual drinking buddies reminded me that our integration patterns have been embraced by a fair number of commercial as well as open source projects. In my eyes this is really the best indicator of success for a pattern language. Latin, a dead language used mostly by doctors to sound more knowledgeable, is not a good model for a pattern language. You want a language that is alive and actually used by people. Talking to a few folks who did embrace our language motivated me to take a quick survey of the places where our patterns pop up. Read more »

I am so clever. In my very first professional project in the US over 10 years ago I used a DSL to solve an EAI problem. If those buzzwords had been around at the time I would have been pretty cool, too. I guess you can’t have everything. Read more »

Some of the most hated people in San Francisco must be the meter maids, the DPT people who drive around in golf carts and hand out tickets to anyone who overslept street cleaning or did not have enough quarters for the meter. On some projects, the most hated people are the metric maids, the people who go around and try to sum up a developer’s hard work and intellectual genius in a number between 1 and 10. Read more »

A lot of discussion these days relates to DSL's (Domain Specific Languages) that allow business models and rules to expressed in a more readable fashion in the source code. Ultimately one might be able to share the executable source of the system with the business users to validate that this is what they wanted. While Java is not necessarily the greatest language to "host" a DSL we can go a lot further than developers generally believe or care for. Read more »

Computer science is known as the field where every problem can be solved by just one more level of indirection. Well, a message channel berween two components is such a level of indirection. As it turns out naming this additional element can have a more profound influence on the system design than might appear at first. Read more »

After joining the big G I put particular emphasis on staying in touch with the .Net community. For one thing, a fair number of my intellectual drinking buddies tend to congregate around the large software company in the Pacific Northwest. Second, as a believer in cross-platform integration, technology-neutral design patterns and peace on earth I want to make sure I switch the koolaid flavor every so often. One opportunity to do that presented itself at OOP in Munich where Ralf Westphal invited me to speak on distributed system architectures as part of the .Net track. During the proverbial hallway discussions we started talking about boxes and lines, but in a profound way... Read more »

Remember the people from high school who had strange hobbies and could never get a date? Well, they all grew up and they are running a conference now, the Object-Oriented Programming, Systems, Languages, and Applications conference, OOPSLA in short. While the conference logistics can be quirky at times, the content is top notch. And it seems like most of them have managed to get a date after all. Read more »

Two years ago you were not cool if you did not have a blog. Today you are not cool if you have not done a podcast. Of course I cannot allow myself to fall behind, so here is my podcast on layered applications: Read more »

(Professional) life can be a little ironic sometimes. Now that I work for Google I was able to go to Microsoft's Professional Developer Conference aka PDC for the first time. It was actually very nice to be able to stay in touch with the "other side". Read more »

Two weeks ago I attended a very small, but special conference on enterprise software development. Here is what I came away with.. Read more »

One of the biggest perceived fears of authors is that someone publishes a book or an article on the same topic just ever so slightly ahead of you that you have done all the work but look like a copy cat if you still publish your work. In reality this actually does not happen very often (at least not for books) as each author has his or her unique style and view even on the same topic. This time, though, Ken Arnold stole a little bit of my show by publishing an excellent article in called "Programmers are People, too". Read more »

As many people have already heard, I changed employers. Before the rumor mill completes its work entirely, here a few comments from my end. Read more »

I just returned from Microsoft TechEd Europe in Amsterdam. I was lucky enough to present on event-driven architectures in front of almost 600 people. Since multiple people asked me for the source files, I figured I'd post a quick summary of the talk, including the source files. Read more »

Having been to a number of conferences and their attached expo floors recently it was not surprising that I was not able to escape the line "this tool does not require programming, everything is done through configuration". Hearing this tag line for the third or fourth time got me wondering, "what really is the difference between coding and configuring?" Read more »

This year I actually missed putting in a proposal for TechEd. However, that has not kept me from actually presenting :-) So I am here fighting the Orlando Summer humidity together with 11000 other geeks and am actually having a good time. Read more »

On a recent project we developed a Web services framework that supports asynchronous callbacks and asynchronous invocation with polling. The difficulties we had describing what we were trying to do to my fellow ThoughtWorkers on other projects prompted me to discuss the different forms of asynchrony in a little more detail. Read more »

When you build a distributed system you have to choose between latency and reliability. Sticking your head in the sand will not help (and you get sand in your ears...). Read more »

In my perennial battle to free distributed system developers' minds from the perils of the call stack mentality I came across another reason why a call stack is not appropriate for distributed applications: the call stack is inherently asymmetric. However, when building distributed applications, that asymmetry really has no place. Read more »

It seems like I spend all my time at conferences. I am speaking at the Patterns and Practices Summit and SD West this week. This one is setup by Keith Pleas in close collaboration with the Patterns and Practices group at Microsoft. So yes, I am still promiscuous when it comes to platforms. But it makes me feel better that Ted Neward seems to beat me in that category, though. Read more »

A number of people have already about the TSSS but that can't stop me from occupying my fair share of the cyberspace with my impressions. Read more »

OK, so I am officially the slowest blogger on the 'Net... last week I attended two conferences, the BTW in Karlsruhe, Germany and TheServerSide Symposium in Las Vegas. Too bad they are over 6000 miles apart :-( Read more »

Today's rambling takes a look at the application side of application integration. We recently looked at a packaged application under development and looked at criteria and options for how this application can live in an evolutionary and connected environment. Read more »

Happy New Year everybody! Over the holidays I tried to sort out some of my thoughts on Web services, SOA, architectural styles, coupling and all the other good stuff. I started to think about service-oriented architectures as an architectural style and how it compares to prior styles, such as distributed component architectures. Maybe it was the New Year's mood but I started to think that the evolution of architectural styles is a little bit like drunk driving... Read more »

One of my favorite pastimes is to argue with people whether a solution is stateless, whether it should be stateless and what it means to be stateless in the first place. Ideally, the debate would involve alcoholic beverages and the other person would pick up the check. After "loosely coupled", "stateless" must be a close runner-up as the ultimate nirvana in buzzword-compliant architectures. It is also equally hotly debated. Today, I 'd like to share my view on state and lessness. If you disagree you are welcome to argue with me, but you are buying! :-) Read more »

Loosely coupled architectures are often portrayed as the holy grail of architecture. But all too often these architectures turn into what Martin Fowler coined the "Architect's dream but the developer's nightmare". This time I want to share some of the challenges we have found working with loosely coupled architectures and how we addressed them. Read more »

People often criticize asynchronous messaging solutions as too complicated and cumbersome. Or, they believe distributed solutions cannot be successful unless they include a distributed transaction model. There is little doubt that asynchronous solutions require us to think in new ways as we have to deal with concurrency, out-of-sequence issues, correlation and other. However, the real world is full of examples of asynchronous processes that deal successfully with exactly the same issues. We don't have to go further than the local coffee shop... Read more »

I just attended the conference in Toronto (I thought a 1/2 day tutorial in integration patterns). This conference is more of an academic events when compared to some of the purely commercial conferences like JavaOne. It was interesting to see what academia are working on these days in the areas of middleware and SOA. Read more »

Last year's OOPSLA tutorial on Enterprise Integration Patterns was quite well received. The most frequent constructive criticism was that the whole tutorial was PowerPoint as opposed to running code. I took that feedback to heart and set out to develop a toolkit that demonstrates the routing and transformation patterns in our book. After a lot of deja-vu in Windows GDI programming I created a toolkit that contains each pattern as a small executable. The toolkit is a lot of fun and makes for great demos, but it also triggered a bigger question. Are patterns meant to be codified as components? Read more »

I just attended the Indigo Strategic Design Review in Redmond. This means we got to interact with the product team and play with some pre-beta bits. Naturally, a lot of details are under NDA and are likely to evolve before the beta release. So here some rather general impressions -- I heard Internet connectivity in jail is really bad :-). Read more »

Our most recent project was to replace a batch-oriented stored procedure with a more flexible, maintainable and scaleable Java-based architecture. We chose an event-driven architectural style that processes events as they occur. For the initial roll-out we did not even have to distribute the solution so it all runs in single JVM (if you ignore clustering for the moment). Read on to learn more about the implementation and our experiences with intra-JVM EDA. Read more »

The notion of Dependency Injection or Inversion of Control has generated quite a bit of interest lately. To a large extend, the move towards this architectural style is driven by developer's dissatisfaction with strong dependencies of individual components on their run-time container (for example, J2EE). These dependencies make it difficult or impossible to test components outside of their container, throwing a major monkey wrench into test-driven development of these types of applications. Dependency Injection avoids these dependencies and therefore improves testability. On a recent project we developed an event-driven architecture (intra-JVM, not distributed) and found that we could realize many of the same benefits. Read more »

It is nearly impossible to keep track of all the Web Services (WS-*) specifications or proposed "standards". The matter is only worsened by the many competing proposals. Should you be using WS-Eventing or WS-Notification? How about WS-Reliability vs. WS-ReliableMessaging? Even if you choose the "right" specification, it still is likely to evolve over time. So are these standards useful at all for Web Services development today? I contend that they are, albeit in a somewhat unconventional way.... Read more »

Many of you have seen my rants on doodleware and top-down modeling. At the same time, anyone who has seen my book must believe me when I say that I am a very visual person. Is Gregor going schizophrenic (or becoming a hypocrite)? Not at all! I am a big fan of visual representations that are harvested from an actual running system and present the user with an accurate, big picture view of a working system. I felt it was time to put some money (time) where my mouth is and try to build such a solution and see what actually happens... Read more »

For a change I spoke at a conference without traveling halfway around the world. At over 15,000 attendees, JavaOne is one of the largest developer conferences. Unfortunately, the signal-to-noise ratio is not always all that great, but it is still a good place to meet old and new friends. This year's conference was littered with talk and demos on SOA, ESB, Web Services, orchestration etc. Here my impressions... Read more »

When using asynchronous messaging, we can no longer rely on the linear, in-sequence execution of events. Therefore, the notion of correlating multiple messages is fundamental to designing robust messaging solutions. While correlation seems to be a simple concept at first glance, there are quite a few interesting nuances. Read more »

A number of integration books classify integration into styles such as database integration, functional integration and so on. However, on closer examination it turns out that there is a fair amount of confusion about what these styles really mean. Let me give my view. Read more »

I had never thought about having guest appearances on my ramblings, but this e-mail response from my friend Hyon made me laugh hard enough that I think it would be a waste not to share it with the community. Read more »

Design decisions can be influenced by many factors. In order to make matters more predictable, many architects look for clear-cut borderlines. How complex does my screen navigation have to be in order to justify a Front Controller as opposed to a Page Controller? How complex does my integration problem have to be to justify use of an EAI suite instead of hand wiring the solution? Invariably, the answer is "it depends". Let's have a closer look at why that is so. Read more »

Encapsulation and abstraction are the holy grail of today's software development. One of the great successes of hiding the implementation details under a common, abstracted interface are TCP/IP stacks. But as so often, overwhelming success can also have a flip side. Read more »

When we assess the quality of a software engineer we don't look as much how long he has worked with a specific language but rather whether he or she has a good grasp of the underlying models and techniques, such as OO design, systems architecture. Still most EAI job offers sound like a software catalog: "x years TIBCO IntegrationManager", "y years Vitria Businessware" etc. etc. Some thoughts on what we should be really looking for... Read more »

The Hub-and-Spoke concept manifests itself in many ways in integration solution. Like any popular concept, it can sometimes cause as much confusion as it provides help. Read more »

Many integration tools offer graphical development tools. Do they really make integration easier? Read more »

Integration appliances aim to make integration cheaper faster and more reliable. Is this technology ready for prime time? Read more »