Gregor’s Ramblings on Integration - Enterprise Integration Patterns

This blog chronicles my journey with asynchronous and distributed system design over two decades. These are my Ramblings related to Integration [See all Ramblings]. Subscribe to my ramblings via RSS.

Find my posts on IT strategy, enterprise architecture, and digital transformation at ArchitectElevator.com.

Queues are key elements of any asynchronous system because they can invert control flow. Their ability to re-shape traffic enables high-throughput systems that behave gracefully under heavy load. But all that magic doesn't come for free: to function well, queues require flow control Read more »

When describing messaging systems, it's natural to focus on data flow. However, control flow is what gives those systems their critical runtime properties like resilience or low latency. It's time to make control flow a first-class consideration. Read more »

Event-driven architectures (EDAs) are frequently pitched as loosely coupled, when compared to other forms of integration. Coupling is a multi-faceted and nuanced property, so as architects we should have a closer look. Read more »

Coupling is integration's magic word. Despite entire books being written on the subject, it's still widely recited as a "loose coupling is good" mantra. Coupling deserves better, so let's clarify the dimensions and nuances of coupling. Read more »

Do serverless solutions lock you in? Let's find out by porting / rebuilding the Serverless Loan Broker on top of Google's Cloud Platform (GCP). Read more »

Infrastructure as Code (IaC) brings repeatability into the provisioning and deployment of cloud applications. However, the vocabulary used by most automation tools describes cloud platform resources as compared to the application's intent. As serverless applications take us further away from the infrastructure, we should also find better abstractions to express our automation. Perhaps, Enterprise Integration Patterns give us a head start? Read more »

Infrastructure as Code (IaC) has a whole new meaning for serverless applications. Rather than provision resources (the serverless frameworks do that for us), automation determines the system composition and configuration. As expected, quite a few options are available from simple command line to programming frameworks. Time to have a look. Read more »

Part 3 of the mini-series on implementing the EIP Loan Broker as a serverless solution with AWS Step Functions uses a Publish-Subscribe Channel and a stand-alone Aggregator to request and process loan quotes. Read more »

In part 2 of this blog series I implement the Loan Broker Example using a Recipient List pattern, implemented in DynamoDB, Step Functions, and Lambda. Read more »

A lot has happened since we implemented the Loan Broker Example in EIP: we have the cloud, serverless computing, machine learning, service meshes and all sorts of other bells and whistles. Nevertheless, the integration patterns have passed the test of time as we shall see by implementing the original loan broker example with AWS Lambda and AWS Step Functions. Read more »

Architects can often be found commenting or complaining that many things in IT are the same old stuff in new packaging, created by marketing departments who were in need of a new buzzword. For example, aren't microservices really just SOA done right while SOA itself stands for "Same Old Architecture"? And the whole reactive movement seems to have re-discovered callbacks. Maybe there is some truth in this as at the recent SATURN conferences I found myself saying more often than I had expected "I blogged about this". So let's see which of my past ramblings are still relevant these days. Read more »

Discussing the Google cloud Pub/Sub system in the last rambling reminded me that the Publish-subscribe Channel pattern makes for a good example of the subtle but important difference between Messaging Patterns and Conversation Patterns. This comparison is timely as I picked up documenting Conversation Patterns again. Read more »

Google released the beta version of their publish-subscribe API just a few weeks ago. I show how to build a very simple demo app using the Java API and map the functionality to integration patterns to illustrate the design choices the team made. Read more »

As indicated a good while ago I spent some time thinking about patterns that instead of following a message through multiple systems, looks at the message exchange over time between a (mostly) fixed set of systems. I call these message exchanges "conversations". Sadly my writing efforts on Conversation Patterns fell very much asleep around 2008, partly due to my fear that with the rise of REST, stateful conversations between systems would step into the background as most integration problems are now solved with a simple POST or GET. A presentation and conversation at Frank Leymann's IaaS PhD Seminar enlightened me that this is not at all true: systems following the REST architectural style very much engage in conversations. Read more »

As I reported from Mashup Camp, an increasing number of vendors play in the mashup space. I am obviously not the only one who noticed. Dion Hincliffe recently discussed 17 different mashup tools. Maybe I have an MBA hidden inside me (or 10 years of consulting left some marks), but I was temped to conduct my own market segmentation. Gartner would be proud of me. Just be aware that my analysis is not meant to be comprehensive and all statements carry a 0.5 probability of being right :-) Read more »

Mashups pull data from different sources, aggregate and transform the data to be used in different contexts. EAI solutions pull data from different sources, aggregate and transform the data to be used in different contexts. Huh? Read more »

Last time I claimed that users like events. This time I want to show how I fulfilled my personal desire for events off the Web. I built two solutions to alert me to new book reviews on Amazon, one using a Python script, the other using Yahoo! Pipes. Read more »

Hanging out with my intellectual drinking buddies reminded me that our integration patterns have been embraced by a fair number of commercial as well as open source projects. In my eyes this is really the best indicator of success for a pattern language. Latin, a dead language used mostly by doctors to sound more knowledgeable, is not a good model for a pattern language. You want a language that is alive and actually used by people. Talking to a few folks who did embrace our language motivated me to take a quick survey of the places where our patterns pop up. Read more »

A number of integration books classify integration into styles such as database integration, functional integration and so on. However, on closer examination it turns out that there is a fair amount of confusion about what these styles really mean. Let me give my view. Read more »

When we assess the quality of a software engineer we don't look as much how long he has worked with a specific language but rather whether he or she has a good grasp of the underlying models and techniques, such as OO design, systems architecture. Still most EAI job offers sound like a software catalog: "x years TIBCO IntegrationManager", "y years Vitria Businessware" etc. etc. Some thoughts on what we should be really looking for... Read more »

The Hub-and-Spoke concept manifests itself in many ways in integration solution. Like any popular concept, it can sometimes cause as much confusion as it provides help. Read more »

Many integration tools offer graphical development tools. Do they really make integration easier? Read more »

Integration appliances aim to make integration cheaper faster and more reliable. Is this technology ready for prime time? Read more »