Hi, I am Gregor Hohpe, co-author of the book Enterprise Integration Patterns. I work on and write about asynchronous messaging systems, distributed architectures, and all sorts of enterprise
computing and architecture topics.
Find my posts on IT strategy, enterprise architecture, and digital transformation at ArchitectElevator.com.
A lot of developers are talking about inversion of control (IoC) and dependency injection these days. I don't want to reiterate what others have already described in much detail. If you are interested in the detailed story behind dependency injection please refer to this article by Martin Fowler or this post by Aslak Hellesoy and Paul Hammant (fellow ThoughtWorkers in the UK). To save you some reading, here is the gist of it (or at least my interpretation). Assume an object A requires a reference to object B because it wants to make use of B's services. The usual approach is for A to instantiate an instance of B using the 'new' command. A now holds a reference to B and can invoke its methods. So far, this could well be a lesson from OO 101 and is not terribly interesting. What, however, if in different situations A needs different versions of B? For example, assume B provides persistence services. During testing you might want to simply write data into a file but during production you might want to use an industrial-strength database. Or, if B provides messaging services, you might want to use a synchronous implementation for unit testing and an asynchronous one for integration testing and production. Or, you might want to use a mock-messaging layer that is not a messaging layer at all but verifies that A sends the correct data to the layer.
Object-oriented constructs like interfaces, inheritance and polymorphism give us well known mechanisms to hide different implementations behind a common interface. This allows A to access B's services via a common interface B, which may be implemented by different implementations B* and B'. The question that now remains is, how does A instantiate the right implementation of B? Ideally A should not be aware that there is a testing and production mode and that there are two different implementation of B. This implies that A should no longer instantiate B' or B* directly. There are multiple ways to resolve this dependency. One options is to allow A to use a B-factory. This way A can ask the factory for an instance of B. The factory could be smart enough to figure out which implementation of B to create behind the interface. Alternatively, A could simply advertise that it requires a reference to B and the external environment (e.g. the run-time container) would pass in the appropriate implementation of B. For example, in Pico Container objects use the class constructor to advertise their needs for other objects. The container inspects the constructor, instantiates the appropriate implementations of the specified types and passes them to the component.
So what does this have to do with messaging? Let's assume a component A is processing data. In a common procedural approach, A would either call the data source for data or the data producer would invoke A with the data. In either scenario there is a direct linkage between A and the source of the data. Let's now assume that there is a chain of these types of data processing components. If each component expects to be called with data it is easy for us to pass in test data: we simply instantiate the component and invoke it with our desired test data. However, the component will in turn call other components. This is often not desirable for a unit test because we prefer to test only one component at a time. Likewise, if each component asks it source for data we need to be able swap out the source for a test data source to feed test data into the component.
Pipes-and-Filters messaging architectures help break these dependencies between components. Component A no longer requests data from a specific component nor does it send data to a specific component. Instead, the component consumes messages of a channel and writes messages to another channel. This makes it ideal to replace a component's ecosystem during testing. This way we can feed test data into the component's input channels and monitor the component's output channels for correct result data. Essentially, component A has no direct dependencies on any other component. However, just as with the Inversion of Control containers somewhere the dependencies must be resolved. In the world of messaging this can happen in one of two ways:
In either scenario the component itself does not acquire references to other component but 'advertises' its needs by subscribing to one or more specific message channels. This property gives us many of the same advantages that drives dependency injection, for example improved testability or the ability to execute a component without having to instantiate all other components.
Even though Pipes-and-Filters architectures exhibit some of the same properties as dependency injection models there is one important difference. Dependency injection schemes are generally used to provide object references to a component. Often these references provide useful services to the component, such as persistence, transactional support etc. In the world of message message data might arrive in the form of an object but the message does not provide any services to the component -- it is simply a holder of data.
One of the nice properties of Pipes-and-Filters architectures is that they can be realized in many different technologies. For example, an implementation could use third party middleware a la JMS, TIBCO, MQ etc. Or it could all just be implemented inside a single JVM using a custom EventChannel class (our most recent project used this approach -- I'll share more about our experiences soon). In either case, this type of architecture enables dynamic composability, extensibility and great testability of individual components.
Gregor is a cloud architect and author. He is a frequent speaker on asynchronous messaging,
IT strategy, and cloud.