Hi, I am Gregor Hohpe, co-author of the book Enterprise Integration Patterns. I work on and write about asynchronous messaging systems, distributed architectures, and all sorts of enterprise
computing and architecture topics.
Find my posts on IT strategy, enterprise architecture, and digital transformation at ArchitectElevator.com.
I am a very visual person. I enjoy sketching and holding facilitated workshops where I draw system diagrams on a whiteboards based on user feedback. My interest in this type of work actually inspired me to create the visual language that accompanies the pattern language in Enterprise Integration Patterns. So you can figure that I am a strong supporter of diagrams and visual notations. However, I see many of the visual editors embedded in integration suites (and increasingly in IDE's) as a double-edged sword. Let me explain...
One of the big benefits heralded by most EAI vendors (and Web services vendors) is the ease of use of the visual development environment (disparagingly called "doodleware" by hard-core developers) and the resulting increase in productivity. These suites typically include visual transformation editors, workflow or orchestration editors, schema editors, component assembly tools and the like.
I believe that graphical representations are a great way to visualize the system architecture and dependency of a complex system. Particularly in loosely coupled, asynchronous systems it can be very difficult for the developers to maintain a "big picture" view of what is going on in the system. A handful of components communicating via publish-subscribe channels or message queues is usually sufficient to cause confusion and make testing and debugging difficult. In these situations, a visual representation of the data or control flow can be invaluable. Since manually created documentation is bound to be outdated and therefore inaccurate, automatic creation of visual documentation can be enormously useful. This realization was one of the main motivations for me to create the TibDoc tool that analyzes the TIBCO middleware repository and automatically renders the system data flow as a diagram.
However, I am skeptical whenever visual representations are positioned as programming tools, essentially replacing our trusted code editors. These presentations are usually accompanied by claims of increased productivity, a reduced learning curve, or independence from expensive integration developers. I believe that the difficulty of developing a solution is primarily tied to the complexity of the underlying programming model. For example, transformations from one document format to another are usually based on a functional programming model, for example XSLT. In order to build such solutions, developers have to understand concepts like templates, regular expressions, or complex functions. Putting a visual veneer over such a programming model is unlikely to suddenly enable business analysts to create complex transformations without any technical help. In some cases, the visual tools even can make developing and debugging more difficult because the user is yet another step further removed from what is actually happening inside the system.
All too often such a visual veneer is used in management demos to demonstrate the apparent ease of use. Instead of being lured by pretty pictures, I advise clients to evaluate the ease of use by asking the vendor to show the realization of common design patterns using the tool. For example, a frequent design pattern for data transformation is a Content Enricher or a structural transformation from a system with a many-to-many relationship to a system with a single entity. A great pattern to assess the capabilities of an orchestration or process modeling tool is the Aggregator because it occurs frequently in real life and is inherently event-driven and stateful.
In some cases, visual notations are very natural and make developing solutions more intuitive. For example, UML activity diagrams are a great way to depict multiple parallel activities with fork and synchronization (join) points. This is something that is much harder to express with line of code which are naturally linear. Therefore, an activity modeling tool can be a very worthwhile addition to a development suite. However, it is important that the model is seen as an addition, not a replacement for actual code. Too often I have seen very pretty visual editors, but dare you have to write a line of code! Coders get a multi-line edit box with proportional fonts and no syntax highlighting. An effective development tool must support the creation of components or activities using state-of-the-art code editors, combined with visual tools for component assembly or orchestration development. This includes syntax coloring, version control, good print features, visual diff tools, automatic code completion, refactoring and other features that we have come to love from successful IDE's such as IntelliJ, Eclipse and Visual Studio. In my opinion, code-free integration is a vision and not a reality. So until we get there, please give us developer-friendly code editors!
Sometimes I actually think that it would benefit developers and vendors alike if EAI vendors stopped attempting to create their own IDE's and instead developed Eclipse plug-ins. This way, the vendor can leverage an existing open-source infrastructure for code editing and can augment it with proprietary plug-in for transformations, orchestrations or metadata editing. If you have seen the beta version of the upcoming Microsoft BizTalk Server 2004 release you can see that Microsoft is heading a similar, if a bit more proprietary route. While integration with Eclipse may be an unlikely step, the new BizTalk editors plug into the Visual Studio .NET IDE, so that designing an orchestration or transformation is done in the same environment as writing C# code.
After some poking around I found that the term "Doodleware" was indeed coined by ThoughtWorks' own Jason Borro.
Gregor is a cloud architect and author. He is a frequent speaker on asynchronous messaging,
IT strategy, and cloud.