The ESB .. Thoughts - Part 1

So here I am after a*very* long time writing about some of the things that have been going on in my mind. Like I mentioned before, I work for a niche consulting company focused around Financial Services sector in the Boston area.

Time and again, this question of the service bus pops up when consulting for my clients. I still havent formed my opinion on whether this is a new concept, evolution vs revolution, nirvana, panacea .. etc etc. The developer within me keeps screaming - "Yea right! Another broker based integration technology..so whats the news?" However, the architect side says - "wait..there definitely is some value in this component".

I do think that it is important to weigh the pros/cons, architectural requirements and capabilities needed for the particular problem domain to figure out if this component is a right fit. Some of the capabilities that I found useful was

- Applying security outside of the application logic
At one of the pharma clients that I worked at, I created an SOA security architecture where we implemented the WS-Security standards outside of the actual application. Although we were not using a traditional service bus component, I understand that this function can be easily embedded into the service bus.
- Reliability of messages (and here I refer to messaging based service bus components)
At the same client, we implemented a store and forward type pattern where persisting the message was greatly important in case the reciever was temporarily incapable of receiving the message
- Message transformation to canonical formats
At another client, there was a scenario where incoming requests were of various formats (due to legacy reasons) and we developed a custom component (based on XSLT transformations) which normalized the various requests into canonical formats. We leveraged the same component to do the same with responses. Sometimes the responses were generated by different endpoints in different business units with different standards! Ultimately, the clients made a service out of this component, much like a translator/mediator between the consumers and providers.
- Avoiding point to point connections
From the CORBA days, a message broker architecture is infinitely more scalable than point to point connections..which is sort of back to the previous bullet. Having a level of indirection is extremely useful, especially over a period of time when new endpoints are added on both consumer and provider ends.

I am sure there are other capabilities that people have found useful from an integration perspective. So if all such capabilities were bundled into one component, it would probably be quite useful.

In the interest of keeping the post small - I will stop here and continue the discussion later. Specifically I do have thoughts on questions that clients have asked me:

How much time does it take to implement a bus?
Can I start without a service bus and then add it later? What would I need to change?
Do I need to have an organization/governance change around the bus?

I would love to hear from others on these thoughts too..