Remember B2B integration using EDI? And later using Rosettanet, OAGIS, HL7 and a whole lot of other standards that gets large global supply chains to work. I’v worked a lot in this specific area and very keenly follow the B2B integration landscape to see what the latest trends and approaches are. “SOA” during the last decade and “cloud” now have completely taken the thunder (if there was any ever) from industry standard based B2B integration. Search for a article on EDI or Rosettanet on google.com and check the published date of the top results. It is obvious that B2B integration is not sexy any more.
But really what does standards based B2B integration solve, and do SOA, cloud or any of the othe emerging technologies offer a better alternative solution to the problem space? Are these new technologies completely unrelated disciplines which unfortunately encroach upon a limited skill-set and IT budget? Or do they make in some way make standards-based integration obsolete, a thing of the past? Or is standards based B2B-integration something that has its own secure space and will gradually evolve at its own leisure pace, unperturbed by all the SOA and cloud madness around? Or is it a solved problem? Here’s my thoughts on some of these questions.
Firstly, B2B – integration itself is a vague term. What does it try to solve? At the most abstract level it simply enables two business to establish a synergy and achieve a “whole” greater than each of their respective individual “part”s. In a supply chain, it typically consists of various links collaborating and sharing information effectively to help achieve greater efficiencies and profits. IT is simply an enabler, the driver being the realization that businesses at various stages of the supply chain can cooperate and gain more by striving for greater global efficiencies over trying to optimizing individual profits in silos.
A key requirement of successful B2B integration – is seamless flow of information between the partnering businesses. This is where the IT aspect kicks in. IT can enable successful B2B in several ways. Shared databases, portal integration, message based integration, good ol’ pen, paer and fax machine + document management systems, all or any of these can help two businesses desiring to collaborate, exchange the information required for successful B2B integration in a efficient or timely manner.
Of all of these, message based B2B integration is of high interest to me. The idea is simple – two businesses willing to exchange information, do so using pre-defined message formats. However the challenge lies in defining these message formats. A supplier sells to multiple retailers or distributers. A retailer recieves supplies from multiple suppliers and distributers. If every supplier sends information in a custom message format, the retailers IT system will need to handle hundreds and thousands of message formats and transform them to a common format which the retailer’s internal IT processes can receive. Translation of messages from one format to another is a manual, labor intensive and thus expensive process. Therefore you need some predefined, standard message formats and associated semantics that all suppliers and retailers (or any two links in a supply chain) can agree upon – so that the need for transformation can be eliminated. This was the promise of industry standard messaging formats such as X12, EDIFACT, Rosettanet, OAGIS, etc etc. They defined libraries of message definition and their usage scenarios and as long as the two business followed a common standard there was no need for any transformation of the message exchanged between them.
So was it all hunky-dory? Not exactly. Defining standards is not an easy job at all. Firstly, competing businesses do not like to agree upon anything. Even the cost benefit of a standard messaging library is not sometimes enough for businesses to make concessions required for a common ground to be reached. Also, on a more abstract level, the nature of information required to be exchanged between two parties for completing a business transaction depends upon the context within which the transaction instance is created. No two such transaction context are 100% identical. Hence the information required by A & B to integrate can be different from the information required by C & D. A common messaging library that address the integration needs of A-B and C-D are is going to be super-set of the messages, message-defintions, used in both the contexts. Extrapolate this over a entire industry, the standard becomes bloated. When a standard message library and each individual message definition within the library becomes bloated – every integration scenario ends up using only a very small subset of the standard. Each subset becomes unique again. Transformation is needed to transform various subsets of this standard into a unique format that the IT systems of a single business can process. The cost-benefit of standards based B2B starts to lose its sheen.
Inspite of the above problem, having standard message-libraries makes B2B integration a lot easier than by not having standard message-libraries. The transformation from one subset of the standard to another is a comparitively easier task than transforming messages encoded in random custom formats. Eventually every closely related group of businesses agreed upon a common subset of the standard (often called variant in the EDI world). EDI based B2B integration became a norm. It came with high CAPEX but relatively low OPEX. It took significant cost and time to build standards based messageing B2B interfaces for your businesses. However, once the IT systems in your business were capable of sending and receiving messages in a certain standard (or variant / subset of the standard) you pretty much could do B2B integration with any business which supported that standard at relatively low additional cost.
Several standards emerged, each targeted for specific industries and verticals (integration contexts) and businesses invested heavily in enabling their systems for messaging using the standards they were interested in.
However a lot of companies could not afford the costs required to standards enable their IT systems. An alternative approach for B2B integration started gaining traction. B2B integration hubs emerged which supported any-to-any integration. These hubs typically operated in a per-transaction-cost model. There hubs allowed businesses to send and receive messages to multiple trading partners using their own message formats. However the hub would take care of transforming the messages to a format required by the recipient. The transformation was provided as a service. Each hub supported multiple such sender-receiver pairs and could reduce the cost of integration for any individual business (spoke system) as result of efficiency of scale. They had the expertise required for transformation and supported out of box integration for interfaces of various ERP packages that made up the IT landspace of the businesses participating in the integration. This was the B2B integration as a service. Or B2B in a SaaS model way before “SaaS” gained popularity in the post-cloud era.
The shared database approach also gained some transaction. Instead of each business exchanging messages every other business global virtual database could be updated with a shared information – and subscribers good fetch relevant information from this shared information source. This option works only when there is common information that is of use by multiple subscribers (e.g. product, symbol or catalog information) but not for information that is used by just 1 or 2 parties (e.g. orders, invoices). The GS1 driven global product data hub is a leading example of how a shared database can be used to solve certain B2B integration requirements. You still required all the businesses which needed to update or read this shared database to support the common API exposed by the shared database. So standard based messages were still needed.
Thus, B2B integration based on standard message libraries + messaging hubs power global supply chains.
Now coming back to the pith of my rant – do any of the emerging technologies challenge message based B2B integration described above?
Lets get the easy ones off the list first.
Social does not really touch this problem space at all. Social has a lot of significance in the B2C space – with social platforms providing a new medium for business and customers to interact with each other. However two businesses needing to exchange information to minimize inventory – will not use social media to exchange information. Not the way things are today.
BigData – again really has nothing to do with the B2B integration problem. Big Data deals with the challenges arising while processing large data sets resulting from internet scale applications and automated devices. It does not deal with specific near-real-time information exchange between partnering business entities.
Mobile – again similar to Social is another channel of communication between businesses and its consumers. Mobile platforms will not be used by business partners to exchange structured information. However it is possible that certain tighly integrated business partners will use mobile and the IoT (internet of things) to get real time visibility into each others business processes. But this will be true only for a very small & insignificant fraction of business partners who are using message based B2B integration. Atleast in the next few years.
SOA & Web Services – The general impression is that Web-services based SOA solves all integration problems. Let us put it in context of the B2B integration problem. Assume every business has its SOA WS-* interfaces. For my business to integrate will all my partners I will need to be able to support the WS-* interfaces of all my partners. If each partner has its own unique WS* interface I will need to transform request and responses from my IT systems to the formats defined by the WS-* interfaces of each of my partner businesses. My business will need to develop and maintain 100s and 1000s of transformations. So the intial problem that was solved by B2B messaging remains un-addressed. However with a strong SOA backbone in place you can unlock more enterprise functionality for all kinds of uses including information sharing with your trading partners. So SOA does help B2B in that way.
Finally, Cloud – At some level one might feel that cloud eliminates the need for B2B integration. If every business’ IT system is in the cloud, then theoretically there is no need for businesses to speak to each other directly. All communication happens in the clouds. Unfortunately there is not one cloud. We have public, private and hybrid clouds. We have SAAS, PAAS and IAAS too. If you are doing PAAS or IAAS then your business IT still has its own applications to manage and thus is responsible for integration with other businesses. You still need messaging standards to do that integration. The integration may be hosted on the cloud, but it still needs to be built and maintained. No pennies gained in solving the B2B integration problem.
However if your business uses SAAS for its IT. Then the SAAS provider can provide pre-buit integration support for a lot of popular B2B messaging standards. So you can just sign-up for a transaction/usage based payment model and have your IT enabled for B2B integration. However the pre-condition here is that your SAAS provider supports the B2B message standards that your partners require for integration. (it really does not matter whether your partners are running their IT in-premise or on the cloud.)
In case your SAAS provider does not support the required B2B messaging standards (and this is mostly likely the case), then your SAAS provider will need to partner with a B2B messaging hub which provides standards based message transformation as a service. So your SAAS instance can be configured to exchange information with the B2B messaing hub, which in turn can do the heavy-lifting of integrating with your business partners using standards based B2B messaging.
So yes, SAAS + B2B messaging hubs do offer a disrputive alternative to standards based B2B integration in its current form. However till a consolidation of SAAS vendors and B2B hubs happen, and while majority of business continue to rely on in-premise IT, standards based messaging such as EDI and various other XML business messaging standards will continue to drive B2B integration.