Biztalk
Application Integration

Application Integration – food for thought

Here are a few basic tips some of the team came up with that helped BizTalk beginners comprehend the EAI (Enterprise Application Integration) domain, especially in a Microsoft BizTalk context.

  1. Integration projects don’t follow ‘water fall model’.
  2. Most basic need of integration is; ‘ messages generated from one application Application Integrationmust be sent to another application and vice versa.
  3. None of the transports, is suitable for all integration scenarios, because each was defined for a particular problem: HTTP for web, MQSeries for reliable queuing & BAPI for SAP.
    On other hand, core advantage of web services is to provide a singular, consistent & normalized transport stack that is built on open standards. Just as goal of XML is to reduce message complexity, goal of web services is to reduce transport complexity.
  4. Integration of two applications require a transport and manipulation of data. Another core requirement for integration services is to manage security issues, including transport and message-format security.
  5. Transport security is often independently ensured depending on the type of transport. Message-format security requires message encryption, with addition of fundamental need of authentication of source of message. Integration services must manage the complexity of multiple authorization schemes in an automated manner to securely connect two applications without human intervention.
  6. Integration architecture Tier – 1 : Connecting two applications
    Integration architecture Tier – 2 : Connecting multiple applications
    Connecting multiple applications point to point requires n*(n-1)/2 connections and forms a integrations spaghetti. Spaghetti integration needs to be converted to a simpler linear integration.
    Integration architecture Tier – 3 : Business Process
    By codifying notion of BP, there are significant benefits in ease of management, modification, tracking and troubleshooting, as well as business-level visibility.
  7. Centralized integration: places a requirement on integration services to be able to receive messages from source application and deliver them to multiple target applications, called message routing.
  8. Message Routing: can be done by targeting application, e.g. if app A sends a message, route it to app B. Another way, is to isolate applications further and target applications on basis of type of message received, this approach makes system very flexible, since replacing a single application won’t affect others.
  9. Publish & Subscribe: Routing of messages based on metadata properties is called publish & subscribe.
  10. Tracking services: Visibility on what messages are passed between applications is important to handle exceptions & provide aggregate analysis across multiple connections.
  11. Integration architecture: There are two stereotypical approaches for instantiating integration services on hardware.
  12. Hub & Spoke: is centralized processing machinery, hub that accept requests from multiple applications, the spokes. Hub manages routing, mapping, and tracking of messages between applications. It provides strong total cost of ownership. But hub models’s scability is limited by scalability of central machine resource. Likewise, central machine was single point of failure.
  13. Message bus: consists of network of message-processing functionality interlinked through a common bus-specific protocol. Although, it reduces connections between applications to a linear factor, it provides high-throughput multicast capabilities utilizing an Ethernet-based network of nodes. But this models comes with issues, such as management of nodes, routing of message between nodes. Per application integration costs for message bus are greater than hub and spoke because each application that talks to message bus needs to adapt data from its own format to proprietary bus format and transport. Logging & monitoring bus model brings with it, immense amount of complexity.
  14. Hub-Bus hybrid: Given that neither hub, nor bus provides 100% solution, an ideal solution is combination of two technologies.
  15. Integration scenarios applicable to internal integration, are also applicable to external integration. But external integration requires much more transport complexity, since different organizations with different protocols.
    Also, when connecting to trading partners, business often have little capability to demand that a message should conform to specific format or demand a specific transport.
  16. State management: At heart of business process management is state management. State is commonly shared by entities within a business process. A key challenge with managing state is duration of business processes. BP are often long lived, and managing state for a significant period of time generates a number of technical challenges. Leaving state in memory is not scalable.
  17. BP vs BO: BP shouldn’t replace BO themselves; rather, they should maintain state, and in doing so, manage transitions between steps of BP at a macro level.
  18. Transaction management: Traditionally, a transactions are handled by resource controllers ( eg MSTDC ), to provide data isolation. For business processes, additional requirement spring up, because of their long-running nature.
    For BPs, transactions don’t mean a rollback, if a process is aborted for some reason, instead it’s trigger of another mini-process called ‘compensation block’.
  19. Business activity monitoring: Benefit from codifying BPs is that tracking infrastructure can be layered on top of B to collect business data. e.g. how long does it take for PO to get processed. This data can be analyzed on per instance or aggregated view to answer interesting business questions.

Author

Muhammad Omer

Muhammad Omer is the founding partner at Allied Consultants. Areas of interest for him are entreprenuership in organizations, IT Management, Integration and Business Intelligence.