Understanding Enterprise Application Integration: The Benefits of ESB for EAI

• By Yokesh Shankar
• 26-04-2024
• Software

In today's enterprise infrastructure, application and system integration is an increasingly common and important concern. The wide variety of approaches and ideas aimed at achieving this goal are proof of this. When you start researching data and application integration solutions, it's easy to get lost in a welter of acronyms, opinions, and confusing marketing jargon.

Rapid advances in EAI technology to address the growing demand for enterprise integration often lead to discussions about what EAI is or is not, or how small differences between one brand's approach or another make it the only viable solution.

In this article, we will dispel the confusion around the evolution of EAI with a structured and easy-to-understand analysis.

We will start with a brief history of the origins of EAI, take a tour of the main advances in EAI architecture and discover how "hub and spoke" EAI systems based on traditional intermediaries are being replaced by enterprise custom software development service bus architectures agile, distributed and standards-based.

The origins of the EAI

Enterprise application integration, EAI, has been around as a technicality since the early 2000s, but the core problem it seeks to solve dates back much further. Simply put, EAI is a general category of approaches that provide interoperability between the disparate systems that typically make up infrastructures.

Enterprise architectures, by nature, are typically composed of numerous systems and applications, which provide services that the company depends on to carry out its daily activities. A single organization can use standalone systems, whether developed internally or licensed from a third-party vendor, to manage its supply chain, customer relationships, employee information, and business logic. This modularization is often a desirable aspect. In theory, dividing the task of running a custom software development company into small functions allows for easy implementation of the latest and greatest technological advances in each area, as well as rapid adaptation to changing business needs.

However, to reap the benefits of this modular and distributed system, an organization must implement technologies capable of dealing with the problems associated with this architecture:

• Interoperability: Different infrastructure components may use different operating systems, data formats, and languages, preventing connection through a standard interface.
• Data integration: For a distributed and modular system to be functional, it is essential to have a standardized method for managing the flow of data between applications and systems, to ensure consistency in the database.
• Solidity, stability and scalability: they are the glue of an infrastructure, so integration solutions must have these characteristics.

When point-to-point integration is not enough

Prior to the development of EAI-type approaches, integration issues were managed using point-to-point integrations. Under this model, a single connector is implemented for each pair of applications or systems that must communicate with each other. This connector handles data transformation and integration, as well as other messaging services, which must occur only between the specific pair of components for which it is designed.

When used in small infrastructures, in cases where only two or three systems need to be integrated, this model works quite well and provides a lightweight integration solution tailored to the needs of the infrastructure. However, as new components are added to the infrastructure, the number of point-to-point connections required to create a complete integration architecture begins to grow exponentially.

A three-component infrastructure only requires three point-to-point connections for its integration to be considered complete. But if you want to add just two more components, you have to increase the number of connectors to 10. We then approach a level of intractable complexity, and once the infrastructure includes 8 or 9 component systems and the number of connections increases In your thirties, point-to-point integration is no longer a viable option.

If we take into account that each of these connectors must be developed and maintained independently in the face of changes in system version and scale, among others (in some cases they must even be purchased from a supplier at a high price), then it is perfectly clear that point-to-point integrations are not suitable for complex business situations.

EAI's approach to integration

To avoid the complexity and errors inherent in integrating complex infrastructures through an end-to-end approach, EAI solutions use various middleware models to centralize and standardize integration practices across the infrastructure.

Thus, each application no longer needs a separate connector to link to the rest of the connectors; Instead, the components of an EAI-based infrastructure use standardized methods to connect to a common system that is responsible for providing integration, message delivery, and reliability functions to the entire network.

In other words, EAI solves the problem of modular system integration by treating it as a system task like any other, rather than a tangle of convoluted connections. EAI systems bundle adapters to improve connectivity, data transformation engines to convert data to a format suitable for the user, modular integration engines to simultaneously handle numerous complex routing situations, and other components to present a solution. unified integration.

EAI loosens the tight connections of point-to-point integration. An application can send a message without knowledge of the user's location or data requirements or the purpose of the message: all of this information is managed through the EAI implementation. Thus, the architecture is more flexible, and parts can be added and removed as necessary; you just have to change the EAI provider settings. In addition, modular development is simplified, with which a software development service can be reused by several applications.
Many modern EAI approaches also take advantage of the opportunity to add a central integration mechanism to further consolidate messaging tasks. In addition to data integration, a modern EAI also includes network management, security, acceleration, scalability, and more.

Traditional EAI

The first EAI solutions on the market started from the idea of ​​literally unifying integration, and incorporated all the functions necessary for integration in resource centers, called intermediaries.

In this section, we will discuss the advantages and disadvantages of this model, and discover why it has been abandoned in favor of the ESB architecture.

The intermediary model

In a broker approach to EAI, a central integration engine, called a broker, resides at the center of the network, providing all transformation, routing, and other cross-application functionality. All communication between applications must go through the resource center, maintaining data concurrency for the entire network.

Typically, implementations of the broker model also provide monitoring and auditing tools that allow users to access information about the flow of messages through systems, as well as tools that speed up the complicated task of configuring mapping and routing between large quantities. of application systems.

Advantages

Like all EAI approaches to integration, the broker model allows for a looser connection between applications.

In other words, applications can communicate asynchronously, sending messages and working without having to wait for a response from the receiver, always knowing exactly how the message will arrive at the connection point and, in some cases, the point itself. message connection.

The broker approach also allows you to perform all integration configuration within a central repository, which means less repetitive configuration.

Disadvantages

As in any other architecture model that uses a central engine, the intermediary can become a single point of failure for the network. Since the broker is responsible for maintaining concurrency between all data sets and application states, all messages between requesters must pass through it.

If under a heavy workload, the broker can become a bottleneck for messages. Being a single central destination for all messages also makes it difficult to use the broker model successfully over large geographic distances.

Finally, broker model implementations are often heavyweight, proprietary products intended to support a vendor's particular technology subset. This can pose problems if the integration situation involves products from multiple vendors, internally developed systems, and legacy products that are no longer supported by the vendor.

ESB: the next step in EAI

The EAI intermediary model was successfully implemented by some companies, but the vast majority of integration projects that used this model ended up failing. The lack of clear standards for EAI architecture, coupled with the fact that many of the early solutions were proprietary, made EAI products expensive, heavy, and sometimes not working as desired unless the system was quite homogeneous.

The consequences of these problems were amplified by the fact that the intermediary model made the EAI system a single point of failure for the network. If a single component stopped working correctly, the entire network failed. In 2003, a study estimated that 70% of integration projects ended up failing due to defects in early middleman solutions.

Bus architecture: a new approach to EAI

In an attempt to circumvent the problems caused by the intermediary "hub and spoke" EAI approach, a new EAI model emerged: the bus. Although it still required a central routing component to transfer messages between systems, the bus architecture sought to alleviate the functionality burden on individual components by distributing some of the integration tasks to other parts of the network.

Thus, these components could be grouped into different configurations using configuration files to handle any integration situation as efficiently as possible, as well as hosted anywhere in the infrastructure or duplicated to better scale across large geographic regions.

The birth of the company service bus

With the evolution of bus-based EAI, a number of necessary functions have been identified, such as security transaction processing, error management, and many more. The bus architecture did not require extensive coding efforts to integrate these functions into the core integration logic, as was the case in the broker architecture, but rather allowed them to be aggregated into separate components.

The end result was lightweight, custom integration solutions that ensured reliability, were completely abstracted from the application layer, followed a consistent pattern, and could be designed and configured without much additional code or modifications to the systems being integrated.

This mature version of this bus-based EAI model became known as the enterprise services bus, or ESB.

ESB Core Functions

Currently, there are different ESB products on the market. Some, such as WebSphere Message Broker or TIBCO BusinessWorks, are traditional EAI products that have been redesigned to offer ESB functionality, but still follow the broker model.

Others, such as MuleSoft's Mule ESB, have been built from the ground up using open integration and messaging standards to implement the ESB model.

Despite their differences, most ESBs include all of the core functions, or "services," listed below:

• Location Transparency: A way to centrally configure endpoints for messages so that the user application does not need information about the producer of the message to receive communications.
• Transformation: The ability of the ESB to convert messages to a format usable by the user application.
  Protocol Conversion: As with the transformation requirement, the ESB must be able to accept messages sent using all major protocols, as well as convert them to the format required by the end user.
• Routing: The ability to determine the end user(s) based on pre-configured rules and dynamically created requests.
• Improvement: The ability to retrieve missing data from incoming messages based on existing messages and attach them to the message before delivery to the final destination.
• Monitoring/management: The goal of an ESB is to make integration a simple task. By definition, an ESB must provide a simple way to monitor system performance, the flow of messages through the ESB architecture, and a simple way to manage the system to deliver the intended value to an infrastructure.
• Security: ESB security encompasses two main components. The first, guarantee that the ESB itself manages the messages in a completely secure way; and the second, mediate between the security guarantee systems used by each of the systems that will be integrated.

The advantages of ESB

These are the advantages that the ESB approach to application integration offers:

• Lightweight: An ESB is made up of different services that interact with each other, rather than a single resource center for all services, so it can be as light or heavy as the organization needs. That is why it constitutes the most efficient integration solution available.
• Easy to expand: If an organization knows it will need to connect applications or systems to its infrastructure in the future, an ESB allows it to integrate its systems immediately and not worry about whether or not the new system will work on the existing infrastructure. When the new application is ready, you just have to connect it to the bus so it works with the rest of the infrastructure.
• Scalable and Distributable: Unlike broker architectures, ESB functionality can be spread across a geographically distributed network as needed. Additionally, because individual components are used to deliver each function, using an ESB makes it much easier and more cost-effective to ensure high availability and scalability for the essential parts of an architecture.
• SOA Ready: ESBs are designed with service-oriented architecture in mind. Thanks to this, any organization looking to migrate to an SOA architecture can do so progressively, while continuing to use its existing systems and connecting reusable services as they are implemented.
• Progressive adoption: At first glance, the number of features offered by the best ESBs may seem intimidating. Therefore, it is best to think of ESBs as integration "platforms" that allow you to use only the components necessary to meet current integration needs. Most modular components offer unparalleled flexibility, enabling progressive adoption of the integration architecture as resources become available, while ensuring that future unforeseen needs will not interfere with the return on investment.