Integration

Things to consider with API (MicroServices) Integration architecture

There’s a lot of interest nowadays in microservices-based integration patterns. This appears to be nothing more than an evolution of SOA – “SOA done right” as some call it. The services are lighter (usually REST/JSON based), smaller/more atomic (hence the “micro”), and built on managed, scalable architecture. Here are a few things to think about if you are considering moving to an API-based architecture for your Application Integration needs.

Coupling and Cohesion

One of the critiques of SOA is that it ended up creating large monolithic services (coarse-grained services as they were referred to at times) that ended up becoming hard to manage and scale over time. The idea with Micro-services is to break that down into a smaller set. That doesn’t necessarily mean that you are exposing the basic CRUD. It just means that you try and create services that don’t depend on each other as much as possible. If one service changes, the impact on other services should be minimal. The basic idea here isn’t new. Good OOP has always dictated it.

Versioning

The basic idea here is to make sure that new versions don’t break older contracts of the same service. Services are usually versioned in the namespaces. This course is not relevant when the interface is changing significantly – in that case, you will need to negotiate a migration with the existing consumers.

Homogenized Semantics

As with SOA, you can publish interfaces any way you like them. However, as you rise in the integration maturity matrix, the services standardizes in terms of semantics. e.g. a “customer” should look at the same (same structure) across any service that reads and writes to it

Monitoring and logging

One of the key challenges with a microservices array is monitoring and logging. figuring out what went wrong and where and how to react to it. Another key challenge is correlating transactions across services. e.g. Order #123 as it flows through the account creation, customer creation, and order creation service. These three might be running on separate servers (even hosting providers). This is particularly complicated because of the lack of standardization there. The two key approaches here are centralized logging vs. log shipping.

Build and deployment automation

Though not strictly a subject of microservices, it is very common to automate the build and deployment of services using deployment tools visual studio online’s continuous integration tools. These tools will automatically unit-test and deploy code once it checks into source control and then deploys it to multiple servers to keep the system up to date. This is key because all the micro-services become unmanageable if a central tool isn’t orchestrating the operations of the services.

Sync vs Async

One key consideration from a service design point of view is whether you should have the caller wait for a response or leave and notify later. Async is more complicated to implement but doesn’t hogg resources so is preferred in many cases. The other thing with Async is to what extent it remains async. You can be wrapping asynchronous communication with a synchronous service wrapper or vice versa. Async implementations usually end up as events published on a pub-sub engine.

Other things to consider are retries and availability. With sync calls the caller has to wait till he receives the response. With Async, the sender may not find the caller at the time of the response. In that case, engines should retry a few times before giving up.

Virtualization and containers

Virtualization and its mini versions (containers) uses to scale out and control resources allocated to services. You will generally have one micro-service run in a container that scales out faster. Cloud platform support for these is pretty common. You are looking at things like Docker,  Rocker, etc.

Scaling

The idea with containers and Micro-services is that as certain bits of functionality in the enterprise are more frequently used, you can scale those out separately on demand. This scale-up and out can vary depending on demand (e.g. scaling out the order service during the Christmas season)

Caching

This is all about caching response for data that doesn’t change frequently. The key factors to consider are the freshness and velocity of data.

Registry

Again, the idea here is not new. it’s the same thing that UDDI was trying to do in SOA however with Microservices the platforms seem to have matured. The registry uses not just as a central repository of services for developers but server-side technologies to update with the availability of services. When they are down, the fact can be published to other services and administrators to be managed accordingly.

I hope this gives you some thinking points if you are planning out a new integration for your organization. How these end up looking exactly is something heavily personal to each enterprise and is usually developed with an interactive discovery engagement.

Author

Muhammad Omer

Muhammad Omer is the founding partner at Allied Consultants. Areas of interest for him are entreprenuership in organizations, IT Management, Integration and Business Intelligence.