Microservice Architecture, a non-introductory introduction.
This article is for you if: you know what microservices are, have experience with them, and thus know, how difficult they are to develop and recognize that you can get better and it.
For simplicity, these are the characteristics of a Microservice Architecture:
- The application is segmented by data domains.
- Each service is the primary-owner of its data.
- Business-logic sharing is discouraged between services.
- Each service has its own CI/CD pipeline and hosting infrastructure.
Hexagonal Architecture.
Traditionally, services are shaped by the 3 tier Architecture (Presentation Layer, Business Logic Layer, Data Access Layer), or the MVC pattern (Model-View-Controller), both were designed to serve one client application from a single database.
A generalization of this is the Hexagonal Architecture, which is better suited for handling multiple sources of input and output data. This means supporting multiple applications (web, mobile, public APIs, admin dashboards, etc.) but also communication with other services.
Look in the below image, how business logic is isolated from ports and adaptors, that handle communication with other services.
In a well-designed hexagonal service business logic is agnostic on who requests the data.
This maintains a consistent data schema and prepares the application for scalability to support additional clients. Note in the picture, the asymmetry between inputs, and outputs, this is as a reflection on how thinking about persistence has evolved. Now, sources of data include temporal streams, ETL pipelines, and a variety of NoSQL databases.
Hexagonal Architecture captures that in any given service, the same data could be either an input or output.
Basic Microservice Design Patterns.
Object-Oriented Programming and Microservices are bounded ideologically by Domain-driven design, which states that the structure and language of the code should match the business domain.
In the same regard as OOP, proven patterns to manage data among services are well developed and understood, to illustrate a few:
SAGA Pattern
Used to “replicate” ACID transactions with data distributed among multiple services. This is done in a series of steps, each one handling a transaction in its own data if any transaction fails there are steps to reverse the previous transactions and restore the application’s state.
This is usually handled by an orchestrator that coordinates the steps, however, beware, that the I in ACID (Isolation) will hardly be ever fulfilled, as there is a small chance that between steps any of the future or past records could be modified.
API Composition
A basic pattern to query multiple services, and “join” or aggregate in-memory the various data sources, not the most efficient method, but surely the simplest one.
Command Query Responsibility Segregation (CQRS)
A more complex pattern to query data where a particular service is developed exclusively to answer queries, this can be done through data replication, but again beware that data might have various degrees of truthfulness.
API Gateway
In this pattern, there is a middleware between the data consumers (clients) and microservices. The Gateway handles common operations as authentication and data presentation, in more advance usages it also aggregates data from the requested services.
For perspective, each API request in Netflix queries between six of seven microservices, and later, we will explore open-source tools to implement this pattern.
Event Sourcing
To understand event sourcing I quote one of Redux’s three principles:
The only way to change the state is to emit an action, an object describing what happened.
In essence, mutations of data can only happen through events that are stored in a “ledger” this adds the ability to recreate the state of a given service at any time and presents a disciplined way of mutating data.
Single Table Design
This patterned championed in DynamoDB leverages a clever design of Primary & Secondary Keys to have access to the full data from a single table. Normally querying capabilities are extended via data replication and assuming data is rightfully modeled and the models are stable enough, data management is grossly simplified.
InterService Communication & Other Nightmares.
Synchronous vs Asynchronous
REST APIs used to be the cool kids on the block, but with Kafka and the revolution of data streaming it is steadily losing ground. Asynchronous messages increase performance, lowers costs, and simplifies communication between multiple services, all at the expense of embracing a new paradigm and adding a layer of complexity to your infrastructure.
Service Discovery
Why Service Discovery? Well, with your application rapidly changing and to prevent breaking changes in the multitude of clients, naming service’s URIs will become creative. At some point, you will probably need some help to track all the changes, a whole gamma of alternatives are at your disposal.
Testing
Hexagonal Architecture will simplify writing Unit Tests for your business layer but do not count on doing the same to inter-service calls. Testing Microservices is a whole separate subject but be ready to sharpen your Integration Testing Skills.
Deployment
Canary deployment is the preferred way of shipping code and whenever given the option for it, do go with it.
Open Source tools to get started.
Netflix and Go are two leaders in developing microservice open-source tools & projects.
From Netflix, I recommend Spinnaker for managing cloud deployments, Zull to provide gateway service, Vizceral that while unmaintained provides great visualizations of microservice communication and possibly Falcor and Eureka for serious practitioners.
From the Go community, I suggest getting started with Prometheus a time series database primarily used for monitoring, Traefik a reverse-proxy and load balancer for deploying microservices, and FAAS & Go Micro both frameworks for developing and deploying microservices.
Attribution
Thanks to Chris Richardson for his wonderful book and Rick Houlihan for his very inspiring video-lecture.
