Challenges of a microservice architecture

What is a Microservice Architecture?

As the name suggests, microservices break up an application (called a monolith) into many smaller parts. Each of these microservices are then deployed and run separately. This practice originated at Amazon where the idea was to break a large monolithic application into what Amazon’s CTO Werner Vogels called “mini services”. The reason for this was to make development cycles faster, letting teams choose their tools and infrastructure, with the thought smaller teams will be more agile and that if there are well-designed interfaces between the microservices, it will all work well. 

The rise of microservices, along with the devops, came with the promise of agility, but had also created a lot of complexity, lost productivity, cognitive load and inefficiencies. These are where the promise of microservices also creates many challenges. Before we go into the challenges, let’s examine the benefits of a microservice architecture.

What are the benefits of a microservice architecture?

A microservice architecture releases engineering organizations from the lengthy deployment cycles that are part and parcel of having large application monoliths. They allow work in smaller teams with greater flexibility and choice of tools, and as a result, change happens faster.

If done well, the benefits of a microservice architecture are as follows:

• Faster time to market: microservices architectures are more agile, since it’s easier for a small team to develop and understand a microservice, by virtue of its size and relative simplicity. Microservices are also easier to deploy and manage. As a result, once the move to a microservice architecture is made, teams that they ship software faster.
• Better engineering resilience: with a microservices architecture it is easier to isolate problems down to a specific microservice, reducing MTTR and making on-call triage much easier.
• Better engineering practices through reusability of elements and clear stadnards for production readiness, AppSec and how the microservices connect
• Better scalability, since microservices run separately and scaling can be done in a more efficient manner.
• Better innovation through agility and speed.

However, microservices also pose some significant challenges that are not trivial. Let’s examine them.

Common challenges of a microservice architecture

In general the fact that microservices are distributed and that teams have autonomy over how they are developers and deployed, poses a set of challenges. There are many potential breaking points, from the inability of other teams to understand which microservices are available and how they are connected, to issues understanding dependencies and interfaces between the microservices, issues related to environments, adherence with standards and more. 

A microservice architecture may be more challenging for developers. 

• There is a knowledge gap. If developers are required to be aware of more technologies for the delivery of microservices, such as K8s knowledge, an understanding of production readiness, managing cloud costs, feature flags and more, the fact that they don’t know all that and that there are no golden paths leads to loss of productivity.
• There is a discoverability issue: if developers can’t find microservices, standards, APIs, cloud assets, or know what lies where, they may waste time just understanding what is going on, as well as potentially duplicate work unnecessarily.
• There is a devops or ticketops issue: If the microservices architecture means that DevOps, SRE or other support is often needed, ad-hoc and that it takes time to be done, developer velocity is adversely impacted. 

In short, microservices architecture challenges can be summarized as follows:

• Code duplication and redundancies: This issue arises when similar or identical code is scattered across multiple parts of the distributed application, leading to increased maintenance efforts and higher risk of inconsistencies and bugs as well as duplication of wrong or faulty code.
• Developer cognitive load: Excessive cognitive load occurs when developers are overwhelmed by too many simultaneous demands on their attention and memory, related to navigation of cloud native microservice environments. 
• Developer lack of knowledge: Developers may struggle with insufficient understanding of the codebase, tools, or technologies they are working with, which can lead to inefficient problem-solving and slower development cycles as well as standards non compliance.
• Difficulty upholding standards and compliance: Ensuring that all code adheres to organizational standards and regulatory compliance can be challenging, especially in large teams or complex projects, potentially resulting in reliability issues, a poor Appsec posture or cost overruns.
• Repetitive manual DevOps work: Many DevOps tasks are manual and repetitive, such as setting up environments or deploying applications, which can be time-consuming and prone to human error, reducing overall efficiency.
• Inability to understand dependencies: Without clear visibility into the dependencies between various components of a system, developers can struggle to predict the impact of changes, leading to integration issues and system failures.
• Taking too long to understand and search for data: Developers often spend significant time searching for the right information or understanding complex data structures, which slows down their progress and hampers decision-making.
• Security posture degradation: Security can deteriorate when there is a lack of consistent enforcement of security practices, leaving systems vulnerable to attacks and data breaches.
• Reliability issues: Maintaining system reliability is challenging when there are frequent outages or performance problems, which can erode user trust and lead to loss of revenue.

Overcoming microservices challenges with internal developer portals

The challenges presented by microservices architectures are prevalent in many engineering organizations. DevOps promised agility that would solve many of these challenges and enable microservice architectures, but in real life, developers can be overwhelmed.

Platform engineering is about creating reusable self-service actions for developers. The core idea around platform engineering is the creation of developer self-service golden paths which are a simple and opinionated way to perform actions such as scaffold a microservice, change it, delete it, create an ephemeral environment, ensure Appsec compliance and more.

The internal developer portal is how developers access the internal developer platform - it’s an interface and central hub for developers and managers, letting them:

• Perform self-service actions
• Understand the enterprise architecture
• Adhere to standards and enforce them
• Track engineering metrics and standards compliance with dashboards

Here’s a deeper dive of how the different pillars of the internal developer portal help overcome these challenges:

Golden paths

Golden paths: the internal developer portal supports developer self-service, created golden paths for developers that abstract the cognitive load associated with many actions, from setting up temporary environments, asking for database access, scaffolding a microservice and more. A golden path is an opinionated and simple way to perform an action, which will make the action standards compliant and reduce cognitive load. Golden paths are also highly relevant for day-2 operations on microservices too and are even relevant to requesting just in time permissions.

Microservice catalog

The Microservice catalog is at the center of the internal developer portal. It essentially consolidates all the microservice architecture information, from services, through CI/CD, AppSec and cost data, cloud resources, databases and more. Since it usually contains much more than just microservice data, it is really a software catalog. To make the microservice catalog useful for developers it’s important to not dump all the aforementioned data into the portal’s catalog, since it won’t relieve cognitive load but rather increase it. The idea of abstracting the data for developers is important, since we need to show them what they need in a way they can use, from showing them API catalog data to kubernetes standards compliance and more. 

Scorecards and standards compliance

One of the big issues around microservices architecture is standards compliance. If we let developers run loose they may be quick but standards won’t be met. On the other hand, if we force developers to wait for SREs, AppSec reviews etc, they will be slowed down. The internal developer portal - and the software catalog at its center - are uniquely positioned to solve this conundrum. Not only do developers set up software catalog entities with golden paths, which have baked-in standards, but they can also track standards over time, using scorecards, which can be set for any software catalog entity, not just services. The result is an ability for managing standards for microservices. The result is that instead of tracking standards manually, a system for alerting on standards degradation and setting up initiatives can be set up quickly, saving time and frustration. 

Observability and dashboards

Internal developer portals also include dashboards that serve different personas, from SREs through DevSecOps, FinOps and engineering management. This provides personalized experience that show metrics, relevant self-service actions, context, standards and more. 

Engineering metrics

Dashboards can also be used to track engineering metrics, containing both the classic software engineering metrics made available in software engineering intelligence platforms as well as DORA metrics and SPACE. Additional metrics can be associated with understanding developer workflows as well as general standards and productivity metrics.

Internal developer portals are the mainstay of platform engineering and are set to be the future of how developers work. Our live demo can show you more use cases of how to use portals and benefit from them.