Solving service sprawl: It’s time to replace the CMDB with a service catalog (and internal developer portal)

Solving service sprawl

There is a price to be paid for the elegance of cloud native microservice architectures: sprawl. We’ve seen this problem emerge with APIs, causing the emergence of API catalogs and API management, and now the same problem repeats itself with microservices. 

This inability to see how services, APIs, CI/CD and resources are all connected isn’t just a problem for developers. DevOps are also burdened by this complexity, and engineering in general is concerned this will reduce productivity or result in unnecessary duplication of services/APIs and non-adherence to standards.

Some people have a solution to this: a spreadsheet with microservice data. In fact, in our 2024 State of Internal Developer Portals showed that while 85% of respondents reported using an internal developer portal or planning to add one, their explanation of what consists of an internal developer portal varied.  53% reported using a technology that can pass as a portal, while 35% reported using spreadsheets with microservice data.

While it’s pretty obvious that you can’t sustainably use spreadsheets as a service catalog, we’ve also seen instances where engineering organizations tried to use Configuration Management Databases (CMDB) in order to track their microservices. 

The TL:DR that just like spreadsheets can’t be a good service catalog, CMDBs can’t be a service catalog either.

If you want to learn why, read on.

Why Spreadsheets don’t work for microservices

Turning to spreadsheets as a makeshift service catalog might seem like a practical, low-cost solution. Why bother implementing a new product category - service catalogs - which is the number one best devops tools in 2024?

To understand this, here are two examples we’ve seen in real life:

1. Using spreadsheets to manually track production readiness (and therefore ensure less incidents and compliance with engineering standards), something you’d do using the scorecards function in an internal developer portal:

2. Using a service catalog to track cloud costs:

As you’ve already guessed, using spreadsheets to track service catalog data has significant drawbacks:

• It’s time consuming
• Error prone
• Static - information such as vulnerabilities or health metrics is dynamic by nature. It changes all the time and therefore becomes stale and unactionable almost immediately
• As with anything in a spreadsheet, you’re bound to have numerous version control issues as different developers update different metrics for different services, all in one spreadsheet

The result? no single source of truth: Since the spreadsheet is stale, you can’t query it or run automations based on it (such as send a notification if a service has a low production readiness score). You can’t simply ask “show me services in production that have vulnerabilities” or “which APIs can I use?” or “what is the cost of running this feature in an ephemeral environment?”

Are CMDBs any better than spreadsheets with microservice data?

A CMDB was made to document the IT landscape's hardware and software, and their interrelations. It’s been around for a long time. Yet it doesn’t work well as a service catalog.

Here are some of its shortcomings:

1. The CMDB is limited to a fixed amount of entity types

The CMDB has a rigid and opinionated data model. As a result, you cannot add the different types of assets that you would like to track. 
While you may want to track applications, components, interfaces, data objects, cloud resources, APIs, you are limited to what the tool contains, mis-representing your microservices ecosystem as a result.
Without the ability to put everything software in the CMDB, you can’t easily add plugin data (e,g, cost or appsec data). It is impossible to cross-reference data with other data sources to add additional context, such as identifying if a service is in production or asking to visualize all dependencies.

2. The static nature of the CMDB creates excessive manual toil

Data in the CMDB is static, while services, APIs, appsec data etc are dynamic. Not only does the data become stale quickly, it also required a lot of manual updates and never could reflect runtime. 

3. The result: poor data quality

Over time, manual updates and opinionated entity categories compromise the quality of the data, making it even worse than dated. It creates a situation where the quality of the data itself becomes questionable. As a result, whatever you wanted to track with CMDB isn’t tracked well, increasing cognitive load on developers, SRE engineers and others.

What service catalogs do better than CMDB

Some things are obvious:

1. Portals can automate data collection into the service catalog. In Port, just use an exporter, define some blueprints and relations, and voila, the catalog is there to be consumed.

2. Portals contain runtime data and are therefore both a single source of truth as well as a resource you can run workflows against.

3. Portals are flexible and can grow. Let’s say that your portal MVP did not require attending to cloud cost management issues or just in time permissions, or that it didn’t include an API catalog. Now, as part of continually improving the developer experience, you decided to add all three to the service catalog. Using a portal platform like Port let’s you do just that.

4. Portals are designed to contain all the data about software, and support plugins that bring 3rd party data into the portal. This means that you can immediately see cost, appsec, feature flag data and more, in context.

A service catalog belongs in an internal developer portal

Most importantly, An internal developer portal can do much more than just a service catalog. It offers developer self-service, scorecards and initiatives and personalized information for the different stakeholders using the internal developer portal. 

Using a portal not only provides developers with service catalog information, It allows them to self-serve using golden paths that ensure compliance with standards.

The Bottom Line

While spreadsheets and CMDBs may offer a semblance of control over service inventories, they are inadequate for the demands of contemporary software development. An internal developer portal, anchored by a robust software catalog, offers a proactive solution to service sprawl. It equips developers with the tools and information necessary for making strategic decisions, streamlining workflows, and delivering high-quality software.