Internal Developer Portal

Solving service sprawl: It’s time to replace the CMDB with a service catalog (and internal developer portal)

April 17, 2024

Roni Floman
Roni Floman
Solving service sprawl: It’s time to replace the CMDB with a service catalog (and internal developer portal)
Example H2

Ready to start?

Solving service sprawl

There is a price to be paid for the elegance of cloud native microservice architectures: sprawl. We’ve seen this problem emerge with APIs, causing the emergence of API catalogs and API management, and now the same problem repeats itself with microservices. 

This inability to see how services, APIs, CI/CD and resources are all connected isn’t just a problem for developers. DevOps are also burdened by this complexity, and engineering in general is concerned this will reduce productivity or result in unnecessary duplication of services/APIs and non-adherence to standards.

Some people have a solution to this: a spreadsheet with microservice data. In fact, in our 2024 State of Internal Developer Portals showed that while 85% of respondents reported using an internal developer portal or planning to add one, their explanation of what consists of an internal developer portal varied.  53% reported using a technology that can pass as a portal, while 35% reported using spreadsheets with microservice data.

While it’s pretty obvious that you can’t sustainably use spreadsheets as a service catalog, we’ve also seen instances where engineering organizations tried to use Configuration Management Databases (CMDB) in order to track their microservices. 

The TL:DR that just like spreadsheets can’t be a good service catalog, CMDBs can’t be a service catalog either.

If you want to learn why, read on.

Why Spreadsheets don’t work for microservices

Turning to spreadsheets as a makeshift service catalog might seem like a practical, low-cost solution. Why bother implementing a new product category - service catalogs - which is the number one best devops tools in 2024?

To understand this, here are two examples we’ve seen in real life:

  1. Using spreadsheets to manually track production readiness (and therefore ensure less incidents and compliance with engineering standards), something you’d do using the scorecards function in an internal developer portal:
  1. Using a service catalog to track cloud costs:

As you’ve already guessed, using spreadsheets to track service catalog data has significant drawbacks:

  • It’s time consuming
  • Error prone
  • Static - information such as vulnerabilities or health metrics is dynamic by nature. It changes all the time and therefore becomes stale and unactionable almost immediately
  • As with anything in a spreadsheet, you’re bound to have numerous version control issues as different developers update different metrics for different services, all in one spreadsheet

The result? no single source of truth: Since the spreadsheet is stale, you can’t query it or run automations based on it (such as send a notification if a service has a low production readiness score). You can’t simply ask “show me services in production that have vulnerabilities” or “which APIs can I use?” or “what is the cost of running this feature in an ephemeral environment?”

Are CMDBs any better than spreadsheets with microservice data?

A CMDB was made to document the IT landscape's hardware and software, and their interrelations. It’s been around for a long time. Yet it doesn’t work well as a service catalog.

Here are some of its shortcomings:

  1. The CMDB is limited to a fixed amount of entity types

The CMDB has a rigid and opinionated data model. As a result, you cannot add the different types of assets that you would like to track. 
While you may want to track applications, components, interfaces, data objects, cloud resources, APIs, you are limited to what the tool contains, mis-representing your microservices ecosystem as a result.
Without the ability to put everything software in the CMDB, you can’t easily add plugin data (e,g, cost or appsec data). It is impossible to cross-reference data with other data sources to add additional context, such as identifying if a service is in production or asking to visualize all dependencies.

  1. The static nature of the CMDB creates excessive manual toil

Data in the CMDB is static, while services, APIs, appsec data etc are dynamic. Not only does the data become stale quickly, it also required a lot of manual updates and never could reflect runtime. 

  1. The result: poor data quality

Over time, manual updates and opinionated entity categories compromise the quality of the data, making it even worse than dated. It creates a situation where the quality of the data itself becomes questionable. As a result, whatever you wanted to track with CMDB isn’t tracked well, increasing cognitive load on developers, SRE engineers and others.

What service catalogs do better than CMDB

Some things are obvious:

  1. Portals can automate data collection into the service catalog. In Port, just use an exporter, define some blueprints and relations, and voila, the catalog is there to be consumed.
  1. Portals contain runtime data and are therefore both a single source of truth as well as a resource you can run workflows against.
  1. Portals are flexible and can grow. Let’s say that your portal MVP did not require attending to cloud cost management issues or just in time permissions, or that it didn’t include an API catalog. Now, as part of continually improving the developer experience, you decided to add all three to the service catalog. Using a portal platform like Port let’s you do just that.
  1. Portals are designed to contain all the data about software, and support plugins that bring 3rd party data into the portal. This means that you can immediately see cost, appsec, feature flag data and more, in context.

A service catalog belongs in an internal developer portal

Most importantly, An internal developer portal can do much more than just a service catalog. It offers developer self-service, scorecards and initiatives and personalized information for the different stakeholders using the internal developer portal. 

Using a portal not only provides developers with service catalog information, It allows them to self-serve using golden paths that ensure compliance with standards.

The Bottom Line

While spreadsheets and CMDBs may offer a semblance of control over service inventories, they are inadequate for the demands of contemporary software development. An internal developer portal, anchored by a robust software catalog, offers a proactive solution to service sprawl. It equips developers with the tools and information necessary for making strategic decisions, streamlining workflows, and delivering high-quality software.

Previous article

Next article

{{cta_1}}

Check out Port's pre-populated demo and see what it's all about.

Check live demo

No email required

{{cta_2}}

Contact sales for a technical product walkthrough

Let’s start
{{cta_3}}

Open a free Port account. No credit card required

Let’s start
{{cta_4}}

Watch Port live coding videos - setting up an internal developer portal & platform

Let’s start
{{cta_5}}

Check out Port's pre-populated demo and see what it's all about.

(no email required)

Let’s start
{{cta_6}}

Contact sales for a technical product walkthrough

Let’s start
{{cta_7}}

Open a free Port account. No credit card required

Let’s start
{{cta_8}}

Watch Port live coding videos - setting up an internal developer portal & platform

Let’s start
{{cta-demo-baner}}

Book a demo right now to check out Port's developer portal yourself

{{port-for-backstage}}

Apply to join the Beta for Port's new Backstage plugin

Apply for beta
{{jenkins}}

It's a Trap - Jenkins as Self service UI

Read more
{{gitops}}

How do GitOps affect developer experience?

Read more
{{ebook}}

It's a Trap - Jenkins as Self service UI. Click her to download the eBook

Download eBook
{{cyberark}}

Learning from CyberArk - building an internal developer platform in-house

Read more
{{cta-demo}}
{{reading-box-backstage-vs-port}}

Further reading:

Read: What are the technical disadvantages of Backstage?

Compare: Backstage vs. Port

Example JSON block

{
  "foo": "bar"
}

Order Domain

{
  "properties": {},
  "relations": {},
  "title": "Orders",
  "identifier": "Orders"
}

Cart System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Cart",
  "title": "Cart"
}

Products System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Products",
  "title": "Products"
}

Cart Resource

{
  "properties": {
    "type": "postgress"
  },
  "relations": {},
  "icon": "GPU",
  "title": "Cart SQL database",
  "identifier": "cart-sql-sb"
}

Cart API

{
 "identifier": "CartAPI",
 "title": "Cart API",
 "blueprint": "API",
 "properties": {
   "type": "Open API"
 },
 "relations": {
   "provider": "CartService"
 },
 "icon": "Link"
}

Core Kafka Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Kafka Library",
  "identifier": "CoreKafkaLibrary"
}

Core Payment Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Payment Library",
  "identifier": "CorePaymentLibrary"
}

Cart Service JSON

{
 "identifier": "CartService",
 "title": "Cart Service",
 "blueprint": "Component",
 "properties": {
   "type": "service"
 },
 "relations": {
   "system": "Cart",
   "resources": [
     "cart-sql-sb"
   ],
   "consumesApi": [],
   "components": [
     "CorePaymentLibrary",
     "CoreKafkaLibrary"
   ]
 },
 "icon": "Cloud"
}

Products Service JSON

{
  "identifier": "ProductsService",
  "title": "Products Service",
  "blueprint": "Component",
  "properties": {
    "type": "service"
  },
  "relations": {
    "system": "Products",
    "consumesApi": [
      "CartAPI"
    ],
    "components": []
  }
}

Component Blueprint

{
 "identifier": "Component",
 "title": "Component",
 "icon": "Cloud",
 "schema": {
   "properties": {
     "type": {
       "enum": [
         "service",
         "library"
       ],
       "icon": "Docs",
       "type": "string",
       "enumColors": {
         "service": "blue",
         "library": "green"
       }
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "system": {
     "target": "System",
     "required": false,
     "many": false
   },
   "resources": {
     "target": "Resource",
     "required": false,
     "many": true
   },
   "consumesApi": {
     "target": "API",
     "required": false,
     "many": true
   },
   "components": {
     "target": "Component",
     "required": false,
     "many": true
   },
   "providesApi": {
     "target": "API",
     "required": false,
     "many": false
   }
 }
}

Resource Blueprint

{
 “identifier”: “Resource”,
 “title”: “Resource”,
 “icon”: “DevopsTool”,
 “schema”: {
   “properties”: {
     “type”: {
       “enum”: [
         “postgress”,
         “kafka-topic”,
         “rabbit-queue”,
         “s3-bucket”
       ],
       “icon”: “Docs”,
       “type”: “string”
     }
   },
   “required”: []
 },
 “mirrorProperties”: {},
 “formulaProperties”: {},
 “calculationProperties”: {},
 “relations”: {}
}

API Blueprint

{
 "identifier": "API",
 "title": "API",
 "icon": "Link",
 "schema": {
   "properties": {
     "type": {
       "type": "string",
       "enum": [
         "Open API",
         "grpc"
       ]
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "provider": {
     "target": "Component",
     "required": true,
     "many": false
   }
 }
}

Domain Blueprint

{
 "identifier": "Domain",
 "title": "Domain",
 "icon": "Server",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {}
}

System Blueprint

{
 "identifier": "System",
 "title": "System",
 "icon": "DevopsTool",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "domain": {
     "target": "Domain",
     "required": true,
     "many": false
   }
 }
}
{{tabel-1}}

Microservices SDLC

  • Scaffold a new microservice

  • Deploy (canary or blue-green)

  • Feature flagging

  • Revert

  • Lock deployments

  • Add Secret

  • Force merge pull request (skip tests on crises)

  • Add environment variable to service

  • Add IaC to the service

  • Upgrade package version

Development environments

  • Spin up a developer environment for 5 days

  • ETL mock data to environment

  • Invite developer to the environment

  • Extend TTL by 3 days

Cloud resources

  • Provision a cloud resource

  • Modify a cloud resource

  • Get permissions to access cloud resource

SRE actions

  • Update pod count

  • Update auto-scaling group

  • Execute incident response runbook automation

Data Engineering

  • Add / Remove / Update Column to table

  • Run Airflow DAG

  • Duplicate table

Backoffice

  • Change customer configuration

  • Update customer software version

  • Upgrade - Downgrade plan tier

  • Create - Delete customer

Machine learning actions

  • Train model

  • Pre-process dataset

  • Deploy

  • A/B testing traffic route

  • Revert

  • Spin up remote Jupyter notebook

{{tabel-2}}

Engineering tools

  • Observability

  • Tasks management

  • CI/CD

  • On-Call management

  • Troubleshooting tools

  • DevSecOps

  • Runbooks

Infrastructure

  • Cloud Resources

  • K8S

  • Containers & Serverless

  • IaC

  • Databases

  • Environments

  • Regions

Software and more

  • Microservices

  • Docker Images

  • Docs

  • APIs

  • 3rd parties

  • Runbooks

  • Cron jobs

You may also be interested in

Building the perfect internal developer platform

Building the perfect internal developer platform

Read more
How to use JQ commands to speed developer workflows

How to use JQ commands to speed developer workflows

Read more
Debugging K8s with K8sGPT in your internal developer portal

Debugging K8s with K8sGPT in your internal developer portal

Read more

Starting with Port is simple, fast and free.

Let’s start