Using Backstage’s C4 Model adaptation to visualize software - creating a software catalog in Port

Standardizing architecture diagrams for developer portals

It all started when spotify engineers decided to visualize software in order to standardize architecture diagrams there, to make the information clear and consumable to everyone within the engineering organization. This model is based on the C4 model with some adaptations. The model’s metadata is presented in a software catalog and the best part is that the question of whether what’s in the diagram and software model is up to date no longer matters, since it automatically updates. 

At Port, we like this model since it goes way beyond a microservices catalog and embarks on cataloging all elements related to the software - the microservices, the resources they are deployed on, the specific deployments and the relationships and dependencies between all of them. This is best said by Renato Kalman and Johan Wallin here:

“By expressing this model as metadata, we have been able to create a software catalog that keeps track of components, ownership, dependencies, and lifecycles in our ecosystem.”

I agree that this is what every engineering organization should do (once it reaches a certain size). But can this be accomplished outside backstage? As you’re probably guessing the answer is yes. This matters because not everyone can or wants to implement spotify’s backstage. Backstage requires some extensive coding and a team around it. Why not go for a SaaS based internal developer portal - which will let you accomplish the same with a builder based approach?

But first, let’s see the logic of the model, before we actually show how to apply it.

{{cta_4}}

What works well in the backstage software catalog

The only way to make sense and derive value from a software catalog is to take a similar approach. Engineering needs a unified and consistent view of everything, and it needs to include dependencies, relations, and a lot of additional metadata.

In the next sections we will explain where the backstage data model works well. Then in the latter part of the blog we’ll explain where it doesn’t work well and how the data model can be improved to support additional use cases and provide greater value to platform teams, developers and engineering managers

Backstage’s model begins with three core entities: API, component and resource, as in the C4 model. To make it easier to consume in large and complex organizations, two additional entities were added: Systems and Domains.

Systems are collections of entities that work together in performing a certain function and domains are systems and entities that “belong” to different parts of the business. 

To best understand the C4 model and its adaptation, let’s imagine you have to model a software design for your organization's new food delivery application. You will have to consider the following:

Now let’s begin the walkthrough: how can this be modeled in a builder-based software catalog?

‍

Software Catalog - Modeling the structure

Port uses entities that support this adaptation of the C4 model. Entities are defined using a Port Blueprint, which is the primary building block of Port. Blueprints represent assets that can be managed in Port, such as microservices, environments, packages, clusters, databases, and many more.

Blueprints are completely customizable, and they support any number of properties the user chooses, all of which can be modified as you go.

Using simple JSON files, I have created five blueprints corresponding with the backstage entities with the exact relations that should be reflected.

The software catalog model elements

Components

Components are Service or Library, which are differentiated by the type, a simple enum of “service” or “library.”

Components are also connected to themselves by many relations, because a service might be related to multiple libraries, like in our case. Or, it can also be related to another service.

{{component-blueprint}}

Resources

Resources are any Infrastructure needed to run a component (S3 buckets, SQL databases, etc..)

{{resource-blueprint}}

API

API is a simple software catalog item that can be consumed or provided by a component.

API also has a type, which tells the exact API type, REST, GRPC, protobuf, etc.

{{api-blueprint}}

Domain

A domain is a collection of systems representing a distinct area of influence, activity, and decision-making within an organization (business).

{{domain-blueprint}}

System

A system is a sub-domain that is focused on a specific branch (business) within the organization.

{{system-blueprint}}

Ingesting the data into the catalog

Once the blueprints are ready, we need to ingest the data into the software catalog, creating entities. Backstage requires putting a YAML inside a git repository or writing a custom entity processor. Port makes it easier, allowing you to ingest entities from your pipelines, K8S, Git, Terraform, API, and more.

For the sake of simplicity, we will use the JSON files below to reflect the entities within the catalog and the relations between them. You can copy and paste them into the UI, or via any other method mentioned above.

{{order-domain}}

{{cart-system}}

{{products-system}}

{{cart-resource}}

{{cart-api}}

{{ckl}}

{{cpl}}

{{cart-service-json}}

{{product-service-json}}

Once the data is ingested, you will gain complete visibility of the service catalog. You can see all the related entities for each entity. For example, let's look at the Cart Service entity page. We can see the Component it interacts with and the API it provides alongside the business-related domain of the service in a single united view.

Congratulations! You have created your own Developer Portal using the C4 adaptations Backstage use Port.

‍

{{cta_3}}

The disadvantages of the backstage C4 model

Backstage also has some significant technical disadvantages that are often missed by teams, since they assume these issues can be either programmed away or  dealt with using Backstage plugins.

The first disadvantage is that Backstage’s C4 model is rigid and difficult to change - it is a fixed data model. This involves two distinct problems: the inability to modify (or add) entity types and the inability to represent various relationships between those entity types. Let's explore this further. Furthermore, this forms the core of backstage and is therefore almost impossible to fix using programming. 

The Backstage framework does not support the creation of custom entity types, likely because it was initially developed for Spotify, which had specific requirements and processes (this is an assumption). Although Backstage recommends contacting maintainers for guidance on modeling new types, this approach might not be agile enough for rapid development or complex, specific needs.

What entity kinds would you want to add to a software catalog? Here are some examples:

• Cloud permissions so you could provide just-in-time access and work more securely
• Alerts so you could unify alerts in the developer portal and make it easier for developers to understand and resolve issues
• Incidents so you could support on-call and make it a better experience for developers and reduce MTTR
• Vulnerabilities so you could bake security into every developer routine
• CI/CD to you the portal as a CI/CD catalog and 
• API data to use the portal for API governance and more. 

Backstage presumes fixed relationships between entities. For instance, it includes a "dependsOn" relationship to link components to resources, indicating that a component relies on specific resources.

However, in practice, you often need to differentiate between various types of dependencies, such as distinguishing between runtime cloud resources (e.g., compute instances) and storage resources (e.g., databases and S3 buckets). In such scenarios, Backstage's model falls short. It doesn't allow for specifying multiple, distinct relationships between entities, leading to a lack of granularity and potential confusion in understanding resource dependencies.

What are the implications of fixed entity types and relationships? The software catalog can't model everything you need, failing to accurately represent the SDLC environment and not providing developers with the necessary context when they need it.

Conclusion

When it comes to an internal developer portal, every organization is different and has a unique set of individual needs and requirements. Choose a portal that can evolve over time and be easy to use - otherwise, despite the logic of the backstage C4 model, you can find it too rigid and not able to support the internal developer portal and platform as they evolve.