What is Spotify Backstage?
Backstage is an open source platform for building internal developer portals. Internal developer portals aim to provide the capabilities of an internal developer platform in an abstracted way, reducing cognitive load on developers, and increasing engineering teams’ efficiency and productivity. Portals are a key category within the growing field of platform engineering; Gartner predicts that by 2026, 80% of large software engineering organizations will establish platform engineering teams, up from 45% in 2022.
Backstage was developed by music streaming company Spotify as an internal project codenamed ‘System Z’, helping its engineers to gain easier access to the company’s software, APIs, services, tools and docs, as well as being able to get a better understanding of each of these areas so that they could seamlessly know about ownership, dependencies, versions and more. The platform enables developers and other stakeholders to create, manage and explore software from a solitary UX layer.
It was the first internal developer portal platform, helping to stimulate an interest in platform engineering. In March 2020, Backstage was open-sourced and donated to the Cloud Native Computing Foundation (CNCF). While gaining momentum as an internal developer portal, Backstage has had to contend with challenges around development time and maintainability - in particular for large enterprises that have complex environments.
This spurred on alternatives like Port, which provide quicker setup, easier maintenance, improved flexibility, more robust self-service actions, and a reduction in time and effort required. To combat this, in 2024, Spotify announced ‘Spotify portal for Backstage’ - a commercial offering, which it claims will enable dev teams to more easily set up an internal developer portal using Backstage. However, the portal is limited in its scope, primarily focusing on just one key pillar of a portal (the software catalog), meaning it may continue to require huge resources to maintain and meet unique business needs.
Key Features of Spotify Backstage
Backstage Software or Service Catalog
The software catalog was the initial focus for the Spotify Backstage team when they developed the internal tool - and it remains the most important pillar of an internal developer portal. With new devops practices, the introduction of Kubernetes, GitOps and cloud native, development teams have encountered a lot of complexity; having to learn to use new tools with different interfaces, switch between tools, and become accustomed to terminology that isn’t universal. Developers have also had to deal with increased responsibilities in coding, shipping and running software.
Spotify’s development team realized that they need a new approach - a service catalog, which aims to provide discoverability and visibility, to provide developers with a better working environment. When properly maintained, the catalog restores a level of transparency that can diminish as enterprise teams grow. The catalog should enable developers to access all relevant details about the organization’s services, including the owner, lifecycle, runtime, version, last deployment link, and more.
The software catalog acts as a centralized source for metadata and ownership information for all team software. It allows you to monitor your services, applications, pipelines, and other assets in a single, cohesive view. The catalog is structured around metadata YAML files, which are stored alongside the application code. These files are collected and displayed in Backstage. Backstage uses a C4 data model and ingests data from integrations, in a bid to serve as a central metadata store about everything in the SDLC.
Backstage Software Templates
The templates plugin forms the self-service aspect of the developer portal, providing DevOps with the ability to define a code skeleton with variables, which can then be pushed to GitHub, GitLab or another Git provider. Developers can then select the relevant template and use a creation wizard to fill in the necessary parameters. This process results in a new repository with all the defined standards and an added component in the Backstage Software Catalog.
The templates simplify the process for teams to quickly and easily perform actions such as the creation of new services. This enables teams to scale best practices across the organization (such as CI documentation, logging, and Kubernetes configuration), as developers can use templates which have standards baked-in, and can get started quickly, without the need to rebuild from scratch each time.
The Software Templates stop short of offering a golden path for developers as they are primarily focused on the creation phase and do not support day-2 operations. As most attention is required on already-provisioned services, it is important for developers to be able to use self-service actions for day-2 operations to correct any issues with resources. It is possible to support these operations in Backstage through substantial custom TypeScript development.
Backstage TechDocs
TechDocs is a plugin that aims to make it easy to manage technical documentation and keep it up-to-date, by managing it all from their code repositories.
Engineers can create technical documentation as markdown files that reside alongside the code, removing friction to creating docs and also making them easier to find. This means the docs are more likely to get used - and this encourages best practices, reducing issues in the SDLC.
Every modification is tracked and managed through the Git process, including the last update, contributors, code reviews, automated tests, ownership, and GitHub issues. The documentation is then rendered in HTML format, making it searchable and editable via Backstage’s Markdown files. This approach fosters a sense of ownership among both writers and developers over the documentation, promoting a culture of collaboration and teamwork.
Backstage Search
The Search plugin is quite literally what it sounds like - a search engine for the Backstage ecosystem, enabling users to find what they need. It is customizable and can be connected with an organization’s own search engine such as Elasticsearch/OpenSearch, Lunr and Postgres. It enables users to search their software catalog, but with additional support, can also enable search for other plugins, wiki and Slack Overflow.
Backstage Plugins
To customize the platform and add functionality, Backstage has a number of plugins, which are React components that are added to each service’s page. The plugins are created externally through the open-source community, as well as through engineering teams.
A popular plugin example is Kubernetes, which enables users to check the status of their K8s services, with information such as autoscaler limits or pods experiencing errors. The idea is that users don’t need to move between multiple K8s dashboards to see the overall service status.
There are also paid-for plugins that Backstage has created which can be purchased.
Top Challenges of Working with Backstage
Resource drain and complexity
One of the key challenges of working with Backstage is the sheer level of resources - both in terms of time and personnel - it takes to get to a level that an organization is content with. For example, populating the Backstage software catalog is carried out by manually creating static YAML files, which requires significant effort from the entire organization.
This requires time and effort, but also a huge learning curve. It requires a huge team to oversee, meaning an enterprise will have to dedicate a lot of resources to getting Backstage right, and that’s before contending with the challenges below.
Fixing its issues with plugins
A common misconception is that issues around Backstage’s software catalog, namely that it has a fixed data model, can be fixed using Backstage plugins. The problem with both fixed entity kinds and fixed relationships between entity kinds, is that the software catalog can’t model everything you want to model, and therefore it can’t represent your entire SDLC as it really is. This diminishes its power of providing the context that developers need. You can of course change the default model with significant coding, but this adds to the resource drain and complexity. Regardless, adding custom entity kinds are not supported, and therefore enterprises miss out on a number of use cases such as incident management, alert management and API management.
No real-time data
The Backstage software catalog doesn’t include real time data, meaning the portal can’t be used for use cases that require runtime data (K8s workload health metrics). For example, keeping track of vulnerabilities.
Maintainability
When code changes happen, YAMLSs require maintenance, resulting in outdated information that can negatively impact operations and decision-making.
Backstage vs Port
There are a number of key differences between Port and Backstage.
Quicker setup
Backstage’s basic set-up is labor-intensive and requires coding. An MVP can take months and many resources. With Port, you can quickly set up a portal MVP, enabling you to progress quickly and not lose momentum.
Data model - flexible vs fixed
Backstage has an opinionated model that can’t be changed and is hardwired into the code - sometimes even coding can’t solve issues. Meanwhile, with Port’s un-opinionated data model, it’s easy to add new use cases (such as adding cost data into the portal, CI/CD, alerting), and it’s easy to change the portal as the organization grows and evolves.
Time and effort involved
Setting up an initial service catalog is difficult in Backstage. Beyond the set-up, there’s extensive coding required for plugins and self-service actions. It can take months to launch and the maintainability issues hurt continuous improvement. It may require five to ten full-time engineers to focus purely on Backstage - and a high effort from the entire R&D department. In comparison, Port doesn’t require the same extensive coding, and requires minimal effort from the platform engineering team.
Populating the catalog
The main way to populate the catalog in Backstage is by manually creating static YAML files, which requires a lot of effort from the entire organization - the knock-on effect is that it becomes an adoption challenge. Port enables the auto-discovery of all relevant data stored in Git, and this can be enhanced manually by updating Port directly or via YAML files.
Third-party data: added and usable
As there’s no central metadata store in Backstage, it presents a challenge to query across plugins, meaning users can’t ask questions such as ‘what services have an open incident’. Customizing the plug requires React coding, there’s a lack of maintenance and support, and the resulting UX and quality of the catalog is inconsistent which adds complexity. In comparison, Port has a unified metadata store, so that all plugin data feeds a unified UI, and users can query the entire catalog to get the answer to any question. Integrations are easily extensible and customizable, supported and maintained.
Software templates and developer self-service
Backstage replaces an organization’s current runners with a proprietary workflow mechanism, whereas Port integrates with the organization’s backend by triggering any CI tool or HTTP endpoint so you will be able to use the tools of your choice. Backstage can’t enable users to perform day-2 actions such as asking for permissions or opening a new incident on a specific service; day-2 ops are vital for developers to be able to follow a golden path. Port supports any action, including day-2 actions. With Port you can also configure permission rules based on catalog data and provision resources with TTL, both of which aren’t possible with Backstage.
For a deeper dive into the technical disadvantages of Backstage, click here.
