Platform engineering is gaining popularity and in June 2024 even got its very own platform engineering hype cycle from Gartner.
The idea at the core of platform engineering is that instead of ticketops or devops, an internal developer platform is built in order to provide developers with self-service, while ensuring that the software that’s developed complies with production readiness, appsec and cost requirements (as well as any other applicable standards), allowing compliance with existing devops standards.
Building an internal developer platform that lets developers work autonomously and improves standards compliance requires careful consideration of developer needs and the technology stack used to deliver the platform. This article provides an overview of the essential components and tools needed to establish a fully functional internal developer platform (IDP).
What is platform engineering and why do you need it?
Platform engineering is the discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering teams in the cloud-native era. It focuses on creating an internal developer platform (IDP) that abstracts away the complexity of infrastructure and operations, allowing developers to work autonomously and efficiently. By leveraging technologies like Kubernetes, CI/CD pipelines, and infrastructure as code (IaC), platform engineering aims to enhance developer productivity, reduce cognitive load, and streamline the deployment and management of applications across diverse environments.
The primary objective of platform engineering is to foster developer autonomy. Whether a developer is proficient in Kubernetes, a junior or an architect, they should effortlessly access the resources they need. For instance, if a developer requires a database or wants to manage an application in Kubernetes, they shouldn't be burdened with extensive knowledge acquisition in order to perform a simple task. Instead, developers should be able to perform these tasks through a simplified and abstracted interface.
What is an IDP?
IDP is actually a confusing term. It is used interchangeably to describe “internal developer portal” and “internal developer platform”. We use IDP for “platform” and its meaning is all the tools used to provide developers with visibility, compliance with standards and reusable self-service actions. The “other IDP”, the internal developer portal, is used as the interface and business process orchestrator for the internal developer platform, allowing developers an abstracted self-service interface to make work simpler and easier and letting managers track standards compliance and productivity quickly and effectively.
The importance of platform engineering tools
Platform engineering tools are essential for modern software development as they enable developers to work more efficiently and autonomously. These tools automate repetitive tasks, manage complex infrastructure, and provide seamless integration across various stages of the development lifecycle.
By leveraging platform engineering tools, organizations can enhance developer productivity, ensure consistent and reliable deployments, and scale applications effortlessly. Moreover, these tools foster improved collaboration and reduce friction among teams by providing a unified platform for resource management and automation.
Key features of internal developer platform tools
While the platform is made of many different tools, some core features of platform engineering tools, or their use alongside each other to deliver the internal developer platform, are:
- A focus on enabling developers: the core of the platform is the idea of self-service, and alongside it is the idea of abstracting information for developers and making a better developer experience for them. This means both golden paths and day-2 operations and anything in between.
- A focus on developer productivity, on measuring it and on impacting it
- A focus on agility and re-use of processes, to reduce time wasted on scattered processes which aren’t repeatable but should be made so
- A focus on security by design: one of the core complexities for developers and DevSecOps alike is being able to deliver secure software, and platforms can greatly drive better appsec using a portal.
- A focus on compliance with software engineering standards: through the use of scorecards and observability, platforms can grow compliance with standards such as production readiness.
Top 10 Platform Engineering Tools
1. Internal developer portals - Port
Internal developer portals are used as the user interface on top of the internal developer platform. They are one of the most central tools in any platform engineering initiative, since they are what provide abstracted self-service to developers, and measurement of engineering effectiveness and standards compliance for managers.
Internal developer portals determine the appropriate level of abstraction for platform users, abstracting unnecessary details that users typically don't care about, such as database storage details.
An internal developer portal is a centralized platform that consolidates tools, resources, and documentation to streamline the development workflow for engineering teams. It provides a user-friendly interface for managing infrastructure, deploying applications, and accessing essential services, promoting self-service capabilities and reducing dependency on operations teams. By offering features like a software catalog, CI/CD pipelines, and integration with version control systems, an IDP enhances developer productivity and ensures consistent and efficient application management.
We chose Port as our top tool for internal developer portals.
Port is a platform for open internal developer portals. Port consolidates everything developers need to know and execute to deliver software more productively and to comply with organizational standards. Platform engineering teams use Port to set up a unified interface for developers, who can use self-service to work independently, from setting up a microservice or environment, performing day-2 operations and more, with the right guardrails in place.
2. Cloud native control planes - crossplane
Crossplane is an open-source project by CNCF, built on Kubernetes to orchestrate anything. It allows you to encapsulate policies, permissions, and other guardrails behind a custom API layer. This enables your developers to self-service without needing to be infrastructure experts. With Crossplane, you can streamline operations and enhance user autonomy, ensuring efficiency and security without compromising control. It’s a powerful tool for modern infrastructure management.
Using Crossplane the entire service lifecycle is covered, from operations, creation, and retirement. The result is velocity that is scalable and reliable, so that any consumers of the platform can easily build what infra they need. As always, this covers day-2 operations and supports policies and their enforcement.
Upbound, the builder of crossplane, built it so that everyone can use a crossplane, just like the cloud vendors do. Using Upbound, organizations can effectively use GitOps for their cloud-native infrastructure and services, deploying in minutes and easily customizable so that performance, cost, and compliance requirements are met.
3. Secret management - Hashicorp Vault
Since platforms operate across distributed environments and cloud resources, there is a need for effective and secure secrets management, that would reduce human error, adhere to safety standards and cover both human interaction as well as workload to workload interaction. For this reason, almost all platform tools include a secrets management solution.
We recommend Hashicorp Vault to manage secrets and protect sensitive data and securely store and tightly control access to tokens, passwords, certificates, and encryption keys.
4. SRE tools - Amazon cloudwatch
Anything developed through the platform needs to be resilient and reliable, meeting service level objectives. As a result, almost no platform can be built without considering SRE practices and SRE tools. Otherwise, developers will still be waiting for SRE review and incident management won’t change much as a result of the introduction of platform engineering.
Amazon CloudWatch is a powerful monitoring service for applications, designed to respond to performance changes, optimize resource usage, and provide comprehensive insights into operational health. By gathering data from across AWS resources, CloudWatch offers visibility into system-wide performance. Users can set alarms, automatically respond to changes, and gain a unified view of their operational health. This enables proactive management and optimization of resources, ensuring smooth and efficient application performance.
6. Developer experience tools - DX
Platforms are there to provide self-service that will let developers act autonomously, improving the developer experience and, as a result, developer productivity. But how can you tell what the developer experience is and what changes can be made?
This is where platforms that continuously measure the developer experience, using proven research methodologies, come in.
Using DX you can measure and improve developer productivity with the insights platform designed by leading software engineering researchers.
7. FinOps tools - cast.ai
Setting developers loose around cloud resources can lead to high cloud bills. This, as well as the constant need to optimize cloud spend, is leading many organizations to adopt FinOps practices. Since an internal developer platform may result in higher cloud spend, FinOps tools can provide a solution and integrate into the portal.
Our choice is Cast AI. Companies use it to analyze their K8s clusters and understand potential savings, as well as optimize their cloud setup automatically.
8. Platform orchestrator tools - Kratix
Also known as self-service environment management tools, platform orchestrator tools are about letting developers provision and manage environments. Instead of letting developers loose, these tools create governance by using infrastructure as code and policies. This also ensures that cost, compliance and security are baked into the resulting environments. As a result of the creation of such ready to use environments, software testing is easier and less orphaned or non-standard environments.
Kratix is an open-source platform framework for building composable internal developer platforms (IDPs).
9. Software engineering intelligence tools - LinearB
Software Engineering Intelligence (SEI) refers to the application of data-driven approaches and intelligent systems to understand and optimize software development processes. In most cases, such tools are supposed to help engineering leadership understand how engineering is using resources, what it is spending time on, how effective it is as well as various metrics, such as DORA metrics.
LinearB is a company that provides software engineering intelligence, providing engineering leaders with the ability to measure productivity, engineering efficiency and more.
10. Cloud development environments - GitPod
A Cloud Development Environment (CDE) in an implementation of containers or virtual machines to support code development activities. GitPod is a company that provides CDEs that are on-demand, pre-configured with all tools, libraries and dependencies required to be ready-to-code.
Book a demo right now to check out Port's developer portal yourself
It's a Trap - Jenkins as Self service UI
How do GitOps affect developer experience?
It's a Trap - Jenkins as Self service UI. Click her to download the eBook
Learning from CyberArk - building an internal developer platform in-house
Example JSON block
Order Domain
Cart System
Products System
Cart Resource
Cart API
Core Kafka Library
Core Payment Library
Cart Service JSON
Products Service JSON
Component Blueprint
Resource Blueprint
API Blueprint
Domain Blueprint
System Blueprint
Microservices SDLC
Scaffold a new microservice
Deploy (canary or blue-green)
Feature flagging
Revert
Lock deployments
Add Secret
Force merge pull request (skip tests on crises)
Add environment variable to service
Add IaC to the service
Upgrade package version
Development environments
Spin up a developer environment for 5 days
ETL mock data to environment
Invite developer to the environment
Extend TTL by 3 days
Cloud resources
Provision a cloud resource
Modify a cloud resource
Get permissions to access cloud resource
SRE actions
Update pod count
Update auto-scaling group
Execute incident response runbook automation
Data Engineering
Add / Remove / Update Column to table
Run Airflow DAG
Duplicate table
Backoffice
Change customer configuration
Update customer software version
Upgrade - Downgrade plan tier
Create - Delete customer
Machine learning actions
Train model
Pre-process dataset
Deploy
A/B testing traffic route
Revert
Spin up remote Jupyter notebook
Engineering tools
Observability
Tasks management
CI/CD
On-Call management
Troubleshooting tools
DevSecOps
Runbooks
Infrastructure
Cloud Resources
K8S
Containers & Serverless
IaC
Databases
Environments
Regions
Software and more
Microservices
Docker Images
Docs
APIs
3rd parties
Runbooks
Cron jobs
Check out Port's pre-populated demo and see what it's all about.
No email required
Contact sales for a technical product walkthrough
Open a free Port account. No credit card required
Watch Port live coding videos - setting up an internal developer portal & platform
Check out Port's pre-populated demo and see what it's all about.
(no email required)