What comes first: developer platform or developer portal?

This article came from an interview with Chris Westerhold from ThoughtWorks. He is the Head of Developer Experience at Thoughtworks and co-founder of Think Big Code Small.  Chris has over 15 years of technology experience across start-ups and large enterprises with a major focus on data, developer platforms, engineering effectiveness, and technical product management.  He is a vocal advocate for developer experience and passionate about using data-driven approaches to improve it.

Most companies do not have a fully fledged internal developer platform when they begin their platform engineering journey. How do they align that with the decision on when to begin implementing an internal developer portal? How do they choose what to begin with - portal or platform - and how do they create something with enough significance for developers?

Chris: It’s confusing; you’ve come to the conclusion that you need to change your existing software development environment, perhaps as part of a shift to platform engineering. You keep hearing the terms ‘Internal Developer Platform’ and ‘Internal Developer Portal’. You know what they are (if you don’t - read this first), but you don’t know which one you need for your organization first. It’s a chicken and egg question.

To get the answer, take a product management approach to identify where your organization’s  waste and friction truly lies. To get answers from your developers,  create a good feedback loop using surveys, interviews and other methods like value stream mapping (more on this later). 

If you have nothing to start with, then perhaps building a platform might be best for you, whereas if you’re trying to go through some kind of transformation, a portal may be best. Too often, engineering teams are spending time, energy and money doing something they think is right, but isn't what developers actually need to improve their processes, save time, or reduce toil.

In some cases, a portal-first approach is best

So let’s say you’re an e-commerce company that is looking to scale but is impeded by having a monolith architecture. There is a plan to undergo a microservice transformation - switching your architecture and becoming API-driven. This could require a portal first because you’re taking a singular element and breaking it into hundreds of elements, and so you’re going to be creating a lot of new things over a short period of time. A portal would enable you to maintain a clear understanding of your ecosystem while creating high-quality microservices. It would also help developers understand the new approach, understand dependencies, and become the single pane of glass for the new and complex systems.

The argument against this is that if you’re creating all of these new microservices, having a good platform underneath would surely help - and that’s completely true. But deciding whether the portal or platform is more important is dependent on the individual company, the timeline in which it's trying to do things and the amount of money it's willing to spend.

For some companies in this type of situation, building a platform before breaking up the monolith may be the way to go. Then, as soon as they’re ready to start the shift to microservices, they can push forward with a developer portal.

The reason the portal is more important at this stage - regardless of there being a platform in place or not - is because the developers working within the organization may only know and understand a monolith world. Old school, monolithic Java developers, for instance, may not have a deep understanding of modern CI/CD. Breaking down the monolith means you would be thrusting upon them new tasks such as writing microservices, owning pipelines security, as well as a new testing paradigm. This is a change management problem that many enterprises are confronted with and it takes the combination of good leadership and surveying the developer community, to try to anticipate the issues and solve them ahead of time. 

Regardless, unlike the shift from monolith to microservices, implementing and using a developer portal doesn’t cause a change management problem of its own, but becomes an enablement and feedback tool to help facilitate the process. In fact, it helps engineering teams to overcome  process problems they face as the portal abstracts away the complexity of the tools and tasks that developers will need.

You should also identify the teams who previously had those responsibilities - to ensure they can be part of the new structure. These  enablement teams, leveraging the portal, can then help developers kick-start what they need to do by providing the tools and techniques they need such as a good set of pipelines or self service scaffolding for a new application. This doesn’t require a full-blown platform underneath but rather the tools that developers need to be successful day-to-day.

In other cases, a platform first approach is better, but you will still need a portal eventually

If your CTO and CFO came over to you and said you needed to cut cloud costs in a way that continues to drive the business forward, then you may think it’s a difficult or even impossible task. But improving developer experience, ensuring good platform engineering, and utilizing DORA, SPACE, and other metrics can help clear up bottlenecks and drive better costs.

One of the biggest mistakes that many organizations fall foul of is not right sizing cloud deployments and hoping each development team has the skills necessary to understand the underlying infrastructure decisions and its impact to overall costs and resiliency. A great example could be an internal application at a company which has a few hundred users, that is deployed into a huge Kubernetes cluster or serverless functions. This application has no need for this kind of scale and this deployment costs significantly more every year without getting the benefits from this type of architecture. In the end, this could run off a single VM costing a fraction of the amount and getting the same outcome.

The balance of team responsibilities is key here. Separating responsibilities and focusing on what each team does best will drive better outcomes across the board, including costs.

Another example could be breaking apart your CI and CD processes. The development team owns the CI process, their job is to get an application into an image repository, and then the platform team takes over and deploys it where it makes the most sense. They would need to consider many factors including cost, scaling, and overall resiliency. These are also factors that change over time, so having good observability will help to ensure the proper alignment.

With the platform team in place, you can then use a developer portal to provide developers with the information they need to understand their application, where it’s deployed, and things like the status of tools across the DevSecOps tool chain.

This essentially enables platform teams to take away some of the burden that app developers have to contend with - leaving developers to build quality applications, and platform teams to deploy them well, and in both cases they can use the developer portal as an enabler.  By using this approach, both positive and negative feedback loops are automatically created and app teams have insights across the platform.

This enables you to work in tandem; having a great platform and having a great developer portal that gives developers and other users what they need - and by doing so significantly driving down costs.

Creating something of significance for developers: the MVP 

Improving the developer experience requires an understanding of where we can meet developers and serve them best. There are few different ways of thinking about this problem, but here are a few areas of focus.

Thin Slicing

Rather than thinking of this as a ‘chicken and egg’ dilemma, think of it as ‘thin slicing’ as you would in the product world; that is, looking at the patterns existing throughout the engineering organization and how the platform and the portal can help provide value quickly.

One of the patterns you may see is that while you may have deep expertise in DevSecOps, platform engineering, SRE and other areas, the wider perspective on the status and quality of individual processes or jobs to be done can be harder to gauge. That can be for areas such as knowledge management, or understanding what your local testing environment looks like. The question is how you optimize these areas of your SDLC because the chances are that nobody owns them. So the task at hand should be to measure these and iterate on them making them better over time. To do that, you need to gather all kinds of data; understand your value stream, map them, find ways to reduce waste and friction, along with using quantitative and qualitative data to measure impact. Then, you can identify solutions, measure them and iterate on them.

‍A Culture of Continuous Improvement

While considering a portal MVP, you should remember that developer experience or platform engineering is a journey that never really ends. It is more of an enablement journey with a goal of driving continuous improvement. For example, you may have come to the conclusion, through the data you have collected, that you have a terrible mean time to recovery.  Upon tracing it back, the problem lies in the fact that you don’t have good testing environments or test data. 

A good MVP for a portal may begin with something like building ephemeral testing environments. Though this can't be done in a vacuum, care should be taken to account for other practicalities such as who is responsible for configuring, maintaining, and its overall execution. Standing up a tool is the easy part, understanding the process and how people interact can be much harder. If you stand something up that hasn’t been configured or no one really owns it, then is it really creating any value? Or is it actually having a negative impact on your overall developer experience?.

Validating Assumptions

When thinking about your portal MVP, it is very easy to assume what problems exist across your ecosystem. It’s also easy to start “fixing” these problems with a portal, only to find out that noone wants to use it.  This is a classic example of a lack of product thinking.  While some of your assumptions may be correct, they may not get over the usefulness hurdle and get people to actually change the way they operate.  

In the product space there are three hurdles that you have to overcome: a person is interested in the product, they are willing to use the product, and they are willing to pay for the product.  One of your assumptions might only get over the first hurdle with the end users not willing to change their process due to lack of value or perceived lack of value..

Collecting feedback

Gathering qualitative and quantitative data is instrumental towards understanding what direction you need to go.  You can survey using tools like DX (getDX.com) to try and better understand developer sentiment.  In addition, you can use DX’s Platform X tool to conduct quick feedback after an event has happened.. For example, if you run a pipeline which includes a CI tool, you can automate messages, via Slack, from a product manager or platform engineer to gather more real time feedback. This feedback mechanism may garner better responses than a full-blown survey, as it requires minimal effort or time to respond.

Using this data to identify hotspots where deeper value stream analysis can be carried out to identify solutions is another key datapoint towards building out a meaningful MVP. Your portal should become a true enablement tool for developers that they want to adopt. If you're struggling with adoption, then this is the place you need to go back to. 

Takeaway

Don’t think about this as a chicken and egg dilemma of what goes first as you’ll be taking a technology-first approach rather than thinking of the portal and platform as products. You have to take a product management approach to work out: what your users need, what your organization needs, what is feasible for your organization, and what the MVP of a platform and portal should look like. To do this, you need the right team structure, ownership and feedback mechanisms in place prior to getting started. 

Next steps

Find out more about what you should consider for your portal here, and how the platform and portal connect, here.

Check out Chris's substack here.