Developer experience metrics: best practices, key components and examples

It may seem like every other day that developers are being assessed by a new metric. There are numerous frameworks and metrics that are used to assess developer productivity. But the validity and tangibility of these metrics have been questioned - particularly by developers who (generally) have a patent distrust towards productivity metrics. 

Productivity is a byproduct of the process itself; and yet this has been left out of the equation for so long. This ‘process’ is the developer experience. In any workplace, those who have less friction and better processes, are more likely to succeed. And so, it makes sense to measure and improve the developer experience. By doing so, organizations can stimulate productivity and have happier employees.

What is developer experience

Developer experience (DevEx) is about ensuring that developers are able to conduct the work they need to do with velocity and quality. This means that rather than focusing on developer productivity metrics alone; such as the number of products developed and the time it has taken to develop those products - organizations are urged to look at the quality of those products and gain a better understanding of the developer’s experience of development - so that they can remove friction points going forward.

So instead of churning out subpar products quickly, burning out developers, focusing too much on cost rather than value, and potentially negatively impacting the overall organization, organizations are focusing their efforts on ways to make their developers' lives easier. How do they do this? Enabling them to get the information they need quickly, be trusted to carry out certain tasks, feel ownership and responsibility over the code they create, and ensure they are able to keep up and properly use modern tech infrastructure for software development. By doing so, they’re able to solve tasks that may have been more difficult previously, better collaborate with peers and be more creative. But more than anything, they’ll be happier, and that will mean they’re more productive.

And recently, research by GitHub and DX, found that DevEx does actually boost productivity and innovation. 

Why do developer experience metrics matter? 

Developer experience metrics focus on getting to the root of a problem or solution, and the impact that the developer’s day-to-day experiences have had on this. For example, better understanding the reason for a sudden change in developer productivity or getting to grips with frustrations that are leading to dissatisfaction.

Developer productivity frameworks like DORA, provide data on software delivery, whereas DevEx metrics look at how the software has been delivered. While productivity metrics are important, they only tell some of the story; using DevEx metrics, managers can better understand the reasoning, and make informed decisions on what to do next. For instance, is the reason for high productivity because of a well-drilled and efficient engineering team, or is it because developers are being asked to work ridiculously long hours - and are burning out (which would cause a devastating impact on software development in future months). 

Key developer experience metrics to track

Measuring developer experience should focus on four key pillars:

Accessibility

Many of the developer’s tools are as good as they say they are in terms of solving the challenge they set out to do - but gaining access to the tool is a nightmare. Sometimes this is because the product has been configured by someone other than the persona who will be using it (without the persona being involved in the process), or because the product hasn’t been enabled for different types of users that may need access.

Findability

Finding relevant information in a presentable way is important. Developers shouldn’t just be given visibility to everything (even if it’s all relevant). They should be able to find the things they want to view at different parts of their day-to-day work. For instance, if they’re looking at a developer portal - at the start of the day they may want to see the PRs to review and incidents to manage so that they can plan your day. But at different stages of the planning to post-production journey, they’ll want to be able to find different information, with the context they need in front of them easily - that can be through different menus, folders, dashboards or search functions. These items should be placed logically and include the right functionality.  

Usability

It’s not just access that developers need; they need to be able to use all of the relevant features, code and documentation easily. A good way to ensure all of the features are accessible is to use feedback tools at the end of crucial points in the journey. This way you can find out how easily your developers can find the relevant information or feature. 

For example: 

• Did you find the information about the API you were looking for?
• Was it easy for you to use this feature?
• Does the tool provide you with the full context you were looking for?
• Was the quality of the documentation up-to-scratch? 

Credibility

The developer tool must be reliable; whether it’s bugs, troubleshooting errors, uptime, or even the ability to ensure that the data within the tool is always accurate - developers can easily sway from thinking that your tool is credible or not, and subsequently this can also affect how they think about the overall team/organization. 

Developer experience metrics and KPIs

Employee turnover and retention rates
You may have an efficient engineering team, but if your developers keep leaving there may be an issue with the developer experience.

Developer Satisfaction score
By surveying your developers on a regular basis, you can check in with them to see how satisfied they are with a platform, tool, service or feature - and this can provide you with some insight into their current satisfaction.
Regular surveys about the overall developer experience and correlating this with employee turnover and retention rates is key. 

Onboarding time
Ensuring that the time to onboard is swift (while also ensuring that developers are satisfied with their knowledge and use of the tool/service they’re using). Organizations may relate this to Time To First Hello World (TTFHW).

Platform stability
Uptime and reliability should be monitored, ensuring developers can focus on coding and solving problems.

API Response Time and Efficiency
Speedy responses mean less frustration, and more productivity for developers.

Breadth and Quality of Documentation
Is the documentation for each tool, feature and action all there? Is it easy to follow and complete? 

Developer experience best practices

There are numerous ways you can get started in using developer experience metrics, but organizations can only rely on these metrics if best practices are put in place. These include: 

Identify bottlenecks in the developer workflow

A better developer workflow leads to a better developer experience. Therefore, optimizing the workflow - from planning to post-production operations - is crucial. This means identifying which stages of this journey require changes, and then prioritizing these.  Some bottlenecks will be known to both engineers and developers, while others require a format where engineers can provide information about any barriers they are experiencing.

Provide autonomy

Organizations can empower developers to access and use resources, environments and tools or perform specific actions through self-service. Self-service actions could include scaffolding a new service, provisioning a new cloud resource, or many other examples. These will inevitably help to improve the developer workflow, and subsequently result in a better developer experience.

Select the right metrics

Once you’ve found your bottlenecks and prioritized your objectives, it’s time to select the metrics that work for your organization. Communicating with the team to ensure that these are agreed upon and properly defined are important steps to a better developer experience. At this stage, it may be worth looking into frameworks such as SPACE, which are focused on DevEx metrics.

Set the baseline

Benchmarks are needed in order to know what the minimum expectation of ‘good’ looks like. This should also be part of the metrics conversation that the engineering team, as well as other stakeholders that could be impacted, should be having. 

Encourage a culture of continuous improvement

It’s important to remember that improving developer experience is a journey with no real end destination, but rather a way to drive continuous improvement. Therefore, short-term results are not the aim here, and neither is a ‘big bang’ approach. This means managing expectations that there will be gradual improvements - with education along the way. 

Feedback loops are crucial

Without feedback from developers, platform engineering and developer experience initiatives will fail. You need their buy-in, but you also need their insights; they’re the ones who are running into roadblocks or have limitations. So, in order to improve their experience, you need to build feedback loops into every step of the journey. 

Check out our list of the top ten developer experience tools, here.