Improving developer workflows through a better developer experience

Introduction

Everyone knows that a bad developer workflow kills developer productivity (and happiness, and retention…). 

But what is a developer workflow?

The developer workflow maps to the Software Delivery Lifecycle. It involves planning the day, coding, scaffolding, testing, deploying, shipping features and then monitoring and operating in production. 

The developer workflow paradox

Being a developer is serious business: there are many responsibilities, disciplines and moving parts you should be aware of. “Shift left” is a euphemism that captures some of the complexities and breadth of responsibilities faced by developers.

Don’t get me wrong, developers love all this responsibility. They want to truly own the application they build, together with the good and the bad. This is why “you build it, you own it” has always resonated with developers. However, one thing isn’t resolved: true ownership is a difficult thing. When it breaks, everyone is frustrated. 

This paradox - the disconnect between the willingness to own what was built and the true difficulty of really accomplishing it is one of the main drivers behind platform engineering. The proliferation of developer tooling, the existence of many microservices, APIs, environments, standards - all of this makes developer workflows even more complex, since there are so many things to connect with and to know.

Platform engineering - and internal developer portals -are a response to the question of how to make the developer workflow better. By streamlining the workflow around core development - from devops through tooling, environments, APIs, packages and more - the developer experience is improved, since developers can just code instead of dealing with all the rest. 

Fixing the developer workflow - first steps

There are some unassailable truths about the developer experience and, therefore, the developer workflow. Some basics need to be covered in order for developers to work productively and stay motivated. Let’s get this out of the way and then explain how developer workflows can be improved with platform engineering.

• Avoid slow builds and let developers be able to see what they did quickly, regardless of whether it's local or not. That’s why it's always worthwhile to invest in tools that let developers build, test, merge and deploy quickly with no configuration hassles. They’ll be able to immediately see what works and what doesn’t, and these feedback loops will drive a better experience and productivity.
• Minimize tech debt, so that developers don’t encounter “surprise” dependencies, insufficient test coverage or an inability to make changes as a result of prior technology-debt-inducing actions. 
• Strive towards centralized feedback and collaboration tools to make work easier. Coding is a group sport and coders should find it easy to see up to date requirements, documentation, comments and more. 
• Make the infrastructure part easy. If infrastructure isn't easy, developers will spend a lot of time trying to control it, while managers will be frustrated with the lack of standards compliance, whether production readiness, cost or application security compliance. And this is where platform engineering and internal developer portals come to the rescue.

Where you need to focus to optimize the developer workflow

The number one thing developers should do is write code. However, there are many more things developers need to do. This is where the developer workflow problem begins.

Platform engineering is a solution to the developer workflow problem, since it makes the infrastructure part easy. In fact, the goal here is to provide everything the developer needs - from code to production - in one place, so that developers:

• Don’t need to context switch
• Know what the golden paths are
• Are served the infrastructure data in a way that’s abstracted and personalized to them
• Don’t need to know tools and technologies that aren’t core to their work
• Can self-serve most actions with a golden path with no need to tap into tribal knowledge or send a ticket to devops. 

To explain how this can happen, let’s break down the responsibilities of a typical developer to determine which parts of the workflow can and should be fixed. Let’s look at the devops software delivery process diagram

‍

For each of the steps the developer is also responsible for quality, security, cost, compliance, documentation, the user experience and continuous improvement. This highlights the many responsibilities that rest squarely on developer shoulders. 

‍

The two dimensions of the developer workflow 

Essentially, developers need to navigate two primary dimensions → 

- External: Considering end user customer needs.

When it comes to the end user, the developer needs to make sure that the delivered features meet expectations, and perform as expected.

- Internal responsibility: Considering organizational needs.

When it comes to the engineering organization the developer belongs to, developers need to ensure that their work is accessible, well-documented, and easily understood by colleagues.

In addition, developers need to make sure they develop code that complies with organizational standards to ensure the organization isn’t exposed to cost, security or compliance issues. Internal responsibilities are key to optimize the R&D organizational efficiency and will often affect developer productivity metrics like DORA or SPACE.

Everything developers do maps to these two dimensions:

• Take engineering quality as an example. From the external point of view, quality means features that work well, delivering an application that is neither buggy nor slow. From the internal POV it means quality code and tests, ensuring that tests are not only effective but also efficiently designed to facilitate quick builds and easy comprehension by fellow developers.

• Another example is documentation. Making sure to document a feature, a microservice or API is mandatory to ensure collaboration and R&D work, while external documentation is also required in case end users need it. 

Take a moment to review the table above and you’ll see that each part maps to both external and internal dimensions. 

Organizing everything in one place can fix the developer workflow

Plan

Everything begins with a plan. Planning your day as a developer requires taking many things into consideration. In addition to the increasing number of tickets you have in the current sprint, internal meetings, PRs to review and incidents to manage and onboarding new team members, you also need to find some time to code. Having it all in one place can make prioritizing and planning easier.

Think about coming into work and seeing all of the following in one place instead of several interfaces:

• Pull request reviews 
• All relevant Jira tickets 
• Future on-call shifts
• Manager initiatives (e.g. need to update helm charts, tag S3 buckets etc)
• Security issues by priority

Code

During the coding phase, developers need to begin to account for APIs, dependencies and a microservices architecture. Needless to say that it’s often difficult to find the documentation for the desired service. Developers can reach out to the service owner, if they know them and if they are still part of the company. In addition, to even scaffold a new service or a package, developers need to perform a set of tasks to make sure it can reach production with all the standards baked in. In many cases, this ends up as a ticket to devops and causes unnecessary waiting, which doesn’t bode well for productivity or motivation. In the platform engineering world, a software catalog provides an answer to many of these questions.

Additional things that belong to this phase and can also be put in the same place are:

• Find API documentation
• Add IaC to service
• Scaffold new service
• Deploy a development environment
• Manage secrets

Build

During the build process, developers need to troubleshoot the application delivery process. But often, errors are not indicative and developers end up sending a slack message on the entire R&D channel which poses another context switch for hundreds of developers at once. This can be frustrating for all. Permissions can be requested and not always granted, and the wait becomes even longer.

Test

Testing code as part of the delivery process is crucial. There are many types of tests to perform, but a common methodology is to use ephemeral development environments which are also known as feature-branch environments. 

Sometimes developers leave those environments up and running which can significantly impact cloud cost bills. In addition, new developers may find it difficult to set up an ephemeral environment that will usually require knowledge of an internal CLI or an outdated guide on confluence.

Deploy

Feature flags, canary, blue green are some of the common methods to gradually roll out a new version. For developers, being able to operate gradual deployments requires a deep knowledge and understanding of the way this ought to be implemented. Managing the thresholds or toggling will often require configuration changes and a deep familiarity with those tools. Having these offered as self-service actions in one interface can make work much easier.

Operate

One of the things developers need to do when an application is in operation is to investigate the root cause of an issue. Often, due to the disparate nature of software development, especially in cloud-native, microservice architecture environments, understanding recent events and changes to a service that resulted in an outage is hard since data is siloed across the infrastructure and developer workflow tools. To do that you have to be a super-detective and create one of those boards created in detective movies. 

‍

Key Strategies to drive Developer Workflow efficiency 

Give developers carrot, not a stick

Developers deserve a good solution to their developer workflow problems, and they should be allowed to work in the way that best fits their needs. Once you have an internal developer portal and platform in place (or any homegrown solution or CI-based interface, whatever you choose) you may want to force them to use that solution.

But the truth is that you need to provide developers with a carrot. Understand what they really need and provide it to them, driving adoption in a natural way instead of forcing a new way to work without necessary explanation.

Don’t try to solve everything at once

While it may be tempting to try to solve it all at once, building a platform to drive a better developer workflow takes time. Launch a simple minimally viable solution to developer needs, analyze and iterate. Don’t try to build a perfect tool and then launch it to discover no-one uses it. To do this, use a portal as product approach - or a platform as product approach. And when you do that, take care to include developers in the discussion and solicit their feedback and opinions. You may well discover the killer feature for your internal developer platform.

Why Developer Workflow Optimization Matters

Focusing on the above you may well reach the following:

Don’t forget to use an internal developer portal to drive platform engineering

Internal developer portals lie at the heart of software development workflows. They create a native experience for developers that helps them get from planning to production with ease. 

In a way, an internal developer portal revolutionizes the way developers ship software by giving them anything they need to truly own the applications they develop.