It’s a trap - Jenkins as self-service UI

Introduction

Jenkins is an excellent tool for CI. I personally love it, and nothing in this post should be taken as a suggestion not to use Jenkins. By all means, use it, but not as a self-service UI, especially as engineering teams grow and your software/infra becomes more complex. Jenkins self-service won’t create a better developer experience (although you may briefly believe in that during a honeymoon period), and it certainly won’t make your DevOps resources happier or better utilized. It can also create visibility and security issues. 

Let’s examine precisely how Jenkins is used as a Self-Service UI for developers and non-developers and understand what the issues are.

‍

This diagram says that once you’re larger than a 30+ people engineering team, you will have issues using Jenkins as a self-serve UI. Over 300 people may get you in trouble because you aren’t providing guardrails and context to the use of Jenkins and that may create a mess, slow everything down or both. I’ll add to that later.

Today, almost every engineering organization relies on Software / Infrastructure for daily operations. Developers scaffold a new microservice, sales reps need to prepare a demo environment, customer success needs to act on a customer’s environment, developers need a VM or an environment,  etc. 

With small engineering teams (1-30 in the diagram), allowing everyone to interact with the infrastructure is fairly simple. Everyone is familiar with the tech stack, and the setup is still naive. If someone isn’t sure what to do, they just reach out and ask. 

As companies start to grow, teams start to formulate, and organizational standards become mandatory. The devops team gets set up. 

This is when you start having dedicated Slack channels for requests, tickets, and meetings with the current on-call DevOps person to fulfill requests. This makes life hard for both developers and DevOps people. They need each other, but many information and best practices gaps need bridging. A lot of manual work is needed to set up an environment, developers need help deploying infra and more...

At some point, DevOps want to bring in self-service for repetitive tasks. After all, isn’t this what the developer experience is all about? Some requests become automated, using scripts, cli’s, and Jenkins jobs.

It’s tempting: the logic exists within the Jenkins job, and all that remains to be done is a form. Once the form exists - Boom! - here’s a self-service UI for developers to consume infra-related services. Is that a great developer experience and a good more for infra teams?

The answer is no….  life ain’t that simple.

‍

Here are 10 Common Self-Service Jobs that organizations throw at Jenkins

1 —> Spin up DevEnv 

2 —> Add a Secret to a Microservice 

3 —> Resize auto-scaling group

4 —> Spin up Demo Environment

5 —> Scaffolding a microservice

6 —> Create a VM for 5 days

7 —> Deploy to Staging

8 —> Get 1-hour Permissions to an S3 Bucket 

9 —> Whitelist IP

10 —> Data ETL pipeline

‍

Jenkins is not built for Self-Service

Lack of Context & State

‍

Every Jenkins job is stateless.

This means that once a job finishes executing, the resource created/deleted/modified by the job is not something Jenkins keeps a state for and continuously syncs against.

Seeing the context of the resource the job interacts with is essential. The representation of the consumed software (provisioned environments, permissions) is a must to understand what is going on. 

Let’s use an example to express the nature of state & context in Jenkins →

‍

Let’s assume we have a Jenkins job to provision an On-Demand DevEnv, used by developers to test new features in the cloud. 

Once the developer creates the environment and begins using it, they would like to see the relevant details with regards to the environment: 

• Link to logs
• Link / IP address to the environment
• TTL
• Status (Active, Terminated, Scheduled, Error)
• etc…

‍

Having a state & context around the environment created by the job is crucial for developers to get around the environment and get the information they need.

Jenkins’s very nature as a CI tool means it doesn’t hold the environment's state.

It’s more tricky when the day after comes (it always comes). If the developer wants to extend the TTL of the environment to keep working on it, or even delete the environment since they finished interacting with it - it becomes ugly..

‍

Because each job is stateless, the developer must carry a unique identifier to perform such actions. 

This is one use case with 3 simple actions to solve the lack of state. But when we think about dozens of self-service jobs, it quickly becomes chaos. Tracking and interacting with the software personas within the organization becomes impossible. Questions arise, and mistakes are made. This forces DevOps to either (1) become a bad cop, (2) act as a bottleneck, or (3) let things happen and hope for the best.

When someone needs to fetch information regarding a job executed a few days ago, usually, they will end up scrolling through the build history to find the relevant piece of information.

‍

Flat User Interface

‍

Jenkins provides a limited set of UI components with very limited capabilities to build forms or wizards..

All these things aren’t possible in Jenkins, and this isn’t a conclusive list:

1. Input validation (both RegEx & validation against 3rd parties)
2. Inputs with data fetched from 3rd parties (for example, having a drop list with all the S3 Buckets that the logged-in user has permission for)
3. “If this, then that” wizards so that you can provide developers with a dedicated form according to how they filled out the previous wizard step.

‍

The lack of such UI capabilities results in very “open” forms with free text inputs. This creates very long forms that are difficult to understand. What happens next is an abundance of mistakes in the filled-up form, and even more frustration when something goes wrong due to mistakes in the form.

Besides the forms & inputs, a significant part of the Jenkins UI is the (in)famous “Console Output.”

‍

This is where developers lose it. Jenkins provides them with a very noisy thread of logs when something goes wrong. They are usually very technical and infra-related, making it difficult for the user to troubleshoot independently. The developer will reach out to the DevOps to fix it, even if the job failed due to something within the developer’s domain. Again, this is the opposite of both a good developer experience and good use of DevOps resources.

When a job fails, it usually looks like this →

‍

Coupled UI, Backend & API

‍

From an architectural standpoint, the use of Jenkins as a Self-Service portal for developers will contain several components:

1. UI - The forms/wizards developers will see
2. API - Jenkin’s API to interact with jobs
3. Jobs - The actual implementation of the task to be done. It’s the “Backend” of your self-service system.
4. DB - This is where the state of provisioned resources is kept. In the case of Jenkins, since it's not built for stateful self-service, this layer is missing but still worth mentioning since it's a crucial part of a stateful self-service system.

‍

As it's easy to understand from the illustration above, in Jenkins, all the software components are tightly coupled. As a result, if you want to change/replace one of the components, you are bound to the limitations of Jenkins.

‍

Organizational standards are hard to implement

Security, cost, compliance, and quality are crucial for everyone. 

In Jenkins, there are plugins like pipeline-input-step. They add a layer of manual approval for executing jobs. 

But usually, you would like more complex guardrails around self-service capabilities for developers. For example:

• TTL for permission granted / Ephemeral environment
• Allow each developer to perform an action X times a week
• Allow each team to consume resources that do not cross $XX a month
• Etc…

{{ebook}}

‍

The downside of using Jenkins as a Self-Service Portal

Bad Developer Experience

A good developer experience is crucial for an engineering culture that works well, is productive, and quality-centric.  Given what we’ve seen with Jenkins’ limitations as a self-service platform, developers are bound to become frustrated. They are not happy customers, and DevOps can’t focus on “product management” of a developer experience - they’re too busy putting out fires. 

It’s a Jungle.

Because Jenkins doesn’t keep state and isn’t aware of the assets it created/modified, or deleted, it becomes impossible to keep track of all the consumed resources used by developers.

This is where weekly scripts that terminate environments/cloud resources or permissions come in. Still, there probably remain some dev environments that still exist for no reason.  I’ve heard many stories of open orphan environments no one tears down, because of the hassle of building them again.

‍

High potential for mistakes

Since the Jenkins interface is very open (free text input, long forms, lack of guardrails & policies, etc.), developers are prone to make mistakes and use the self-service the wrong way 

‍

Compliance & Security Issues

Jenkins was designed to let anyone do things like install plugins, build, test and deploy. Speed and autonomy are great but at some point, especially with complex systems, it all becomes a mess, which will slow down work and cause unnecessary friction. 

For any plugin installed with Jenkins, it’s impossible to tell who the owner is, how often it is used, and what depends on it. Some plugins have dubious quality, but since Jenkins allows anyone to install them, vulnerabilities can be accidentally introduced. You can try to solve it with permissions and approvals, but that grows friction and creates more tasks for teams.

‍

A Developer Portal is what you need 

A Developer Portal is a unified, self-service view of the architecture, microservices, software, resources, and dependencies. It doesn’t just provide visibility. It allows self-service with guardrails and consolidates everything developers need so that it's quick and easy to find. This saves a ton of time for DevOps too because developers now have the context and tools to serve themselves.

Do not throw Jenkins away. Remember that Jenkins & a developer portal can be good friends and play together perfectly.

‍

Decoupled UI, Backend & API, Database

‍

As you can see in the illustration above, in an architecture where a developer portal takes the UI, DB & Controls responsibility, Jenkins becomes a great integration to perform the actual action. 

This way, you can implement your Self-Service “Backend” on Jenkins, and the rest is done within the developer portal. 

Such an architecture will allow you to decouple the UI layer from the backend layer, manage them separately, and enjoy the Single Responsibility Principle we usually follow as developers.

‍

Stateful

A developer portal is designed to be aware of the resources created & managed by developers. Having the state managed in the developer portal allows you to implement stateless jobs in Jenkins and build a reusable set of jobs to be integrated by the developer portal.

The stateful nature of the developer portal will allow every user to see all the different assets consumed by it, the status, the relevant metadata, and more. 

‍

Rich UI

A developer portal solution should provide DevOps with a rich set of UI components to be used by the developers. 

Things like:

- Input validation (Both RegEx & validation against 3rd parties)

- Inputs with data fetched from 3rd parties (for example, having a drop list with all the S3 Buckets the logged-in user has permission for)

- “If this, then that” wizards allow you to provide your developers with a dedicated form according to how they filled out the previous wizard step.

These things will allow developers to consume services simple while minimizing mistakes developers might make while doing so.

‍
Responsible Self-Service

A Developer Portal will allow you to self-service with responsibility in mind.

You define your organizational standards, costs, and quality pre-requisites. That’s why DevOps choose what to show developers and teams and set the guardrails needed to let developers roam freely and securely. Allow self-service only when actions conform with policies, approvals, and service maturity. This creates a culture of quality and compliance, and with a better developer experience, there will be fewer issues and frustrations for DevOps engineers

{{cta}}

‍

Summary 

The future is about complex architectures that need a catalog - a service/resource catalog - but the catalog is there to power self-service, not exist on its own. The world is complex, and there is so much complexity a human or a developer can contain, track ownership of, ascertain standards etc. If you push the limits, you’re into cognitive load, impossible developer onboarding, and a strained developer-devops relationship. 

Everyone agrees that great developer experiences must happen, but using Jenkins as a self-service enabler for developers just doesn’t cut it. The democratic approach of Jenkins is too open and can make things worse through flat forms and statelessness. Developer portals that are built on a visible microservice catalog and allow responsible and standards-based self-service are where the future is. With Jenkins, of course, as a backend.