How we use our own developer portal: being our worst critics

October 3, 2022

How we use our own developer portal: being our worst critics

Ready to start?

Introduction

At Port we take Developer Experience seriously. Our goal is to provide an exceptional developer experience to our customers’ developers, so that retention, productivity and the joy of a job well done will ensue. But at the same time we have developers of our own, and they also deserve a better developer experience. This requires us to use our product as our very own internal developer platform.  

We use Port to form our internal (builder-based) Software Catalog, and to increase observability, ownership and developer independence.

This is all part of our dogfooding philosophy. In product development, “dogfooding” which originates from “eating your own dog food” means using your own product as a way to evaluate it fairly and critically, the same way a client would.

We believe that dogfooding is a must in order to develop a mature and efficient platform that improves the developer experience of an organization.

We use Port for a variety of use cases, including:

  • Deployment tracking;
  • Environment locks;
  • Log source consolidation;
  • Allow developers to provision development environments that will self-terminate (TTL);
  • Service health and maturity dashboards;
  • Service architecture mapping;
  • Developer self-service actions (such as provisioning resources, scaffolding a microservice etc);
  • And more…

In this blog we will focus on two specific use cases:

  • Deployment tracking
  • Environment locks

But, before we dive in, let’s go through a quick refresher about Software Catalogs and Port’s microservice architecture.

{{ebook}}

Defining our software catalog for our internal developer portal

A Software Catalog is a collection of your software, cloud and development resources and assets, including your Kubernetes cluster, your microservices, the packages you use, etc. This is a broad definition, since we believe it’s the only way to get the best value in terms of developer self-service and observability.

Since architectures and their underlying logic and infra vary greatly among engineering orgs, we took a builder-based approach that lets you document your infrastructure in Port exactly the way it is. The end result is a Software Catalog that gives everyone in the organization a single-source-of-truth for the organization’s tech stack, architecture and service ownership.

The core of our software catalog architecture in Port follows a similar layout to the one shown by our CTO, Yonatan in his blog A Microservice Catalog Isn’t Enough:

Of course, our actual Port configuration is more complex and includes other bits and pieces of our infrastructure, but the layout shown above is enough to demonstrate the locking mechanism we use, which we’ll discuss later in this post.

Needless to say that our software catalog keeps ingesting new data about changes, new services, new deployments, self-service actions our developers are making and more, so it’s always live.

Now that we’re all on the same page about Software Catalogs and the Port R&D team Port architecture, let’s move on to our first use case:

Deployment tracking with an internal developer portal

We use a microservice architecture to develop and deploy Port, that means at any given moment there are multiple developers working on different microservices, writing new features, fixing bugs or writing tests.

Such a high development velocity - with multiple deployments to production every day, and multiple microservices constantly changing and improving - makes it difficult to keep track of everything that’s going on. Where is my service deployed? What is the current deployed version and what is the corresponding commit hash? Who made the latest commit? Which services were deployed with regards to a new feature? All these are exactly what deployment tracking is all about.

With deployment tracking, every time our CI/CD deployment process ends, we use our Github Action to report about the new service version that has been deployed, the environment it was deployed to, relevant new service metadata, including the commit hash, a link to the CI job and more.

The result is a configurable dashboard with answers to all of the above questions:

Our dashboard includes some additional links and dynamic data which we use for our own debugging and troubleshooting of our live services:

  • Link to Cloudwatch logs
  • Link to Sentry issues
  • Link to Logrocket sessions
  • Link to the latest X-Ray traces and service map
  • A health check status which updates according to our Canary tests

While this dashboard might seem complex and daunting to create, in Port it is extremely simple, by using the powerful tools Port provides such as Mirror Properties and Formula Properties. DevOps and developers need less than 5 minutes of setup before a fully fledged, dynamic, customizable and functional dashboard is ready to go.

Without Port, tracking all of our deployments in one convenient place would have been much harder and would have involved some manual updates. Having  a clear and configurable view of our deployments makes our developers’ lives much easier.

Environment locks using an internal developer platform

Sometimes a developer needs to evaluate a feature in a shared environment with some live data, but without fear of other developers stepping on their toes.

In addition, in some cases you want to disable deployments to production altogether, for instance if you want to troubleshoot some issue and isolate additional code changes from the equation, or you’re demoing the system in real-time and want to avoid any unexpected changes.

For these use cases and more, we use Port as a “locking” layer over our environments.

Here is our Deployment Configs dashboard, which shows which microservices are locked for deployment and in which environment:

Just like the previous dashboard, this one is also very simple to create in Port, and while it is fully functional, it will take you no more than 5 minutes to set up.

In addition, by using Port’s Changelog capabilities, when a developer changes the “locked” field on one of the deployed services, a slack message is sent to a dedicated channel, and an audit log of the status change is also written to Port:

And finally, a locked environment is truly locked, using our own Github Action, when a deployment process runs in our CI, we check for the value of the locked field, and if the service is locked, the deployment will stop:

Here is a live example:

Conclusion and what’s next for Port

We created Port because we wanted to make the developer experience better. Today we’re using  Port to help us develop faster and better features, so that our customers can superpower their developers and their developer experience.

We use our own product every day, both during development and for managing our very own production infrastructure. We believe that by being our own worst critics we can really perfect our platform and deliver the best internal developer portal.

Both of the examples shown in this post about how we use Port can also be found in our documentation, deployment tracking is implemented as part of the software catalog use case, and environment locking is featured in a use case doc page of its own.

I hope this small example of the power of Port helps you think of ways to improve the developer experience in your organization.

Stay tuned for future blogs about our dogfooding efforts - some of our future plans include adding real-time metrics to our running services, using self-service actions to give our developers even more independence and more.

{{cta}}

{{cta_1}}

Check out Port's pre-populated demo and see what it's all about.

Check live demo

No email required

{{cta_2}}

Contact sales for a technical product walkthrough

Let’s start
{{cta_3}}

Open a free Port account. No credit card required

Let’s start
{{cta_4}}

Watch Port live coding videos - setting up an internal developer portal & platform

Let’s start
{{cta_5}}

Check out Port's pre-populated demo and see what it's all about.

(no email required)

Let’s start
{{cta_6}}

Contact sales for a technical product walkthrough

Let’s start
{{cta_7}}

Open a free Port account. No credit card required

Let’s start
{{cta_8}}

Watch Port live coding videos - setting up an internal developer portal & platform

Let’s start
{{cta-demo}}
{{reading-box-backstage-vs-port}}

Example JSON block

{
  "foo": "bar"
}

Order Domain

{
  "properties": {},
  "relations": {},
  "title": "Orders",
  "identifier": "Orders"
}

Cart System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Cart",
  "title": "Cart"
}

Products System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Products",
  "title": "Products"
}

Cart Resource

{
  "properties": {
    "type": "postgress"
  },
  "relations": {},
  "icon": "GPU",
  "title": "Cart SQL database",
  "identifier": "cart-sql-sb"
}

Cart API

{
 "identifier": "CartAPI",
 "title": "Cart API",
 "blueprint": "API",
 "properties": {
   "type": "Open API"
 },
 "relations": {
   "provider": "CartService"
 },
 "icon": "Link"
}

Core Kafka Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Kafka Library",
  "identifier": "CoreKafkaLibrary"
}

Core Payment Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Payment Library",
  "identifier": "CorePaymentLibrary"
}

Cart Service JSON

{
 "identifier": "CartService",
 "title": "Cart Service",
 "blueprint": "Component",
 "properties": {
   "type": "service"
 },
 "relations": {
   "system": "Cart",
   "resources": [
     "cart-sql-sb"
   ],
   "consumesApi": [],
   "components": [
     "CorePaymentLibrary",
     "CoreKafkaLibrary"
   ]
 },
 "icon": "Cloud"
}

Products Service JSON

{
  "identifier": "ProductsService",
  "title": "Products Service",
  "blueprint": "Component",
  "properties": {
    "type": "service"
  },
  "relations": {
    "system": "Products",
    "consumesApi": [
      "CartAPI"
    ],
    "components": []
  }
}

Component Blueprint

{
 "identifier": "Component",
 "title": "Component",
 "icon": "Cloud",
 "schema": {
   "properties": {
     "type": {
       "enum": [
         "service",
         "library"
       ],
       "icon": "Docs",
       "type": "string",
       "enumColors": {
         "service": "blue",
         "library": "green"
       }
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "system": {
     "target": "System",
     "required": false,
     "many": false
   },
   "resources": {
     "target": "Resource",
     "required": false,
     "many": true
   },
   "consumesApi": {
     "target": "API",
     "required": false,
     "many": true
   },
   "components": {
     "target": "Component",
     "required": false,
     "many": true
   },
   "providesApi": {
     "target": "API",
     "required": false,
     "many": false
   }
 }
}

Resource Blueprint

{
 “identifier”: “Resource”,
 “title”: “Resource”,
 “icon”: “DevopsTool”,
 “schema”: {
   “properties”: {
     “type”: {
       “enum”: [
         “postgress”,
         “kafka-topic”,
         “rabbit-queue”,
         “s3-bucket”
       ],
       “icon”: “Docs”,
       “type”: “string”
     }
   },
   “required”: []
 },
 “mirrorProperties”: {},
 “formulaProperties”: {},
 “calculationProperties”: {},
 “relations”: {}
}

API Blueprint

{
 "identifier": "API",
 "title": "API",
 "icon": "Link",
 "schema": {
   "properties": {
     "type": {
       "type": "string",
       "enum": [
         "Open API",
         "grpc"
       ]
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "provider": {
     "target": "Component",
     "required": true,
     "many": false
   }
 }
}

Domain Blueprint

{
 "identifier": "Domain",
 "title": "Domain",
 "icon": "Server",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {}
}

System Blueprint

{
 "identifier": "System",
 "title": "System",
 "icon": "DevopsTool",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "domain": {
     "target": "Domain",
     "required": true,
     "many": false
   }
 }
}
{{tabel-1}}

Microservices SDLC

  • Scaffold a new microservice

  • Deploy (canary or blue-green)

  • Feature flagging

  • Revert

  • Lock deployments

  • Add Secret

  • Force merge pull request (skip tests on crises)

  • Add environment variable to service

  • Add IaC to the service

  • Upgrade package version

Development environments

  • Spin up a developer environment for 5 days

  • ETL mock data to environment

  • Invite developer to the environment

  • Extend TTL by 3 days

Cloud resources

  • Provision a cloud resource

  • Modify a cloud resource

  • Get permissions to access cloud resource

SRE actions

  • Update pod count

  • Update auto-scaling group

  • Execute incident response runbook automation

Data Engineering

  • Add / Remove / Update Column to table

  • Run Airflow DAG

  • Duplicate table

Backoffice

  • Change customer configuration

  • Update customer software version

  • Upgrade - Downgrade plan tier

  • Create - Delete customer

Machine learning actions

  • Train model

  • Pre-process dataset

  • Deploy

  • A/B testing traffic route

  • Revert

  • Spin up remote Jupyter notebook

{{tabel-2}}

Engineering tools

  • Observability

  • Tasks management

  • CI/CD

  • On-Call management

  • Troubleshooting tools

  • DevSecOps

  • Runbooks

Infrastructure

  • Cloud Resources

  • K8S

  • Containers & Serverless

  • IaC

  • Databases

  • Environments

  • Regions

Software and more

  • Microservices

  • Docker Images

  • Docs

  • APIs

  • 3rd parties

  • Runbooks

  • Cron jobs

Starting with Port is simple, fast and free.

Let’s start