Software Catalog

Software Catalog Templates: pre-made taxonomies for microservice, resource and software catalogs

March 23, 2023

Dvir Segev
Dvir Segev
Software Catalog Templates: pre-made taxonomies for microservice, resource and software catalogs
Example H2

Ready to start?

Introduction

‍

Port’s secret power (well, it’s not that secret) is its software catalog. Port’s software catalog is a central metadata store for software, environments, resources, pipelines, CI/CD, cloud resources and more. The fact that it contains all this data supports all kinds of cool use cases from a CI/CD catalog to creating a Kubernetes catalog.

‍

If you follow what we write about the data model underlying the software catalog - the heart of our no-code approach - we place a lot of emphasis on being able to choose your own software catalog. We talk about being able to use a “running service” blueprint and how to think about a taxonomy for your software catalog. But should we provide platform engineers that set up Port with a blank slate to begin defining their software data model? Up until now, we did. Our demo, for instance, contains an elaborate set of blueprints that form the software catalog. But we do know that each type of engineering environment and use case works best with specific approaches, or templates, to the software catalog data model. That’s why we’re announcing templates.

‍

Why software catalog templates?

With templates, we’re truly delivering on the ability to quickly set up your internal developer portal with a software catalog template that best works with your use cases, data model and needs. We don’t expect your choice of template to be static - you can play with templates and improve them, introducing any type of blueprint you need and creating a super-opinionated software catalog that fits how your engineering organization works.

‍

In the several days since we’ve started using templates, we’ve seen a huge change in how people interact with Port. But there’s more to it than just a faster and better setup for our clients. What this does is pave the way to think about Internal developer portals and how they fit with the core use cases in platform engineering. We’re certain this isn’t the last time we’ll explain the different templates we’ve started with and the best practices associated with them, because each company can find itself in one or some of the templates. 

‍

{{cta_4}}

‍

Six options to build a software catalog and counting…

This post will explore the six options covered in our first software catalog templates, letting you build a software catalog (through predefined Port Blueprints) by

‍

  • Mapping git provider data
  • Using GitOps
  • Connecting your CI/CD pipeline
  • Mapping your Kubernetes ecosystem
  • Mapping your cloud ecosystem
  • Using IaC

‍

Once you set up Port, you’re asked to select a template (or begin writing blueprints yourself). Whatever choice you made isn’t final. You can always edit templates or combine them.

‍

Let’s do a deeper dive into each of the different templates. We’ll deal with IaC based software catalogs in a later post.

‍

A microservice catalog using git providers

This is the classic “microservice catalog” use case which is about giving developers visibility into the software development lifecycle.  In this case, the microservice catalog will collect data from git providers and populate the blueprint templates, creating software catalog entities that will show developers the information they need, in-context, with the right abstractions, permissions and more. This template also comes with a built-in maturity scorecard to ensure quality standards are met.

‍

It’s pretty simple to add additional blueprints to this template, turning the microservice catalog into a metadata-rich software catalog containing additional information about CI/CD, cloud resources, Kubernetes and more.

‍

‍

A software catalog using GitOps

This template uses GitOps and generates a backstage-style C4 software catalog. The strength of this approach is that it goes beyond microservices, and catalogs underlying cloud resources, the deployments and the related dependencies. Note the API blueprint, which shows your API specifications through a swagger UI, and the built-in scorecards.

‍

As always, you can add additional blueprints, such as running service and any other blueprint that comes in handy.

‍

‍

A CI/CD based microservice catalog

This method uses CI/CD to build a microservice catalog. This type of catalog unifies CI/CD data that is spread across multiple systems and repositories, so you can see all build and deployment configurations, artifacts, version numbers, environment variables, dependencies and more.  By creating a single source of truth for CI/CD, you can also use it as part of automated workflows and solve version and security issues. For example, notify the relevant developers on Slack when a deployment has failed.

‍

‍

A Kubernetes catalog

This catalog is based on our Kubernetes exporter which supports any type of CRD, such as ArgoCD. This catalog takes the different Kubernetes resources (Deployments, pods, namespaces etc) and maps them into blueprints for a kubernetes catalog, creating Kubernetes entities. It also supports showing the relevant abstractions to developers, in order to reduce cognitive load, for instance, using the “workload” blueprint to abstract the different workload types (replicasets, deployments, daemonsets, statefulsets). To see how you can combine the K8s and microservice catalog templates to create a view of your services and deployments, watch this webinar.

‍

By providing developers with an abstracted view of Kubernetes clusters, some of the complexity can be taken away, and standards enforced through Kubernetes scorecards to avoid misconfigurations. You can also use scorecards for deployment health, pod node placement and CPU limits so developers can troubleshoot issues.

‍

‍

Cloud resource catalog

Based on our cloud exporters this template builds a cloud resource software catalog. It brings live cloud resource data and maps it according to the blueprints into software catalog entities.

‍

Adding cloud resource data into the software catalog enriches catalog data immensely. In the case of AWS you can add data about Lambda functions, S3 buckets, SQS queues, ECS services and anything in the AWS cloud control API (500 resources!). 

‍

Natural additions to this template can be SNS & SQS Topics, S3 Buckets, and RDS DB instances.

‍

‍

What’s Next

Our next template release will also add developer self-service templates to these blueprints. This will allow you to quickly implement popular actions such as scaffolding a microservice, creating a new incident, creating a cloud resource and more. Stay tuned!

‍

{{cta_3}}

‍

Previous article

Next article

{{cta_1}}

Check out Port's pre-populated demo and see what it's all about.

Check live demo

No email required

{{cta_2}}

Contact sales for a technical product walkthrough

Let’s start
{{cta_3}}

Open a free Port account. No credit card required

Let’s start
{{cta_4}}

Watch Port live coding videos - setting up an internal developer portal & platform

Let’s start
{{cta_5}}

Check out Port's pre-populated demo and see what it's all about.

(no email required)

Let’s start
{{cta_6}}

Contact sales for a technical product walkthrough

Let’s start
{{cta_7}}

Open a free Port account. No credit card required

Let’s start
{{cta_8}}

Watch Port live coding videos - setting up an internal developer portal & platform

Let’s start
{{cta-demo-baner}}

Book a demo right now to check out Port's developer portal yourself

{{port-for-backstage}}

Apply to join the Beta for Port's new Backstage plugin

Apply for beta
{{jenkins}}

It's a Trap - Jenkins as Self service UI

Read more
{{gitops}}

How do GitOps affect developer experience?

Read more
{{ebook}}

It's a Trap - Jenkins as Self service UI. Click her to download the eBook

Download eBook
{{cyberark}}

Learning from CyberArk - building an internal developer platform in-house

Read more
{{cta-demo}}
{{reading-box-backstage-vs-port}}

Further reading:

Read: What are the technical disadvantages of Backstage?

Compare: Backstage vs. Port

Example JSON block

{
  "foo": "bar"
}

Order Domain

{
  "properties": {},
  "relations": {},
  "title": "Orders",
  "identifier": "Orders"
}

Cart System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Cart",
  "title": "Cart"
}

Products System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Products",
  "title": "Products"
}

Cart Resource

{
  "properties": {
    "type": "postgress"
  },
  "relations": {},
  "icon": "GPU",
  "title": "Cart SQL database",
  "identifier": "cart-sql-sb"
}

Cart API

{
 "identifier": "CartAPI",
 "title": "Cart API",
 "blueprint": "API",
 "properties": {
   "type": "Open API"
 },
 "relations": {
   "provider": "CartService"
 },
 "icon": "Link"
}

Core Kafka Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Kafka Library",
  "identifier": "CoreKafkaLibrary"
}

Core Payment Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Payment Library",
  "identifier": "CorePaymentLibrary"
}

Cart Service JSON

{
 "identifier": "CartService",
 "title": "Cart Service",
 "blueprint": "Component",
 "properties": {
   "type": "service"
 },
 "relations": {
   "system": "Cart",
   "resources": [
     "cart-sql-sb"
   ],
   "consumesApi": [],
   "components": [
     "CorePaymentLibrary",
     "CoreKafkaLibrary"
   ]
 },
 "icon": "Cloud"
}

Products Service JSON

{
  "identifier": "ProductsService",
  "title": "Products Service",
  "blueprint": "Component",
  "properties": {
    "type": "service"
  },
  "relations": {
    "system": "Products",
    "consumesApi": [
      "CartAPI"
    ],
    "components": []
  }
}

Component Blueprint

{
 "identifier": "Component",
 "title": "Component",
 "icon": "Cloud",
 "schema": {
   "properties": {
     "type": {
       "enum": [
         "service",
         "library"
       ],
       "icon": "Docs",
       "type": "string",
       "enumColors": {
         "service": "blue",
         "library": "green"
       }
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "system": {
     "target": "System",
     "required": false,
     "many": false
   },
   "resources": {
     "target": "Resource",
     "required": false,
     "many": true
   },
   "consumesApi": {
     "target": "API",
     "required": false,
     "many": true
   },
   "components": {
     "target": "Component",
     "required": false,
     "many": true
   },
   "providesApi": {
     "target": "API",
     "required": false,
     "many": false
   }
 }
}

Resource Blueprint

{
 “identifier”: “Resource”,
 “title”: “Resource”,
 “icon”: “DevopsTool”,
 “schema”: {
   “properties”: {
     “type”: {
       “enum”: [
         “postgress”,
         “kafka-topic”,
         “rabbit-queue”,
         “s3-bucket”
       ],
       “icon”: “Docs”,
       “type”: “string”
     }
   },
   “required”: []
 },
 “mirrorProperties”: {},
 “formulaProperties”: {},
 “calculationProperties”: {},
 “relations”: {}
}

API Blueprint

{
 "identifier": "API",
 "title": "API",
 "icon": "Link",
 "schema": {
   "properties": {
     "type": {
       "type": "string",
       "enum": [
         "Open API",
         "grpc"
       ]
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "provider": {
     "target": "Component",
     "required": true,
     "many": false
   }
 }
}

Domain Blueprint

{
 "identifier": "Domain",
 "title": "Domain",
 "icon": "Server",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {}
}

System Blueprint

{
 "identifier": "System",
 "title": "System",
 "icon": "DevopsTool",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "domain": {
     "target": "Domain",
     "required": true,
     "many": false
   }
 }
}
{{tabel-1}}

Microservices SDLC

  • Scaffold a new microservice

  • Deploy (canary or blue-green)

  • Feature flagging

  • Revert

  • Lock deployments

  • Add Secret

  • Force merge pull request (skip tests on crises)

  • Add environment variable to service

  • Add IaC to the service

  • Upgrade package version

Development environments

  • Spin up a developer environment for 5 days

  • ETL mock data to environment

  • Invite developer to the environment

  • Extend TTL by 3 days

Cloud resources

  • Provision a cloud resource

  • Modify a cloud resource

  • Get permissions to access cloud resource

SRE actions

  • Update pod count

  • Update auto-scaling group

  • Execute incident response runbook automation

Data Engineering

  • Add / Remove / Update Column to table

  • Run Airflow DAG

  • Duplicate table

Backoffice

  • Change customer configuration

  • Update customer software version

  • Upgrade - Downgrade plan tier

  • Create - Delete customer

Machine learning actions

  • Train model

  • Pre-process dataset

  • Deploy

  • A/B testing traffic route

  • Revert

  • Spin up remote Jupyter notebook

{{tabel-2}}

Engineering tools

  • Observability

  • Tasks management

  • CI/CD

  • On-Call management

  • Troubleshooting tools

  • DevSecOps

  • Runbooks

Infrastructure

  • Cloud Resources

  • K8S

  • Containers & Serverless

  • IaC

  • Databases

  • Environments

  • Regions

Software and more

  • Microservices

  • Docker Images

  • Docs

  • APIs

  • 3rd parties

  • Runbooks

  • Cron jobs

You may also be interested in

Building the perfect internal developer platform

Building the perfect internal developer platform

Read more
How to use JQ commands to speed developer workflows

How to use JQ commands to speed developer workflows

Read more
Debugging K8s with K8sGPT in your internal developer portal

Debugging K8s with K8sGPT in your internal developer portal

Read more

Starting with Port is simple, fast and free.

Let’s start