Product News

Announcing the GitHub and Bitbucket Exporters

January 5, 2023

Dudi Elhadad
Dudi Elhadad
Announcing the GitHub and Bitbucket Exporters
Example H2

Ready to start?

Introduction

‍

As 2023 begins, things are really warming up at Port.

‍

Our GitHub and BitBucket exporters are our newest features. To understand them in context, we wanted to first explain the interplay of GitOps and internal developer portals and then the power of using exporters to populate the software catalog with GitOps data.

‍

Git and an Internal Developer Portal

Git contains a lot of information about the different entities and components you manage. From readmes, microservice names (repo names if you operate in poly repos), IaC files, K8S manifests, ArgoCD files, issues, API Documentation and pull requests. The list goes on.

On our mission to bring everything into one Internal Developer Portal, for context, self-service and even workflow automations, Git providers are a key data source to deliver on this promise.

‍

Ingesting data into the software catalog in the developer portal from your git provider

A simple ingestion of data from GitHub or Bitbucket would essentially sync the relevant YAML about the relevant component (any of service, IaC or k8s related) to the developer portal. It’s almost manual - whatever was written into the YAML  in Git is sent to the software catalog. 

This isn’t necessarily a bad practice, since requiring a service owner to maintain a YAML file is a good driver of “you build it, you own it”, since it forces the service owners to put the relevant data in the YAML file. But it potentially misses out on the valuable data contained in the Git provider.

‍

GitHub and Bitbucket contain very valuable data that would enrich any software catalog. This information (which our GitHub and Bitbucket exporters fetch) is made of two types:

1. Data about user activity: think of issues, pull requests, who ran which workflow. Perhaps hundreds of these data points are created each and every day. This is valuable real time data which isn’t truly accessible and is at no-one's fingertips. 

2. Data about services, IaC, Kubernetes & everything GitOps. The Git provider contains multiple files with important data about repos, environments and how they are structured. This appears in Terraform files for the environment, helm charts with the K8s setup and the values files for K8s, as well as the argo app. Reflecting this data in the developer portal can provide much needed insight about the state of services and infrastructure - in real-time. 

‍

{{cta_3}}

‍

The GitHub and the Bitbucket Exporters - what are they?

Our new GitHub and Bitbucket exporters are integrations that allow you to automatically collect live data to be digested by Port’s software catalog. These integrations export GitHub or Bitbucket objects (like pull requests, issues, workflows, and repositories) to Port. They also let you map your GitHub or Bitbucket resources into Port’s software catalog. 

This means that you can:

  • List and export existing Git Provider data from your organization, such as pull requests or repositories and more
  • Watch for diff changes in specific git files (commit new file / change / delete) in real-time. The changes will be automatically applied to the relevant entities in the software catalog;
  • Configure which objects are relevant, and how to map and transform objects to the relevant entities in the software catalog.

‍

How does it work?

Port collects data across your entire stack, processes it and shows it just where you want it to appear in the software catalog.

‍

Why This Matters

We’ve already explained our approach of streaming exporter data into the right place in the software catalog instead of to the microservice. Similarly, this means that the relevant Bitbucket or GitHub data will appear in the right place in Port’s software catalog. 

And the best news? We’re working on a GitLab exporter too.

Previous article

Next article

{{cta_1}}

Check out Port's pre-populated demo and see what it's all about.

Check live demo

No email required

{{cta_2}}

Contact sales for a technical product walkthrough

Let’s start
{{cta_3}}

Open a free Port account. No credit card required

Let’s start
{{cta_4}}

Watch Port live coding videos - setting up an internal developer portal & platform

Let’s start
{{cta_5}}

Check out Port's pre-populated demo and see what it's all about.

(no email required)

Let’s start
{{cta_6}}

Contact sales for a technical product walkthrough

Let’s start
{{cta_7}}

Open a free Port account. No credit card required

Let’s start
{{cta_8}}

Watch Port live coding videos - setting up an internal developer portal & platform

Let’s start
{{cta-demo-baner}}

Book a demo right now to check out Port's developer portal yourself

{{port-for-backstage}}

Apply to join the Beta for Port's new Backstage plugin

Apply for beta
{{jenkins}}

It's a Trap - Jenkins as Self service UI

Read more
{{gitops}}

How do GitOps affect developer experience?

Read more
{{ebook}}

It's a Trap - Jenkins as Self service UI. Click her to download the eBook

Download eBook
{{cyberark}}

Learning from CyberArk - building an internal developer platform in-house

Read more
{{cta-demo}}
{{reading-box-backstage-vs-port}}

Further reading:

Read: What are the technical disadvantages of Backstage?

Compare: Backstage vs. Port

Example JSON block

{
  "foo": "bar"
}

Order Domain

{
  "properties": {},
  "relations": {},
  "title": "Orders",
  "identifier": "Orders"
}

Cart System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Cart",
  "title": "Cart"
}

Products System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Products",
  "title": "Products"
}

Cart Resource

{
  "properties": {
    "type": "postgress"
  },
  "relations": {},
  "icon": "GPU",
  "title": "Cart SQL database",
  "identifier": "cart-sql-sb"
}

Cart API

{
 "identifier": "CartAPI",
 "title": "Cart API",
 "blueprint": "API",
 "properties": {
   "type": "Open API"
 },
 "relations": {
   "provider": "CartService"
 },
 "icon": "Link"
}

Core Kafka Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Kafka Library",
  "identifier": "CoreKafkaLibrary"
}

Core Payment Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Payment Library",
  "identifier": "CorePaymentLibrary"
}

Cart Service JSON

{
 "identifier": "CartService",
 "title": "Cart Service",
 "blueprint": "Component",
 "properties": {
   "type": "service"
 },
 "relations": {
   "system": "Cart",
   "resources": [
     "cart-sql-sb"
   ],
   "consumesApi": [],
   "components": [
     "CorePaymentLibrary",
     "CoreKafkaLibrary"
   ]
 },
 "icon": "Cloud"
}

Products Service JSON

{
  "identifier": "ProductsService",
  "title": "Products Service",
  "blueprint": "Component",
  "properties": {
    "type": "service"
  },
  "relations": {
    "system": "Products",
    "consumesApi": [
      "CartAPI"
    ],
    "components": []
  }
}

Component Blueprint

{
 "identifier": "Component",
 "title": "Component",
 "icon": "Cloud",
 "schema": {
   "properties": {
     "type": {
       "enum": [
         "service",
         "library"
       ],
       "icon": "Docs",
       "type": "string",
       "enumColors": {
         "service": "blue",
         "library": "green"
       }
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "system": {
     "target": "System",
     "required": false,
     "many": false
   },
   "resources": {
     "target": "Resource",
     "required": false,
     "many": true
   },
   "consumesApi": {
     "target": "API",
     "required": false,
     "many": true
   },
   "components": {
     "target": "Component",
     "required": false,
     "many": true
   },
   "providesApi": {
     "target": "API",
     "required": false,
     "many": false
   }
 }
}

Resource Blueprint

{
 “identifier”: “Resource”,
 “title”: “Resource”,
 “icon”: “DevopsTool”,
 “schema”: {
   “properties”: {
     “type”: {
       “enum”: [
         “postgress”,
         “kafka-topic”,
         “rabbit-queue”,
         “s3-bucket”
       ],
       “icon”: “Docs”,
       “type”: “string”
     }
   },
   “required”: []
 },
 “mirrorProperties”: {},
 “formulaProperties”: {},
 “calculationProperties”: {},
 “relations”: {}
}

API Blueprint

{
 "identifier": "API",
 "title": "API",
 "icon": "Link",
 "schema": {
   "properties": {
     "type": {
       "type": "string",
       "enum": [
         "Open API",
         "grpc"
       ]
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "provider": {
     "target": "Component",
     "required": true,
     "many": false
   }
 }
}

Domain Blueprint

{
 "identifier": "Domain",
 "title": "Domain",
 "icon": "Server",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {}
}

System Blueprint

{
 "identifier": "System",
 "title": "System",
 "icon": "DevopsTool",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "domain": {
     "target": "Domain",
     "required": true,
     "many": false
   }
 }
}
{{tabel-1}}

Microservices SDLC

  • Scaffold a new microservice

  • Deploy (canary or blue-green)

  • Feature flagging

  • Revert

  • Lock deployments

  • Add Secret

  • Force merge pull request (skip tests on crises)

  • Add environment variable to service

  • Add IaC to the service

  • Upgrade package version

Development environments

  • Spin up a developer environment for 5 days

  • ETL mock data to environment

  • Invite developer to the environment

  • Extend TTL by 3 days

Cloud resources

  • Provision a cloud resource

  • Modify a cloud resource

  • Get permissions to access cloud resource

SRE actions

  • Update pod count

  • Update auto-scaling group

  • Execute incident response runbook automation

Data Engineering

  • Add / Remove / Update Column to table

  • Run Airflow DAG

  • Duplicate table

Backoffice

  • Change customer configuration

  • Update customer software version

  • Upgrade - Downgrade plan tier

  • Create - Delete customer

Machine learning actions

  • Train model

  • Pre-process dataset

  • Deploy

  • A/B testing traffic route

  • Revert

  • Spin up remote Jupyter notebook

{{tabel-2}}

Engineering tools

  • Observability

  • Tasks management

  • CI/CD

  • On-Call management

  • Troubleshooting tools

  • DevSecOps

  • Runbooks

Infrastructure

  • Cloud Resources

  • K8S

  • Containers & Serverless

  • IaC

  • Databases

  • Environments

  • Regions

Software and more

  • Microservices

  • Docker Images

  • Docs

  • APIs

  • 3rd parties

  • Runbooks

  • Cron jobs

You may also be interested in

Building the perfect internal developer platform

Building the perfect internal developer platform

Read more
How to use JQ commands to speed developer workflows

How to use JQ commands to speed developer workflows

Read more
Debugging K8s with K8sGPT in your internal developer portal

Debugging K8s with K8sGPT in your internal developer portal

Read more

Starting with Port is simple, fast and free.

Let’s start