TL;DR: Backstage provides a central developer portal where teams can organize, discover, and manage information about software, services, infrastructure, and ownership.

As infrastructure and software environments grow, knowing what exists, where it runs, and who owns it becomes increasingly difficult.

A Git repository can tell us where the code is. Kubernetes can tell us where an application is running. CI/CD can tell us how it is being deployed. Monitoring can tell us whether it is healthy.

But these tools usually answer only one part of the question.

Backstage brings this information together into a developer portal. Instead of replacing Git, Kubernetes, CI/CD, monitoring, or other infrastructure tools, Backstage provides a common place to discover and connect information from them.

One of the most important parts of Backstage is the Software Catalog.

How Backstage Indexes Anything

Before looking at each specific feature, it helps to understand the underlying Catalog & Search ingestion loop:

[ Data Source ]  >  [ Entity Provider / Processor ]  >  [ Backstage Catalog (DB) ]  >  [ Search Indexer (Lunr / Postgres / Elasticsearch) ]
(Git / K8s API)       (Discovers YAML & Resources)  (Entities: Component, Template, etc.)    (Full-text Search Index)

Providers & Processors: Background tasks constantly poll Git repositories or APIs to fetch definitions.

Software Catalog: Parses raw inputs into unified Entities (e.g., kind: Component, kind: Template, kind: Location) and saves them to the backend database.

Search Engine Plugin: Periodically reads entities from the Software Catalog, extracts searchable text (metadata, descriptions, labels), and builds an internal search index (using Lunr by default, or external systems like Elasticsearch/PostgreSQL).

1. How Backstage Indexes Kubernetes Resources

Backstage handles Kubernetes resources using a dynamic, runtime lookup model rather than storing live cluster states as permanent static entities in the database.

Step-by-Step Flow

1.1 Mapping Annotations:

When a service component is registered in the catalog, its catalog-info.yaml includes annotations linking it to Kubernetes resources:

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: dice-roller
  annotations:
    backstage.io/kubernetes-id: dice-roller # Label selector matching Pods/Deployments

1.2 The Kubernetes Backend Plugin:

Instead of constantly scraping every cluster into the database, the Kubernetes Backend Plugin queries the live Kubernetes API servers on-demand or during catalog refreshes.

1.3 Cluster Locators:

Backstage discovers target clusters through configured Cluster Locators (e.g., hardcoded in app-config.yaml, discovered dynamically via cloud APIs like AWS EKS/GCP GKE, or registered as ‘kind: Resource` catalog entities).

1.4 Indexing & Visibility:

  • 1.4.1 In the UI: When a user visits the dice-roller component page, Backstage calls the API server to pull Pod health, deployment status, and logs in real time.

  • 1.4.2 In Search: If Kubernetes clusters are registered as catalog resources (kind: Resource with type kubernetes-cluster), their metadata is indexed by the Search engine just like any other catalog entity.

2. How Backstage Indexes Software Templates (Scaffolder)

Software Templates allow engineers to spin up new microservices or infrastructure using predefined skeletons.

Step-by-Step Flow

2.1 Template Definition:

Templates are defined in Git repositories using kind: Template:

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: spring-boot-template
  title: Spring Boot Microservice
  description: Creates a basic Spring Boot service on Kubernetes.
spec:
  type: service
  parameters: [...]
  steps: [...]

2.2 Catalog Registration & Ingestion:

The catalog-info.yaml file (or a location reference in app-config.yaml) tells Backstage to watch the target Git path. The Catalog Entity Provider fetches the template YAML, validates its schema, and saves it into the catalog DB.

2.3 Search Indexing:

The Backstage Search Indexer queries all entities of kind: Template from the Catalog API. It extracts:

  • Metadata (Name, Title, Tags, Owner)

  • Description and parameters

2.4 Rendering in the Portal:

When developers open the Create / Scaffolder page or use the top search bar, Backstage queries the search index to filter templates by tags, type, or full-text keywords.

3. How Backstage Indexes TechDocs (Docs-as-Code)

TechDocs uses a Docs-as-Code model. Documentation is written in Markdown inside the application repository alongside code, compiled into static HTML using MkDocs, and indexed for full-text search.

Step-by-Step Flow

3.1 Annotation in Catalog:

A component points to its documentation source using the techdocs-ref annotation:

metadata:
  annotations:
    backstage.io/techdocs-ref: dir:. # Points to mkdocs.yml in repository root

3.2 The Prepare, Generate, and Publish Pipeline:

  • Prepare: TechDocs fetches the source Markdown and mkdocs.yml configuration from Git.

  • Generate: MkDocs (running in Docker or a CI runner) converts Markdown into static HTML files.

  • Publish: Built HTML files are uploaded to cloud storage (e.g., AWS S3, Google Cloud Storage, or local disk).

3.3 Full-Text Document Indexing:

  • The TechDocs Search Collator (a search plugin worker) runs periodically.

  • It downloads or reads the search_index.json file generated by MkDocs during the build step.

  • It transforms individual Markdown headings, paragraphs, and page titles into search documents inside the Backstage Search Engine.

3.4 Search Experience:

Searching for specific terms (e.g., “How to rotate database credentials”) returns direct links to exact sections and headers within TechDocs sites across all registered repositories.

Back to Certified Cloud Native Platform Engineering Associate Preparation