epamedp/tekton-pipeline-queue repository overview

⁠Tekton Pipeline Queue

A Kubernetes operator that queues Tekton⁠ PipelineRuns. It serializes or limits concurrent PipelineRuns per lane — an arbitrary combination of label values such as codebase, branch, pull request, or CD pipeline + stage — instead of letting every run start at once and compete for cluster resources.

Part of the KubeRocketCI⁠ platform, usable with any Tekton installation.

⁠How it works

The operator builds on Tekton's native pause primitive, spec.status: PipelineRunPending:

1. Producers (Tekton TriggerTemplates, operators, UI) create PipelineRuns with spec.status: PipelineRunPending and identifying labels. Tekton accepts the run but starts nothing — no TaskRuns, no pods.
2. A PipelineRunQueue resource selects the runs it governs (label selector), derives a lane for each run from the values of the queueKey labels, and defines per-lane concurrency and a strategy.
3. The controller watches PipelineRuns and, on every change, re-derives each lane from the live cluster state: pending runs are admitted FIFO (by creation time) by clearing spec.status while the number of running runs is below concurrency; superseded runs are gracefully cancelled (CancelledRunFinally) depending on the strategy.

There is no stored queue: the live PipelineRun set is the single source of truth, so controller restarts, out-of-band deletions, and manual cancellations converge automatically. status.lanes is a read-only projection for observability (portal, kubectl).

⁠Example

Review pipelines, one lane per pull request: each new push to a PR supersedes the previous commit's run — queued predecessors are cancelled and a still-running one is gracefully stopped — so CI spends resources only on the newest commit, the one that can actually merge:

apiVersion: edp.epam.com/v1alpha1
kind: PipelineRunQueue
metadata:
  name: review-queue
spec:
  selector:
    matchLabels:
      app.edp.epam.com/pipelinetype: review
  queueKey:
    - app.edp.epam.com/codebase
    - app.edp.epam.com/git-change-number
  concurrency: 1
  strategy: CancelInProgress

$ kubectl get pipelinerunqueue review-queue
NAME           QUEUED   RUNNING   READY   AGE
review-queue   2        3         True    2d

Different pull requests run in parallel (each is its own lane); pushes to the same pull request replace each other.

⁠Strategies

Typical lane keys: codebase + branch for build pipelines, codebase + change number for review pipelines (with ReplaceQueued or CancelInProgress), cdpipeline + cdstage for deployments.

Full recipes — global caps, per-codebase serialization, latest-commit-wins reviews, freshest-payload deploys, lane per Pipeline name, and how to keep queues disjoint: docs/use-cases.md⁠. Ready-to-apply manifests: config/samples⁠.

⁠Installation

Requires an existing Tekton Pipelines installation (the operator watches tekton.dev/v1 PipelineRuns and never installs or owns that CRD).

Helm chart (includes CRDs):

helm install tekton-pipeline-queue ./deploy-templates -n krci

Or with kustomize during development:

make install          # CRDs
make deploy IMG=docker.io/epamedp/tekton-pipeline-queue:<tag>

⁠Annotations

The operator stamps every PipelineRun it acts on, in the same patch that changes spec.status: app.edp.epam.com/queue, .../queue-lane, and .../queue-admitted-at on admission; .../queue, .../queue-lane, and .../queue-cancel-reason: superseded on cancellation. A run without these annotations was never touched by the queue. How to read them and diagnose queue behavior: docs/debugging.md⁠.

⁠Metrics

Prometheus metrics exposed by the controller:

⁠Development

make build            # compile the manager
make test             # unit + envtest integration tests
make start-kind && make e2e   # Chainsaw e2e on Kind (suite self-provisions Tekton + chart)
make lint             # golangci-lint
make manifests        # regenerate CRDs (config/crd/bases + deploy-templates/crds) and docs/api.md
make helm-docs        # regenerate deploy-templates/README.md

Documentation index: docs/README.md⁠ — use cases, debugging, API reference. Chart values: deploy-templates/README.md⁠.

⁠License

Apache-2.0 — see LICENSE.txt⁠.