Operating Tekton

The Tekton command-line client: start, list, describe, follow logs of, cancel and delete Tekton objects.

It is a convenience layer over the Kubernetes API. Everything it does is reading or writing the same CRDs kubectl sees, plus following Pod container logs in DAG order.

Packages Tasks and Pipelines into an OCI image (a Tekton Bundle) and pushes it to a registry.

Bundles version and distribute pipeline definitions like container images; the bundles resolver fetches them by digest at run time.

A web UI that lists PipelineRuns and TaskRuns and shows their tasks, steps and logs.

It reads the same live objects as tkn. When an object is pruned it disappears from the Dashboard too, unless logs and records are served from an external store like Results.

Deleting finished PipelineRuns and TaskRuns (and with them their Pods) so they don't accumulate in etcd.

Every run leaves several status-heavy objects behind. Without cleanup, etcd grows until it hits its quota and the API server starts refusing writes, which breaks far more than CI.

The operator pruner's two retention rules: keep the newest N runs, or keep runs younger than M minutes. They are mutually exclusive.

The operator implements them as a CronJob, so objects still pile up between schedule firings. Keep N bounds the count; keep-since bounds age but lets busy days grow large.

A newer, event-driven pruner controller that deletes runs based on TTL after completion and per-status history limits.

Reacting to completions instead of a schedule removes the CronJob sawtooth and allows global, namespace or per-pipeline policies.

A service that stores finished PipelineRuns and TaskRuns as Results and Records in a database, with an API to query them.

It separates history from live cluster state: once a run is archived it can be deleted from etcd and still be queried, audited and shown in UIs.

The Results controller that watches runs in the cluster and sends each one to the Results API over gRPC, plus its logs.

It only deletes an object after it was stored and a grace period passed, so an outage of the API makes objects linger instead of losing history.

Where Results keeps step logs after the Pods are gone: object storage like S3 or GCS, or a PVC.

Pod logs vanish with the Pod. Archiving them is what keeps a failed build debuggable a week later.

A Kubernetes operator that installs, configures and upgrades Tekton components from a single TektonConfig resource.

It reconciles continuously: delete a controller by hand and it comes back; bump the operator and every component rolls forward together.

spec.profile picks which components the operator installs: lite (Pipelines), basic (adds Triggers and Chains) or all (adds the Dashboard).

One field decides the footprint. Changing it adds or removes component CRs, and the operator handles the rest.

A Tekton project that runs PipelineRuns kept in the repository's .tekton/ directory in response to Git provider events.

Pipeline definitions are versioned and reviewed with the code they build, and a PR can change its own CI.

The folder Pipelines-as-Code scans on each event for PipelineRun YAML files to match.

Each file is one candidate PipelineRun; annotations on it decide whether it runs for this event.

Annotations that match a PipelineRun to an event type and target branch (globs allowed); on-cel-expression allows richer matching.

Matching happens before anything is created, so unmatched events cost nothing and you can keep separate runs for PRs, merges and tags.

PR comments that drive CI: /retest, /test <name>, /cancel, and /ok-to-test to approve a run for an untrusted author.

Running a fork's PR means running a stranger's code with your credentials, so it waits for a maintainer; comments from users outside the ACL are ignored.

Pipelines-as-Code reports each PipelineRun back to the commit as a check run (or commit status), with a summary of its tasks.

Developers see CI where they already are, and branch protection rules can require the checks to pass before merge.

Prometheus metrics exposed by the controller, such as pipelinerun_duration_seconds (histogram) and running_pipelineruns (gauge), all prefixed tekton_pipelines_controller_.

Duration percentiles and running counts are the first signal of a slow registry, an overloaded cluster or a stuck controller. config-observability controls levels and histogram vs last-value.

Kubernetes Events the controller emits on runs (Started, Running, Succeeded, Failed) and its structured JSON logs keyed by namespace/name.

Events explain one run's story; logs explain what the controller decided and why. Both are short-lived, so ship them somewhere if you need them later.

Running several controller replicas that split the key space into buckets, each guarded by a Lease that one replica holds at a time.

With one bucket only one replica works and the rest wait. More buckets spread the load, and a crash stalls only the dead replica's share until its leases expire.

Performance knobs: kube-api-qps and kube-api-burst limit the controller's client-side API rate; threads-per-controller sets workqueue workers.

Defaults are conservative. Under a spike of runs the rate limiter, not CPU, is usually the bottleneck, so raise QPS and burst before adding threads.