Pipelines & the DAG
the DAG · critical path highlighted
PipelineRun ci-run-x7k2 over time
TaskRuns the controller created
- ci-run-x7k2-cloneRunning
Running: Tasks Completed: 0 (Failed: 0, Cancelled 0), Incomplete: 5, Skipped: 0
apiVersion: tekton.dev/v1kind: Pipelinemetadata: {name: ci}spec:tasks:- name: clonetaskRef: {name: git-clone}- name: lintrunAfter: [clone]taskRef: {name: lint}- name: unit-testrunAfter: [clone]taskRef: {name: go-test}- name: buildrunAfter: [lint, unit-test]taskRef: {name: build}- name: deployrunAfter: [build]taskRef: {name: deploy}
Try: run it and watch lint and unit-test start together as soon as clone finishes. Shorten lint: nothing changes, because lint isn't on the critical path. Make lint 6s: the critical path moves to lint and the wall clock grows by 1s. Only the longest chain decides how long the PipelineRun takes.
edges: solid = runAfter, dashed = result reference
4 edgesresulting schedule
spec:workspaces: [{name: source}]tasks:- name: cloneworkspaces: [{name: output, workspace: source}]- name: unit-testrunAfter: [clone]workspaces: [{name: src, workspace: source}] # no edge!- name: buildrunAfter: [unit-test]params:- name: revisionvalue: $(tasks.clone.results.commit)workspaces: [{name: src, workspace: source}]- name: deployparams:- name: imagevalue: $(tasks.build.results.IMAGE_DIGEST)
Try: untick “build runAfter unit-test”. build still waits for clone because it reads clone's result. Now untick “unit-test runAfter clone”: unit-test starts at t=0 next to clone. Both use the same workspace, but that doesn't create an edge, so it races.
edit the graph
valid DAGhow the controller will run it
- wave 1: checkout
- wave 2: test-1, test-2, test-3 (in parallel)
- wave 3: report
- wave 4: publish
With equal durations each wave starts together. With real durations a task starts the moment its own parents finish.
kind: Pipelinespec:tasks:- name: checkout- name: test-1runAfter: [checkout]- name: test-2runAfter: [checkout]- name: test-3runAfter: [checkout]- name: reportrunAfter: [test-1, test-2, test-3]- name: publishrunAfter: [report]
Try: start from fan-out / fan-in, then click report followed by test-2. That makes test-2 run after report, which already runs after test-2. The validator finds the cycle and the Pipeline is rejected before anything runs.
DAG
timeline
# kubectl get pipelinerun ci-run-x7k2 -o yamlstatus:conditions:- type: Succeededstatus: "Unknown"reason: Runningmessage: "Tasks Completed: 0 (Failed: 0, Cancelled 0), Incomplete: 6, Skipped: 0"childReferences:- kind: TaskRunname: ci-run-x7k2-clonepipelineTaskName: clone
Try: lint fails at 3.5s while unit-test and docs are still running. They run to completion, but build and deploy are never created (“PipelineRun was stopping”). Switch to “when → false” on unit-test: only unit-test is skipped and build still runs, unless build consumes unit-test's result. Then it's “Results were missing” and deploy is skipped with “Parent Tasks were skipped”.
onError: continue on that pipeline task. To run cleanup no matter what, use finally. Both, along with when expressions and matrix, are covered in Control flow.The PipelineRun controller, one reconcile at a time
pipelinerunpipelinerun-taskrun-nameschild-referencespipeline-timeouts-overviewreconcile 1 of 6
t = 0.0s- 1 observenew PipelineRun; resolve Pipeline ci, build the DAG
- 2 decideready: clone
- 3 actcreate ci-run-x7k2-clone
- 4 statusRunning: Tasks Completed: 0 (Failed: 0, Cancelled 0), Incomplete: 5, Skipped: 0
apiVersion: tekton.dev/v1kind: PipelineRunmetadata: {name: ci-run-x7k2}spec:pipelineRef: {name: ci}timeouts:pipeline: "1h0m0s" # defaultstatus:conditions: [{type: Succeeded, status: "Unknown", reason: Running}]childReferences:- {kind: TaskRun, name: ci-run-x7k2-clone, pipelineTaskName: clone}
kind: TaskRunmetadata:name: ci-run-x7k2-clonelabels:tekton.dev/pipeline: citekton.dev/pipelineRun: ci-run-x7k2tekton.dev/pipelineTask: cloneownerReferences:- {kind: PipelineRun, name: ci-run-x7k2, controller: true}spec:taskRef: {name: clone}
timeline · cursor = this reconcile
Try: step through the reconciles. The controller never runs anything itself. Each time a child TaskRun changes it re-reads the state, creates the TaskRuns that just became ready, and rewrites the status. Then set timeouts.pipeline to 10s: running TaskRuns are cancelled, the rest are skipped, and the PipelineRun ends with PipelineRunTimeout.
fan-out to shards, fan-in to merge · hatched = TaskRun waiting on quota
apiVersion: v1kind: ResourceQuotametadata: {name: ci-quota, namespace: ci}spec:hard:requests.cpu: "4"---# each TaskRun Pod requests 1 CPU# Pipeline: shard-1..8 all runAfter [checkout]
Try: with no quota, all shards start the moment checkout finishes. Set the quota to 2 CPU: the controller still creates every TaskRun right away, but the API server only admits two Pods at a time and the rest wait. Tekton has no maxParallel setting; the namespace's quota (or the nodes' capacity) is the throttle.
kind: PipelineRunmetadata: {name: ci-run-x7k2}spec:pipelineRef:name: ciparams: [{name: revision, value: main}]
objects the controllers create
4 objects- Pipeline cireferenced, not owned
- PipelineRun ci-run-x7k2
- TaskRun ci-run-x7k2-clone
- TaskRun ci-run-x7k2-build
- TaskRun ci-run-x7k2-deploy
The usual form: a reusable Pipeline object, referenced by name (or through a resolver: git, bundles, hub). The resolved spec is copied into status.pipelineSpec when the run starts.
Try: flip between the forms and watch the ownership tree. However a Pipeline is written, the result is the same: one PipelineRun owning one TaskRun per pipeline task, named after the run and the task. The exception is the alpha nested form, which adds a child PipelineRun in between.
- takeawayA Pipeline is a graph. Edges come from runAfter and from result references, never from shared workspaces. Tasks start when their own parents finish, so wall-clock time is the critical path.
- takeawayOne failure stops the run from starting anything new. Running siblings finish, everything else is listed in skippedTasks with a reason, and the PipelineRun ends False/Failed. A false when expression skips just that task.
- takeawayThe PipelineRun controller only creates TaskRuns named <run>-<task> and summarizes them in childReferences and one condition. Parallelism is limited by the cluster (quota and nodes), not by Tekton.
Key concepts · 21
book chapter 4- Pipeline
A namespaced object (tekton.dev/v1) listing pipeline tasks, their ordering, and the params and workspaces they share.
It is a template, like a Task: it runs nothing until a PipelineRun points at it. Ordering is declared as data (edges), not as a script, which is what lets Tekton run independent work in parallel.
kind: Pipelinespec.tasks[]tkn pipeline start ci↑ see it in the visualization - PipelineRun
One execution of a Pipeline with concrete params and workspace bindings. It owns one TaskRun per pipeline task that runs.
It is both the request and the record. The controller reconciles it repeatedly, creating TaskRuns and summarizing their outcomes in a single Succeeded condition.
kind: PipelineRuntkn pipelinerun logs -fstatus.conditions↑ see it in the visualization - Pipeline task
An entry in spec.tasks: a name plus a taskRef or taskSpec, and optionally runAfter, params, workspaces, when, retries and timeout.
The pipeline task name, not the Task name, is the node in the graph. The same Task can appear many times under different names, and $(tasks.<name>.results.x) refers to that name.
spec.tasks[].namespec.tasks[].taskRefspec.tasks[].taskSpec↑ see it in the visualization - runAfter
An explicit edge: this task starts only after the listed tasks have finished successfully.
Use it for ordering with no data flow, such as tests after a clone that only shares a workspace. Without it, tasks with no edges between them start at the same time.
runAfter: [clone]↑ see it in the visualization - Implicit edges from results
Referencing $(tasks.X.results.Y) in a task's params or when expressions makes it depend on X, with no runAfter needed.
Data dependencies order themselves. Sharing a workspace, though, does NOT create an edge: two tasks that write and read the same volume race unless you add runAfter.
$(tasks.clone.results.commit)$(tasks.build.results.IMAGE_DIGEST)↑ see it in the visualization - DAG (directed acyclic graph)
The graph of pipeline tasks and their edges, built when the run starts.
Edges only point forward, so there is always an order in which every task's parents finish first. The controller walks this graph on every reconcile to find what is ready to run.
status.pipelineSpec↑ see it in the visualization - Fan-out
One task with several children that become ready together, e.g. lint, unit-test and e2e all after clone.
This is where Pipelines get their speed: independent children run as parallel Pods. The cost is cluster capacity, since every branch is a Pod with its own resource requests.
↑ see it in the visualization - Fan-in
One task that runs after several parents, e.g. build after lint and unit-test.
A fan-in task waits for its slowest parent, so a single slow branch holds up everything downstream of the join.
↑ see it in the visualization - Critical path
The chain of dependent tasks with the largest total duration. With unlimited capacity it equals the PipelineRun's wall-clock time.
Making a task faster only helps if it is on the critical path. Speeding up anything else just gives it more slack.
↑ see it in the visualization - Cycle detection
Validation that rejects a Pipeline whose edges form a loop, and also catches runAfter pointing at a task that doesn't exist.
A cycle has no valid order: every task in it waits on another. The admission webhook rejects it on kubectl apply. A Pipeline that arrives through a resolver fails the PipelineRun instead (PipelineValidationFailed), before any TaskRun is created.
cycle detectedreason: PipelineValidationFailed↑ see it in the visualization - Stopping after a failure
Once any task fails, the PipelineRun is stopping: running TaskRuns are allowed to finish, but no new ones are created.
A failure doesn't kill siblings that are already running, but branches that haven't started never will, even ones that don't depend on the failed task. Only finally tasks still run.
reason: PipelineRunStoppingPipelineRun was stopping↑ see it in the visualization - status.skippedTasks
The list of pipeline tasks that never got a TaskRun, each with a reason: PipelineRun was stopping, When Expressions evaluated to false, Results were missing, Parent Tasks were skipped, or a timeout.
A skipped task has no TaskRun to inspect, so this list is the only place that explains why it didn't run.
status.skippedTasks[].reason↑ see it in the visualization - status.childReferences
The list of child objects (TaskRuns, CustomRuns, child PipelineRuns) the PipelineRun created, with each one's pipelineTaskName.
It is the index from graph node to real object. The full child statuses live on the children themselves, which keeps the PipelineRun object small.
status.childReferences[].nametekton.dev/pipelineTask label↑ see it in the visualization - <pipelinerun>-<task> TaskRuns
Child TaskRuns are named after the run and the pipeline task, labelled tekton.dev/pipelineRun and tekton.dev/pipelineTask, and owned by the PipelineRun.
The owner reference means deleting the PipelineRun garbage-collects its TaskRuns and Pods. The labels let you select every TaskRun of a run with kubectl get tr -l tekton.dev/pipelineRun=….
ci-run-x7k2-build-l tekton.dev/pipelineRun=ci-run-x7k2↑ see it in the visualization - PipelineRun reasons
The Succeeded condition reasons: Running, PipelineRunStopping, Succeeded, Completed (succeeded with some tasks skipped), Failed, PipelineRunTimeout, Cancelled, PipelineValidationFailed…
As with TaskRuns, status True/False/Unknown says whether it finished, and the reason plus a message like "Tasks Completed: 4 (Failed: 1, Cancelled 0), Skipped: 2" says how.
reason: Completedreason: PipelineRunTimeout↑ see it in the visualization - timeouts.pipeline / tasks / finally
Deadlines on the whole PipelineRun, on its DAG tasks, and on its finally tasks. timeouts.pipeline defaults to 60 minutes.
When timeouts.pipeline expires, running TaskRuns are cancelled, unstarted tasks are skipped, and the run ends with PipelineRunTimeout. The Control flow chapter covers splitting the budget between tasks and finally.
spec.timeouts.pipelinespec.timeouts.tasksspec.timeouts.finally↑ see it in the visualization - No maxParallel: the cluster throttles
Tekton creates every ready TaskRun immediately and has no per-pipeline concurrency limit. Throughput is bounded by namespace ResourceQuota and node capacity.
When a quota refuses a Pod, the TaskRun waits as Unknown/ExceededResourceQuota and the controller retries. A wide fan-out under a tight quota runs in waves. That is safe but slower, and the waiting counts against timeouts.
ResourceQuota requests.cpureason: ExceededResourceQuotaLimitRange↑ see it in the visualization - pipelineRef
A PipelineRun pointing at a Pipeline by name or through a resolver (git, bundles, hub, cluster).
It lets many runs share one reviewed definition. The resolved spec is copied into status.pipelineSpec, so later edits to the Pipeline don't change a run in progress.
spec.pipelineRef.namespec.pipelineRef.resolver↑ see it in the visualization - pipelineSpec (inline)
A PipelineRun that carries its whole Pipeline definition.
Common for generated runs (Triggers templates, Pipelines-as-Code). The trade-off is the same as taskSpec: nothing to install, nothing to share.
spec.pipelineSpec.tasks↑ see it in the visualization - Pipelines in Pipelines
An alpha feature (TEP-0056) where a pipeline task embeds a pipelineSpec, so the controller creates a child PipelineRun.
It lets you compose whole sub-pipelines (e.g. a security stage) as one node in a parent graph. It is alpha and only partly implemented, so check your release before relying on it.
tasks[].pipelineSpecenable-api-fields: alpha↑ see it in the visualization - displayName
A free-text label on a Pipeline or a pipeline task, shown by tkn and the Dashboard.
Task names must be DNS-label-safe and stay stable because results refer to them. displayName lets humans read "Build the binary" without renaming the node.
spec.displayNamespec.tasks[].displayName↑ see it in the visualization