Definitions and versions

Description and operation

General

A pipeline is a name for an owner and a list of numbered versions. Each version holds one definition and never changes. A run records the version that it ran, so a later change does not alter a run that exists. The format of a definition is in Pipeline definition.

A save checks the whole definition and reports every problem at once, each with its job and step. A save also checks facts about the owner. Each secret that the definition names exists. The pipeline of an after trigger exists. Each volume or snapshot that a job names exists, or an earlier job saves to it.

A definition larger than 64 KiB is refused with 422 before any other check. A definition that saves is a definition that can run.

To do these tasks with an AI agent, see Agentic workflows – General.

Canonical form

The platform stores each definition in a canonical form. It keeps the order of jobs, inputs and each matrix as you wrote them, sorts every other map and leaves out every default value. Two spellings of one definition have the same definitionSha256, so a save of an equal definition makes no new version.

Delete rules

You cannot delete the newest version. You cannot delete a version, or a pipeline, while a queued or running run uses it. A finished run of a deleted version stays readable, without its definition. Run records are kept for 90 days after the run ends.

Tasks

Save a new version

Reason for the task: Change what a pipeline does, and keep the earlier definition.

Job set-up

• Access: you. For a team, an admin.
• Conditions: the pipeline nightly-report exists, and nightly-report.json holds its changed definition. The name in the file is nightly-report.
• References: PUT /v1/pipelines/{owner}/{name}, SDK pipelines.update. In the CLI, --file - reads the definition from standard input, and --owner defaults to your handle.

Procedure

1. Send the changed definition as the next version.

   PythonCopy

   import jsonfrom kloudlite import Kloudlitekl = Kloudlite()with open("nightly-report.json") as f:    doc = kl.pipelines.update(owner="acme", name="nightly-report", definition=json.load(f))print(doc["version"], doc["latestVersion"])

   TypeScriptCopy

   import { readFileSync } from "node:fs";import { Kloudlite } from "./index.ts";const kl = new Kloudlite();const definition = JSON.parse(readFileSync("nightly-report.json", "utf8"));const doc = await kl.pipelines.update({ owner: "acme", name: "nightly-report", definition });console.log(doc.version, doc.latestVersion);

   GoCopy

   kl := kloudlite.New(kloudlite.Options{})raw, err := os.ReadFile("nightly-report.json")if err != nil {    log.Fatal(err)}var d kloudlite.PipelineDefinitionif err := json.Unmarshal(raw, &d); err != nil {    log.Fatal(err)}doc, err := kl.Pipelines.Update("acme", "nightly-report", d)if err != nil {    log.Fatal(err)}fmt.Println(doc.Version, doc.LatestVersion)

   RubyCopy

   require "json"require_relative "kloudlite"kl = Kloudlite::Client.newdoc = kl.pipelines.update(owner: "acme", name: "nightly-report", definition: JSON.parse(File.read("nightly-report.json")))puts doc.values_at("version", "latestVersion").join(" ")

   JavaCopy

   var kl = new Kloudlite();System.out.println(kl.request(new Kloudlite.RawRequest().method("PUT").path("/v1/pipelines/acme/nightly-report")    .body(Files.readString(Path.of("nightly-report.json")))));

   CLICopy

   kloudlite pipelines update acme/nightly-report --file nightly-report.json

Close-out

1. Make sure that version is one more than before. An equal definition returns the same version.

Show the versions of a pipeline

Reason for the task: Find which definition a version holds, for example the version that a run used.

Job set-up

• Access: you. For a team, an active member.
• References: GET /v1/pipelines/{owner}/{name}/versions, GET /v1/pipelines/{owner}/{name}, SDK pipelines.versions, pipelines.get.

Procedure

1. Show the list of versions, oldest first.

   PythonCopy

   print(kl.pipelines.versions(owner="acme", name="nightly-report"))

   TypeScriptCopy

   console.log(await kl.pipelines.versions({ owner: "acme", name: "nightly-report" }));

   GoCopy

   versions, err := kl.Pipelines.Versions("acme", "nightly-report")if err != nil {    log.Fatal(err)}fmt.Printf("%+v\n", versions)

   RubyCopy

   p kl.pipelines.versions(owner: "acme", name: "nightly-report")

   JavaCopy

   System.out.println(kl.pipelines().versions("acme", "nightly-report"));

   CLICopy

   kloudlite pipelines versions acme/nightly-report

2. Show the definition of version 1. Leave out the version to show the newest.

   PythonCopy

   print(kl.pipelines.get(owner="acme", name="nightly-report", version=1)["definition"])

   TypeScriptCopy

   console.log((await kl.pipelines.get({ owner: "acme", name: "nightly-report", version: 1 })).definition);

   GoCopy

   doc, err := kl.Pipelines.Get(kloudlite.GetPipeline{Owner: "acme", Name: "nightly-report", Version: 1})if err != nil {    log.Fatal(err)}fmt.Printf("%+v\n", doc.Definition)

   RubyCopy

   p kl.pipelines.get(owner: "acme", name: "nightly-report", version: 1)["definition"]

   JavaCopy

   System.out.println(kl.pipelines().get("acme", "nightly-report", 1));

   CLICopy

   kloudlite pipelines get acme/nightly-report --version 1

Close-out

1. Examine definitionSha256 of the version and of a run. Equal values mean that the run used that definition.

Go back to an earlier version

Reason for the task: Undo a change that broke a pipeline, and keep a record of the change.

Job set-up

• Access: you. For a team, an admin.
• Conditions: the earlier version exists. See Show the versions of a pipeline.
• References: SDK pipelines.get, pipelines.update.

Procedure

1. Read the definition of the earlier version, then send it as the next version.

   PythonCopy

   old = kl.pipelines.get(owner="acme", name="nightly-report", version=1)["definition"]doc = kl.pipelines.update(owner="acme", name="nightly-report", definition=old)print(doc["version"])

   TypeScriptCopy

   const old = await kl.pipelines.get({ owner: "acme", name: "nightly-report", version: 1 });const doc = await kl.pipelines.update({ owner: "acme", name: "nightly-report", definition: old.definition });console.log(doc.version);

   GoCopy

   old, err := kl.Pipelines.Get(kloudlite.GetPipeline{Owner: "acme", Name: "nightly-report", Version: 1})if err != nil {    log.Fatal(err)}doc, err := kl.Pipelines.Update("acme", "nightly-report", old.Definition)if err != nil {    log.Fatal(err)}fmt.Println(doc.Version)

   RubyCopy

   old = kl.pipelines.get(owner: "acme", name: "nightly-report", version: 1)["definition"]puts kl.pipelines.update(owner: "acme", name: "nightly-report", definition: old)["version"]

   JavaCopy

   // The SDK returns raw JSON. Take "definition" from it with your JSON library, here Jackson.String doc = kl.pipelines().get("acme", "nightly-report", 1);String old = new ObjectMapper().readTree(doc).get("definition").toString();System.out.println(kl.request(new Kloudlite.RawRequest().method("PUT").path("/v1/pipelines/acme/nightly-report").body(old)));

   CLICopy

   kloudlite pipelines get acme/nightly-report --version 1 --json | jq .definition > old.jsonkloudlite pipelines update acme/nightly-report --file old.json

Close-out

1. Make sure that the new version is the newest, and that its definitionSha256 equals that of version 1.

Delete an old version

Reason for the task: Remove a version that is no longer needed. Version numbers are never reused, so this does not free a number.

Job set-up

• Access: you. For a team, an admin.
• Conditions: the version is not the newest, and no queued or running run uses it.
• References: DELETE /v1/pipelines/{owner}/{name}/versions/{n}. The SDKs have no method for this route, so the procedure sends it as a raw request.

Procedure

1. Send the delete for version 1.

   PythonCopy

   kl.request("DELETE", "/v1/pipelines/acme/nightly-report/versions/1")

   TypeScriptCopy

   await kl.request("DELETE", "/v1/pipelines/acme/nightly-report/versions/1");

   GoCopy

   if err := kl.Request("DELETE", "/v1/pipelines/acme/nightly-report/versions/1", kloudlite.RequestOptions{}); err != nil {    log.Fatal(err)}

   RubyCopy

   kl.request("DELETE", "/v1/pipelines/acme/nightly-report/versions/1")

   JavaCopy

   kl.request("DELETE", "/v1/pipelines/acme/nightly-report/versions/1");

Close-out

1. Show the versions of the pipeline. Make sure that version 1 is not listed.

Delete a pipeline

Reason for the task: Remove a pipeline that is no longer used, with all its versions and triggers.

Job set-up

• Access: you. For a team, an admin.
• Conditions: no run of the pipeline is queued or running. See Cancel a run.
• References: DELETE /v1/pipelines/{owner}/{name}, SDK pipelines.delete.

Procedure

1. Delete the pipeline.

   PythonCopy

   kl.pipelines.delete(owner="acme", name="nightly-report")

   TypeScriptCopy

   await kl.pipelines.delete({ owner: "acme", name: "nightly-report" });

   GoCopy

   if err := kl.Pipelines.Delete("acme", "nightly-report"); err != nil {    log.Fatal(err)}

   RubyCopy

   kl.pipelines.delete(owner: "acme", name: "nightly-report")

   JavaCopy

   kl.pipelines().delete("acme", "nightly-report");

   CLICopy

   kloudlite pipelines delete acme/nightly-report

Close-out

1. Show the pipelines of the owner. Make sure that nightly-report is not listed.
2. Read a finished run of the pipeline. It stays readable for 90 days after it ended.

Troubleshooting