Secrets, registries and environments

Description and operation

General

A job gets credentials and services only from its owner and its definition. It reads the secrets of the owner of the pipeline. It logs in to the container registries of that owner when its definition asks. It reaches an environment through a session that the platform makes for the run.

To do these tasks with an AI agent, see Agentic workflows – General.

Secrets

A job reads a secret only by reference. The pipeline reads the secrets of its owner: your own secrets for a personal pipeline, the secrets of the team for a team pipeline. A pipeline never reads the personal secrets that a member keeps in a team. The secret stores are described in Secrets.

The reference must be the whole value. Each start of a run checks the secrets again, and a secret that was deleted after the save refuses the run with 422. A save checks that each secret exists, and fails with secret NAME does not exist when one does not. A secret that the newest version names, or that a queued or running run took, cannot be deleted.

Masking

The log of a job shows *** in place of each value that the job holds: its secrets, HF_TOKEN and the credentials that the platform gives it. A value shorter than 4 bytes is not masked, because a short value matches too much ordinary text. Do not use short secret values. The Git credential KL_GIT_TOKEN of a job with permissions.git: read is masked in the same way.

Hugging Face token

When the owner of the pipeline has the secret HF_TOKEN, every job gets it as the variable HF_TOKEN. Hugging Face tools read this variable. A job that sets HF_TOKEN in its own env or secrets uses its own value. See Connectors.

Registry push

"permissions": {"registry": "push"} on a job logs it in to each container registry of the owner, before the first step. A pull from those registries needs no grant.

The login uses docker, so the image of the job must contain the docker command. A job without it fails with registry: push needs docker in the job's image.

Environments

A job can run against the services of an environment. The platform connects the jobs of the run to the environment through a session that it makes for the run.

Set environment on the pipeline, on a job, or in the start request. The value of a job wins over the start request, and the start request wins over the pipeline. A copy counts against the quota of the owner while it exists. If the run does not delete a copy, the platform deletes it after one hour of non-use.

Tasks

Give a secret to a job

Reason for the task: Let a job use a credential, such as a database URL, without writing it in the definition.

Job set-up

• Access: you. For a team, an admin.
• Conditions: the owner has the secrets DB_URL and API_TOKEN. See Set or replace a secret.
• References: Secrets, Pipeline definition – Jobs.

Procedure

1. Add the secrets to the definition. DB_URL reaches every step, and TOKEN reaches one step.

   jsonCopy

   {  "name": "load-orders",  "jobs": {    "load": {      "packages": ["postgresql", "curl"],      "secrets": ["DB_URL"],      "steps": [        { "name": "check", "run": "psql \"$DB_URL\" -c 'select 1'" },        {          "name": "fetch",          "run": "curl -fsS -H \"Authorization: Bearer $TOKEN\" -o orders.json https://api.example.com/orders",          "env": { "TOKEN": "${{ secrets.API_TOKEN }}" }        }      ]    }  }}

2. Create the pipeline, or send the definition as a new version. See Definitions and versions.

3. Start a run and read the log of load. See Start a run and follow it.

Close-out

1. Make sure that the log shows *** where a secret value is printed.

Push an image to a container registry

Reason for the task: Publish an image from a job, for example to copy a tested image to a release tag.

Job set-up

• Access: you. For a team, an admin.
• Conditions: the owner has a container registry connector. See Connectors.
• References: Registry push.

Procedure

1. Give the job permissions.registry and an image that contains docker.

   jsonCopy

   {  "name": "release-image",  "jobs": {    "copy": {      "image": "docker:27-cli",      "permissions": { "registry": "push" },      "steps": [        { "name": "copy", "run": "docker buildx imagetools create -t \"$KL_REGISTRY/alpine:$KL_RUN_NUMBER\" alpine:3.20" }      ]    }  }}

2. Create the pipeline from the file, and start a run. See Start a run and follow it.

Close-out

1. Make sure that the job is success.
2. Make sure that the registry lists the tag with the number of the run.

Run jobs against an environment

Reason for the task: Test or load data against real services, in a copy that the run deletes at its end.

Job set-up

• Access: you. For a team, an admin to change the definition, and an active member to start the run.
• Conditions: the environment staging exists for the owner. See Environments.
• References: Environments.

Procedure

1. Set environment on the job. A job reaches each service by its name.

   jsonCopy

   {  "name": "smoke",  "jobs": {    "smoke": {      "environment": { "clone": "staging" },      "packages": ["curl"],      "steps": [{ "name": "health", "run": "curl -fsS http://api:8080/health" }]    }  }}

2. Create the pipeline from the file. See Create a pipeline.

3. Start a run. To use another environment for one run, give it in the start request.

   PythonCopy

   run = kl.pipelines.run(owner="acme", pipeline="smoke")print(run.wait())

   TypeScriptCopy

   const run = await kl.pipelines.run({ owner: "acme", pipeline: "smoke" });console.log(await run.wait());

   GoCopy

   run, err := kl.Pipelines.Run(kloudlite.RunPipeline{Owner: "acme", Pipeline: "smoke"})if err != nil {    log.Fatal(err)}state, err := run.Wait(kloudlite.WaitRun{})if err != nil {    log.Fatal(err)}fmt.Println(state)

   RubyCopy

   run = kl.pipelines.run(owner: "acme", pipeline: "smoke")puts run.wait

   JavaCopy

   var run = kl.pipelines().run(new Kloudlite.RunPipeline().owner("acme").pipeline("smoke"));System.out.println(run.waitFor());

   CLICopy

   RUN=$(kloudlite pipelines run acme/smoke --json | jq -r .id)kloudlite pipelines wait-run acme smoke "$RUN"

Close-out

1. Make sure that the run is success.
2. Show the environments of the owner. Make sure that the copy of staging is gone.

Troubleshooting