Adding a Provider

This guide walks through adding a deployment target to Pilum. You can add one with only a recipe YAML file, or add Go handlers to Pilum for commands that need logic.

Three Approaches

Approach	When to use	Example
Explicit commands	Static commands that only need the built-in variables	command: ["kubectl", "apply", "-f", "k8s/"]
Generic handlers	Reuse Pilum’s build, Docker build and push steps	Steps named build binary, build docker image, publish to registry
Provider handlers	Commands built from service config in Go	deploy to cloud run for gcp

The first approach needs no Go code. The other two are combined in one recipe when you add a provider to Pilum itself.

How Recipes Are Found

Recipes are embedded in the pilum binary. pilum recipes lists them and pilum recipes <name> shows one recipe’s fields and steps.

• Each recipe is indexed by <provider>-<service>, taken from its provider: and service: keys. A service selects it with type: <provider>-<service>. Give the recipe the same name so the two don’t drift apart.
• If a service’s type matches no recipe, Pilum uses the first recipe for the service’s provider, in alphabetical file order. Set type whenever a provider has more than one recipe.
• provider is filled in from type only for the built-in types. For a new type, set provider: explicitly in pilum.yaml, or the service fails with “missing required field: provider”.

There are two ways to use a new recipe:

1. --recipe-path <dir> loads every .yaml/.yml file in that directory instead of the embedded recipes. It does not merge with them. Copy any built-in recipes you still need (from the Pilum repository’s recipes/ directory) into the same directory.
2. Add the file to recipes/ in the Pilum source and rebuild. It is then embedded, and shows up in pilum recipes and pilum check.

Caution

pilum check and pilum recipes only read the embedded recipes; neither accepts --recipe-path. A service that uses a recipe from --recipe-path gets “No recipe found” from check as a warning and still passes. required_fields are only enforced by pilum check, never by deploy, so in a --recipe-path recipe they serve as documentation.

Approach 1: Explicit Commands Only

The simplest approach: no Go code required.

Create the Recipe

File: my-recipes/k8s-recipe.yaml

name: k8s-deployment
description: Apply Kubernetes manifests with kubectl
provider: k8s
service: deployment

required_fields:
  - name: project
    description: kubectl context to deploy to
    type: string

steps:
  - name: build
    command: go build -o ./dist/app .
    execution_mode: service_dir
    timeout: 300
    tags:
      - build

  - name: apply manifests
    command: ["kubectl", "--context", "${project}", "apply", "-f", "k8s/"]
    execution_mode: service_dir
    timeout: 120
    tags:
      - deploy

  - name: wait for rollout
    command: ["kubectl", "--context", "${project}", "rollout", "status", "deployment/${name}", "--timeout=300s"]
    execution_mode: service_dir
    timeout: 330
    tags:
      - deploy

Create a Service

File: services/my-app/pilum.yaml

name: my-app
type: k8s-deployment
provider: k8s          # required: Pilum can't infer it for a custom type
project: prod-cluster  # used as ${project}

Test

pilum dry-run --recipe-path ./my-recipes --tag=test   # Preview
pilum deploy  --recipe-path ./my-recipes --tag=v1.0.0 # Deploy

Remember that --recipe-path replaces the embedded recipes, so in this run only k8s-deployment exists.

Limits of explicit commands

• Only the built-in variables are substituted. A custom key such as cluster: in pilum.yaml cannot be referenced as ${cluster}; map it onto project or region, or write a handler.
• A required field can only be checked if it is a top-level string key. Dotted names such as k8s.cluster are only understood for built-in fields.

---

Approach 2: With Handlers

For targets that need logic, add an ingredient (a Go command generator) and register handlers. This example adds GKE: it reuses the generic build and push handlers, and adds two GCP-specific steps.

1. Create the Recipe

File: recipes/gcp-gke-recipe.yaml

name: gcp-gke
description: Build, push and roll out a GKE Deployment
provider: gcp
service: gke

required_fields:
  - name: project
    description: GCP project ID
    type: string
  - name: region
    description: Cluster region (also the Artifact Registry region)
    type: string
  - name: registry_name
    description: Artifact Registry repository name
    type: string
  - name: template
    description: Dockerfile template name under _templates/
    type: string

optional_fields:
  - name: gke.cluster
    description: Cluster name (defaults to the project ID)
    type: string

steps:
  # Generic handlers: same step names as the Cloud Run recipe
  - name: build binary
    execution_mode: service_dir
    timeout: 300
    tags: [build]

  - name: build docker image
    execution_mode: root
    timeout: 300
    tags: [build]

  - name: publish to registry
    execution_mode: root
    timeout: 120
    tags: [push]

  # Provider handlers, registered below for provider "gcp"
  - name: get gke credentials
    execution_mode: root
    timeout: 60
    tags: [deploy]

  - name: deploy to gke
    execution_mode: root
    timeout: 300
    tags: [deploy]

Because the image is built by the generic handlers and provider is gcp, it is named and pushed exactly as for Cloud Run: <region>-docker.pkg.dev/<project>/<registry_name>/<name>:<tag>.

2. Create the Ingredient

File: ingredients/gcp/gke.go

package gcp

import (
  "fmt"

  "github.com/sid-technologies/pilum/lib/configutil"
  serviceinfo "github.com/sid-technologies/pilum/lib/service_info"
)

func gkeCluster(svc serviceinfo.ServiceInfo) string {
  cluster := configutil.GetNestedString(svc.Config, "gke", "cluster")
  if cluster == "" {
    return svc.Project
  }
  return cluster
}

// GenerateGKECredentialsCommand points kubectl at the cluster.
func GenerateGKECredentialsCommand(svc serviceinfo.ServiceInfo) []string {
  return []string{
    "gcloud", "container", "clusters", "get-credentials", gkeCluster(svc),
    "--region", svc.Region,
    "--project", svc.Project,
  }
}

// GenerateGKEDeployCommand swaps the image and waits for the rollout.
// It returns a string, so it runs through `sh -c` and && works.
func GenerateGKEDeployCommand(svc serviceinfo.ServiceInfo, imageName string) string {
  return fmt.Sprintf(
    "kubectl set image deployment/%[1]s %[1]s=%[2]s && kubectl rollout status deployment/%[1]s --timeout=280s",
    svc.Name, imageName,
  )
}

A handler can return []string (run directly, no shell) or string (run with sh -c). Shell operators such as |, && or > only work in a string: as an element of a []string they are passed to the program as a literal argument. Returning nil skips the step and counts it as a success, which is how optional steps such as execute job turn themselves off.

3. Register Handlers

File: lib/registry/commands.go

func registerGCPGKEHandlers(reg *CommandRegistry) {
  reg.Register("get gke credentials", "gcp", func(ctx StepContext) any {
    return gcp.GenerateGKECredentialsCommand(ctx.Service)
  })

  reg.Register("deploy to gke", "gcp", func(ctx StepContext) any {
    return gcp.GenerateGKEDeployCommand(ctx.Service, ctx.ImageName)
  })
}

func RegisterDefaultHandlers(reg *CommandRegistry) {
  // ... existing handlers
  registerGCPGKEHandlers(reg)
}

StepContext carries Service (the parsed pilum.yaml, with the raw map in Service.Config), ImageName (the full image reference with tag), Tag (the raw --tag value, possibly empty), Registry and TemplatePath.

4. Finish the wiring

• Add "gcp-gke" to the type → provider switch in NewServiceInfo (lib/service_info/service_info.go) so services don’t need provider:.
• Add "gcp-gke" to the type enum in schemas/pilum.yaml.schema.json, plus any new config block (gke).
• To make pilum check enforce a dotted required field such as gke.cluster, add a getter for it in getServiceField (lib/recipe/recipe.go). Unknown dotted names always read as empty.
• For pilum status, logs or rollback support, add cases in lib/query/dispatch.go and lib/rollback/rollback.go.

---

Handler Matching

A step gets its command in this order:

1. If the step has a command, that command is used (after variable substitution). No handler is consulted.
2. Otherwise Pilum looks up a handler by the exact step name (case-insensitive) and the service’s provider:
   1. a handler registered for that name and provider;
   2. then a generic handler registered for that name with provider "".
3. If neither exists, the step does nothing and is reported as successful.

There is no substring or pattern matching, and handlers are keyed by provider, not by type. A handler registered as ("deploy", "gcp") runs only for a step named exactly deploy in a GCP service, in any GCP recipe.

Step name	Registered as	Provider of service	Matches?
build binary	build binary, ""	any	Yes (generic)
Build Binary	build binary, ""	any	Yes (case-insensitive)
build docker image	docker, ""	any	No (not exact)
deploy to cloud run	deploy to cloud run, gcp	gcp	Yes
deploy to cloud run	deploy to cloud run, gcp	aws	No (wrong provider)
ecr login	ecr, aws	aws	No (not exact)

Caution

A misspelled step name with no command silently succeeds and does nothing. After adding a recipe, run pilum dry-run and check that every step prints a command.

Built-in generic handlers, usable from any recipe:

Step name	What it runs
build binary	build.cmd (plus build.flags) via /bin/sh -c; skipped if build.cmd is empty
build docker image	docker build -f <template path> . with the provider’s image name
publish to registry	docker push of the image

---

Available Variables

These are substituted in explicit command values:

Variable	Source
${name}	Service name
${service.name}	Service name
${provider}	Service provider
${region}	Service region (one region per instance for multi-region services)
${project}	Service project
${tag}	--tag value (empty if not passed)
${build.version}	Same as ${tag}

No other ${...} is replaced, including custom keys and recipe field defaults. In a string command (run with sh -c) each value is shell-quoted. In a list command, values are inserted as-is, one argument each.

command: aws s3 sync ./dist s3://${project}-${name}/${tag}

${tag} is the raw CLI value: unlike image tags, it does not fall back to version: or latest. Always pass --tag to recipes that use it.

---

Step Configuration

Field	Type	Description
name	string	Step name; used for exact handler matching
command	string or list	Explicit command. A string runs with sh -c; a list runs directly. Overrides any handler.
execution_mode	string	service_dir (the directory containing pilum.yaml) or root (the directory pilum runs in). Default root.
timeout	int	Seconds. Overrides --timeout when set.
retries	int	Retry count. Overrides --retries when set. (default_retries, seen in built-in recipes, is not read.)
env_vars	map	Extra environment variables for this step. They override build.env_vars.
tags	list	Step tags; see below
always_run	bool	Run this step even after an earlier step failed, like defer or finally. Used for cleanup.
build_flags	map	Parsed but not used by the built-in handlers
debug	bool	Parsed; prefer the --debug CLI flag

Tags

Tags decide which commands run a step and how it is scheduled:

Tag	Selected by	Scheduling
build	pilum build (--only-tags=build), deploy, publish	Respects depends_on waves; CPU-bound worker pool
push	pilum push (--only-tags=push), deploy, publish	All services in parallel, no waves
deploy	pilum deploy; excluded by publish	Respects depends_on waves
execute	pilum deploy, publish	Respects depends_on waves

• An untagged step runs under deploy and publish (and --exclude-tags), but not under build, push or any --only-tags filter. It uses the CPU-bound worker pool but ignores depends_on waves. Tag every step.
• A step can have several tags; it is selected if any tag matches. --exclude-tags skips a step if any tag matches.
• After a step fails, every later step is skipped except those with always_run: true.

---

Checklist

• Recipe YAML with name, description, provider, service; name equals <provider>-<service>
• Required fields with descriptions; optional fields with defaults
• Steps ordered build → push → deploy, every step tagged
• Every step either has a command or exactly matches a registered handler name
• Timeouts set on each step (they override --timeout)
• Handlers registered and ingredient added (Approach 2)
• type → provider inference and schema enum updated (Approach 2)
• Services set type (and provider for custom types)
• pilum dry-run shows a command for every step

Testing

# Validate configuration (embedded recipes only)
pilum check

# List embedded recipes, or inspect one
pilum recipes
pilum recipes gcp-gke

# Preview generated commands
pilum dry-run --tag=test

# Run build steps only
pilum build --tag=test

# Full deployment
pilum deploy --tag=v1.0.0

Add --recipe-path ./my-recipes to dry-run, build and deploy when testing a recipe that is not embedded.

Next Steps

• Recipes Overview — How the recipe system works
• GCP Cloud Run — A complete built-in provider
• CLI Commands — Tag filters and other flags