GCP Cloud Run Jobs

Deploy batch workloads, migrations, and run-to-completion tasks with Google Cloud Run Jobs.

Prerequisites

Google Cloud SDK installed and configured
Docker installed
A GCP project with Cloud Run API enabled

# Install gcloud
brew install google-cloud-sdk

# Authenticate
gcloud auth login
gcloud auth configure-docker

# Set project
gcloud config set project YOUR_PROJECT_ID

Service Configuration

name: db-migrations
provider: gcp
type: gcp-cloud-run-job
project: my-gcp-project
region: us-central1
registry_name: us-docker.pkg.dev/my-project/my-repo

build:
  language: go
  version: "1.23"
  cmd: "go build -o ./dist/migrate ./cmd/migrate"
  env_vars:
    CGO_ENABLED: "0"

job:
  timeout: "300s"
  memory: "1Gi"
  cpu: "1"
  max_retries: 2
  execute_on_deploy: true
  service_account: [email protected]

secrets:
  DATABASE_URL: projects/123456/secrets/database-url:latest

depends_on:
  - my-api

Required Fields

Field	Description
`name`	Job name
`project`	GCP project ID
`region`	GCP region (default: `us-central1`)
`registry_name`	Docker registry prefix
`template`	Dockerfile template name

Optional Fields

Job Configuration

Field	Type	Default	Description
`job.timeout`	string	`600s`	Maximum execution time per task
`job.max_retries`	int	`0`	Retries on task failure
`job.parallelism`	int	`1`	Maximum concurrent tasks
`job.task_count`	int	`1`	Total number of tasks
`job.cpu`	string	`1`	CPUs per task
`job.memory`	string	`512Mi`	Memory per task
`job.execute_on_deploy`	bool	`false`	Auto-execute after deployment
`job.service_account`	string	-	Custom service account email

Cloud Run Jobs vs Cloud Run Services

Feature	Cloud Run Service	Cloud Run Job
Runs	Continuously (HTTP server)	To completion (batch)
Triggered by	HTTP requests	Manual, scheduler, or on deploy
Scaling	0-N based on traffic	Fixed task count
Use case	APIs, web apps	Migrations, ETL, data processing

Deployment Steps

The GCP Cloud Run Jobs recipe executes:

build binary — Compile the application
build docker image — Create container image
publish to registry — Push to Artifact Registry
deploy job — Deploy with gcloud run jobs deploy
execute job — Run the job (only if execute_on_deploy: true)

Dependency Ordering

Cloud Run Jobs support depends_on for wave-based deployment. When a job depends on a service, Pilum deploys the service first:

name: db-migrations
type: gcp-cloud-run-job
depends_on:
  - database-proxy  # Deploy proxy before running migrations

Pilum groups services into waves by dependency depth and deploys each wave sequentially.

Execute on Deploy

When job.execute_on_deploy is true, Pilum automatically runs the job after deployment with --wait to block until completion:

gcloud run jobs execute db-migrations \
  --region us-central1 \
  --wait

If execute_on_deploy is false (the default), the execute step is skipped.

Example Commands

# Deploy and execute job
pilum deploy --tag=v1.0.0

# Deploy without executing (override execute_on_deploy)
pilum deploy --exclude-tags=execute --tag=v1.0.0

# Execute step only (job must already be deployed)
pilum deploy --only-tags=execute --tag=v1.0.0

# Preview commands
pilum dry-run --tag=v1.0.0

Generated Deploy Command

Pilum generates:

gcloud run jobs deploy db-migrations \
  --image us-docker.pkg.dev/my-project/my-repo/db-migrations:v1.0.0 \
  --region us-central1 \
  --project my-project \
  --task-timeout=300s \
  --max-retries=2 \
  --parallelism=1 \
  --tasks=1 \
  --cpu=1 \
  --memory=1Gi \
  --set-secrets=DATABASE_URL=projects/123456/secrets/database-url:latest \
  [email protected]

Troubleshooting

Job Fails to Execute

Step 5: execute job
  db-migrations  ✗ - execution failed

Check job logs:

gcloud run jobs executions list --job db-migrations --region us-central1
gcloud logging read "resource.type=cloud_run_job AND resource.labels.job_name=db-migrations" --limit 50

Timeout Exceeded

If your job exceeds the timeout, increase job.timeout:

job:
  timeout: "1800s"  # 30 minutes

Execute Step Skipped

If the execute step isn’t running, ensure execute_on_deploy is set:

job:
  execute_on_deploy: true

Or run the execute step explicitly:

pilum deploy --only-tags=execute --tag=v1.0.0

Next Steps

GCP Cloud Run — Deploy HTTP services
Service Configuration — Full reference
CLI Commands — All commands