Operator guide

Deployment profile

Use the OCI chart with values-production.yaml as a starting point. The production profile expects externally operated Postgres and NATS:

kubectl create namespace test-scale
kubectl create secret generic test-scale-postgres -n test-scale \
  --from-literal=password='replace-me'

helm install test-scale \
  oci://ghcr.io/mohamedbentaher/charts/test-scale \
  --version 0.1.0 \
  --namespace test-scale \
  -f values-production.yaml

Replace the example database and NATS endpoints. Configure network policies so only the control plane, aggregator, and workers can reach them. Keep the API and NATS private unless an authenticated ingress is explicitly required.

Credentials

The chart generates API, worker, and target-verification credentials and preserves them across upgrades. Inject explicit values from a secret manager when predictable rotation is required. Retrieve the API key with the command printed by Helm NOTES.txt.

Target request headers and plan auth values can contain credentials. Restrict read access to plans, avoid committing tokens into shared YAML, and prefer short-lived tokens for load tests. Treat database backups and exported run JSON as credential-bearing data.

Retention and export

Set ctrlPlane.retentionDays (or RUN_RETENTION_DAYS) to delete terminal runs after a fixed number of days. Zero disables automatic retention. Reports are deleted with their run.

Export one run before pruning:

curl -fsS \
  -H "X-Test-Scale-Key: ${TEST_SCALE_API_KEY}" \
  "${CONTROL_PLANE_URL}/api/v1/runs/${RUN_ID}/export" \
  -o "run-${RUN_ID}.json"

Backup and restore

Use a least-privilege PostgreSQL connection:

DATABASE_URL='postgres://…' scripts/backup-postgres.sh backup.dump
DATABASE_URL='postgres://…' CONFIRM_RESTORE=yes \
  scripts/restore-postgres.sh backup.dump

Test restores regularly. Back up before every chart upgrade that contains a new database migration. Restores are destructive and should target an empty or disposable database first.

Upgrade and rollback

1. Read the changelog and back up Postgres.
2. Render the new chart with production values.
3. Upgrade with --atomic --timeout 10m.
4. Run a low-throughput smoke plan.
5. Roll back the chart if workloads fail, but restore the database only when release notes explicitly mark a migration as non-backward-compatible.

Uninstalling the chart does not delete externally operated Postgres or NATS.