Architecture

test-scale is six cooperating services plus your target under test. This page explains responsibilities and data flow; deployment specifics live under Deployment.

Components

#	Component	Responsibility
1	CLI / UI	Submit plans, watch runs, compare history
2	Control plane	REST API, orchestrator, Postgres — source of truth for run state
3	Job queue (NATS)	Durable hand-off so workers can scale and fail independently
4	Autoscaler	Brain computes worker count; scaler applies it (Docker or KEDA)
5	Worker pool	Stateless HTTP load generators
6	Metrics pipeline	Aggregator merges histograms → Prometheus → Grafana

flowchart LR
    Client --> CP[Control Plane]
    CP --> JQ[NATS]
    CP --> AS[Autoscaler]
    AS --> WP[Workers]
    JQ --> WP
    WP --> TGT[Target]
    WP --> AGG[Aggregator]
    AGG --> PROM[Prometheus]
    PROM --> GF[Grafana]

Control plane

Validates and normalizes plans (HTTP method, auth → headers, JSON → body).
Persists plans and runs (pending → running → completed / failed).
Orchestrates pre-scale, dispatch, heartbeats, worker replacement, and report collection.
Exposes /metrics including testscale_desired_workers for KEDA.

Autoscaler: brain vs scaler

Layer	Question it answers	Docker	Kubernetes
Brain	How many workers for this `target_rps`?	Shared Go code	Shared Go code
Scaler	How do I create that many processes?	`docker compose scale`	KEDA + Prometheus metric

The brain uses ceil(target_rps / per_worker_rps) and pre-scales before load starts so workers are ready when the run begins. CPU-based HPA is intentionally avoided for load tests because it reacts too slowly.

See Kubernetes and KEDA for the full loop.

Workers

Workers are stateless:

Receive a slice of the plan via NATS (URL, headers, body, per-worker RPS, ramp pattern).
Execute the HTTP client loop with timeouts and status classification.
Stream OpenTelemetry metrics to the aggregator.
Send heartbeats every 5s during load; missing heartbeats for 15s mark a worker dead.

If a worker dies mid-run, the orchestrator can spawn a replacement with remaining_sec so the run still completes with partial time coverage.

Metrics and percentiles

Each worker tracks latency in a histogram. The aggregator merges histograms so global p95/p99 are statistically meaningful — averaging per-worker percentiles would lie.

Live dashboards read Prometheus; completed runs store summarized reports in Postgres for the UI and diff.

Technology stack (v0.1)

Concern	Choice
Control plane	Go, Gin
Persistence	Postgres, versioned SQL migrations
Queue	NATS
Workers / aggregator	Go, OTLP
Metrics	Prometheus, Grafana
Local orchestration	Docker Compose
Cluster orchestration	Kubernetes, Helm, KEDA