Ingress and TLS

Expose the control plane (and optionally Prometheus) outside the cluster with ingress-nginx and cert-manager — no kubectl port-forward required.

What gets exposed

Service	Default host	Backend	Notes
Control plane (API + UI)	test-scale.local	test-scale-ctrl-plane:8080	Primary entry point
Prometheus	prometheus.test-scale.local	test-scale-prometheus:9090	Opt-in via Helm values

Postgres, NATS, workers, and the OTLP aggregator port are not exposed.

Quick start (kind / local)

1. Bring up the cluster with Ingress enabled

make k8s-ingress-up
# or
INGRESS_ENABLED=1 deploy/kubernetes/scripts/up.sh

This installs:

• ingress-nginx (NodePort 30080 / 30443)
• cert-manager + CRDs
• self-signed ClusterIssuer (test-scale-selfsigned)
• Helm release with ingress.enabled=true

2. Add a hosts entry

echo '127.0.0.1 test-scale.local' | sudo tee -a /etc/hosts

3. Open the UI

open https://test-scale.local:30443
# Browsers will warn about the self-signed cert — proceed or use HTTP:
open http://test-scale.local:30080

4. Run the smoke test through Ingress

make k8s-ingress-smoke
# or
CONTROL_PLANE_URL=https://test-scale.local:30443 INSECURE_TLS=1 \
  deploy/kubernetes/scripts/smoke.sh

Full loop:

make k8s-ingress-verify

Note: Existing make k8s-verify still uses port-forward and keeps ingress.enabled=false — CI stays unchanged.

---

Helm values

ingress:
  enabled: true
  className: nginx
  ctrlPlane:
    enabled: true
    host: test-scale.example.com
    path: /
    pathType: Prefix
  prometheus:
    enabled: false          # set true to expose Prometheus UI
    host: prometheus.test-scale.example.com
  tls:
    enabled: true
    secretName: ""          # defaults to <release>-tls
    certManager:
      enabled: true
      createCertificate: true
      clusterIssuer: letsencrypt-prod

Install on an existing cluster:

helm upgrade test-scale deploy/kubernetes/helm/test-scale \
  --namespace test-scale \
  --set ingress.enabled=true \
  --set ingress.ctrlPlane.host=test-scale.example.com \
  --set ingress.tls.certManager.clusterIssuer=letsencrypt-prod

---

Production TLS (Let’s Encrypt)

Prerequisites

1. A real DNS name pointing to your ingress load balancer
2. ingress-nginx installed with a LoadBalancer or public NodePort (not kind)
3. cert-manager installed

1. Install the ingress stack (without kind NodePort overrides)

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace

helm repo add jetstack https://charts.jetstack.io
helm upgrade --install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --set crds.enabled=true

2. Configure the issuer

Edit deploy/kubernetes/manifests/clusterissuer-letsencrypt-prod.yaml with your email, then apply:

kubectl apply -f deploy/kubernetes/manifests/clusterissuer-letsencrypt-prod.yaml

3. Deploy test-scale with Ingress

helm upgrade --install test-scale deploy/kubernetes/helm/test-scale \
  --namespace test-scale \
  --set ingress.enabled=true \
  --set ingress.ctrlPlane.host=test-scale.example.com \
  --set ingress.tls.certManager.clusterIssuer=letsencrypt-prod

cert-manager creates a Certificate and stores the TLS secret referenced by the Ingress. Check status:

kubectl get certificate -n test-scale
kubectl describe certificate test-scale-tls -n test-scale

---

Architecture

flowchart LR
    User[Browser / CLI] --> ING[ingress-nginx]
    ING -->|TLS secret| CM[cert-manager]
    CM -->|ClusterIssuer| LE[Let's Encrypt / self-signed]
    ING -->|test-scale.local| CP[ctrl-plane :8080]
    ING -.->|optional| PROM[prometheus :9090]

Troubleshooting

Symptom	Fix
If make k8s-down or make k8s-ingress-up fails with connection refused,
the kind cluster container may have stopped. Recreate it:

kind delete cluster --name test-scale
make k8s-ingress-up

up.sh auto-detects stale clusters and recreates them, but a manual delete is fastest if Docker was restarted. | Certificate stuck Pending | Check kubectl describe certificate; verify DNS and ClusterIssuer | | 404 from ingress | Confirm ingress.ctrlPlane.host matches the URL hostname | | Browser TLS warning (local) | Expected with self-signed issuer; use -k curl or HTTP on :30080 | | ACME challenge fails (prod) | DNS must resolve to ingress; port 80 must reach ingress-nginx |

Files

Path	Purpose
helm/test-scale/templates/ingress.yaml	Ingress rules
helm/test-scale/templates/certificate.yaml	cert-manager Certificate
manifests/clusterissuer-selfsigned.yaml	Local/kind issuer
manifests/clusterissuer-letsencrypt-prod.yaml	Production issuer template
scripts/install-ingress.sh	Installs nginx + cert-manager + issuer

See also 05-phase3-kubernetes.md for the full K8s walkthrough.