Skip to content

Installing Troupe on a cluster that has nothing on it

For a Kubernetes cluster and nothing else: no CNI that enforces policy, no ingress, no database, no object store, no key manager, no identity provider. On a laptop, scripts/remote-up does all of this into kind and prints where everything is; it is the same steps automated for one target, kept working by mix troupe.e2e. On a managed cluster with a cloud's services beside it, section 6 is a worked example.

No cluster at all, one box in a corner for a small team: plane-light.md runs the plane and its own worker from one compose file, and names what that gives up.

0. Before the first command

why without it
Kubernetes ≥ 1.30 ValidatingAdmissionPolicy the operator still refuses a bad profile; the cluster no longer does
kubectl, helm everything below —
A CNI that enforces NetworkPolicy a worker's isolation is a NetworkPolicy policies are accepted and enforced by nothing (kind's default CNI)
Cilium, optionally egress by hostname: a worker reaches its allowlist and nothing else a worker reaches any public host on 443 and 80, and the allowlist is checked at admission, not on the wire (profiles-and-policy.md §4)
An ingress controller how a client reaches a worker sessions are created and cannot be attached to from outside
Nodes that allow unprivileged user namespaces, and the chart's seccomp profile on them every command on a pod runs in bubblewrap, which builds its sandbox from a user namespace (the workers' sandbox) a pod refuses every command, saying bubblewrap could not start
Where nodes enforce AppArmor (Ubuntu's), the chart's AppArmor profile loaded on them the runtime's default AppArmor profile, and Ubuntu's restriction on unprivileged user namespaces, stop bubblewrap (the workers' sandbox) a pod refuses every command, saying bubblewrap could not start; on nodes without AppArmor, worker.appArmor.enabled: false
Kubernetes ≥ 1.33, containerd ≥ 2.0 (runc ≥ 1.2), Linux ≥ 6.3, storage that takes idmapped mounts each worker pod runs in a user namespace of its own with its /proc unmasked, which bubblewrap's own /proc needs; the operator labels each worker namespace with the Pod Security level that takes (the workers' sandbox) a worker pod is refused or does not start; with worker.userNamespaces: false it starts and refuses every command

Every worker admits HTTP only from a namespace labelled troupe.dev/ingress=true. Name the ingress controller's namespace in operator.ingressNamespace (ingress-nginx in the example values) and the operator puts the label there and keeps it. Left empty, the label is yours to put on, and until some namespace that runs a pod carries it each profile's IngressNamespaceMissing condition and the console's Overview say so, with the kubectl label namespace line.

1. What Troupe does not run for you

The chart installs none of these, deliberately: each is something an organisation already has opinions about. dev/kind/dependencies.yaml installs a development-grade one of each — read it as a worked example, not a production install.

  • PostgreSQL: one database and a role that owns it. A Helm hook Job migrates it.
  • An object store: one bucket, with versioning on — the one requirement here that is not a preference, because erasure destroys prior versions.
  • OpenBao (or Vault): a transit mount for signing, a KV v2 mount, and a Kubernetes auth role or a token, carrying the policies in openbao.md. The plane cannot write a policy itself; it checks what its role holds when it starts and on the console's Overview, look-only, and prints the lines for a grant it lacks. A plane with neither a role nor a token says so the same way.
  • An identity provider: any OIDC provider that issues a group claim and the device grant. Troupe assigns no roles of its own; a platform admin is a member of a group you name. Requirements in detail: integrations.md.

Create the Secrets the chart expects before installing (configuration.md Part D).

2. CRDs, then the chart

version=X.Y.Z-beta     # the release you install: https://github.com/it-minds/troupe/releases
chart=oci://ghcr.io/it-minds/charts/troupe
helm show crds "$chart" --version "$version" | kubectl apply --server-side -f -
helm upgrade --install troupe "$chart" --version "$version" \
  --namespace troupe-system --create-namespace --values my-values.yaml

Every release publishes the chart twice, as oci://ghcr.io/it-minds/charts/troupe and as troupe-<version>.tgz on its release page, and either works as $chart; so does charts/troupe in a checkout of the release's tag. The chart's image tags default to its version, and the images it names are public on ghcr.io/it-minds (troupe-plane, troupe-operator, troupe-worker, troupe-a2a, troupe-gui), so nothing needs a pull secret. A cluster that pulls through a registry of its own copies them there and sets each *.image.repository, policy.allowedImageRepositories and imagePullSecrets.

The chart makes troupe-system itself unless createNamespace: false. Set that where the namespace exists before the chart (made for the Secrets of section 1, or for OpenBao): Helm does not take over a namespace it did not make, and the install stops, saying so. Either way, uninstalling leaves the namespace and everything in it (section 7).

The CRDs go first because Helm does not upgrade CRDs it installed. Start from values.small.yaml or values.example.yaml (the overlays); what you cannot leave unset:

plane:
  host: troupe.example.com
  baseUrl: https://troupe.example.com   # where a client reaches this plane
  platformAdminGroup: <group id>        # whose members administer this
  oidc:
    issuer: https://login.example.com
    clientId: troupe
    deviceUrl: https://login.example.com/device
    tokenUrl: https://login.example.com/token
    # no client secret: the console and the GUI sign in as a public client
objectStore:
  endpoint: https://s3.example.com
  bucket: troupe-sessions
bao:
  address: https://bao.example.com
policy:
  workersDomain: workers.example.com    # where a worker pod is reached
  allowedEgress: ["*.anthropic.com", api.openai.com, github.com]

For artifacts, the pages people and agents publish, give them a host of their own: plane.artifactsHost (best a domain apart from the plane's, such as artifacts.example.org; never plane.host, which the chart refuses) and plane.artifactsTlsSecretName. The chart routes it, through an Ingress of its own, to the plane's artifacts port and nothing else. Left empty, artifacts are published and listed, and opening one says the plane has no artifacts host.

The database URL is the troupe-plane-database Secret. ../egress-allowlist.md is generated from what each component declares it dials: read it before narrowing allowedEgress, because a host the code needs and the policy refuses is a tool that fails the first time somebody uses it.

Every command a session runs on a pod runs in bubblewrap, and three things on the nodes decide whether it can start there. The workers' sandbox says what each needs, how to check a cluster before installing, and what bubblewrap's refusals mean.

3. Prove it is up

kubectl -n troupe-system get pods
curl -sS https://troupe.example.com/healthz
curl -sS https://troupe.example.com/.well-known/troupe | jq .plane.build

The build names the commit, time and version; if it is not what you deployed, the rollout has not finished or the tag did not move.

Besides the API, the plane's host serves /, the page a person gets when they are handed the URL, which says what this host is and the ways in; /docs, the concepts; /console, the console (console.md); /admin, which sends an old bookmark to the console; and /healthz, for Kubernetes. plane.appUrl says where the graphical client is (the chart's own at /app by default; a team with its own client sets gui.enabled: false and points it there), and plane.cliUrl where the terminal client is published (empty, the page says to ask an administrator).

4. The first administrator

Sign in at https://troupe.example.com/console. You are a platform admin because you are in the group platformAdminGroup names, so a wrong group locks everybody out, including you. The console checks a group before saving it — how many people would administer the platform afterwards, and whether you are one — and TROUPE_BREAKGLASS_TOKEN, if you set one, is Break glass on the console's sign-in screen (roles-and-permissions.md §4).

5. The rest is the console

Troupe.Plane.ConsoleWalkthroughTest drives this sequence on every build:

  1. Identity: check the admin group.
  2. Policy and settings: save it, and the ceilings every team inherits.
  3. Teams: turn a provider group into a team.
  4. Profiles: write a worker profile.
  5. Configuration bundles: publish what a session on it gets, reading the diff first.
  6. Teams: grant the profile to the team.
  7. Identity: a service principal for work nobody starts by hand.
  8. Triggers: a schedule that fires as it.
  9. Budgets: which ceiling would refuse, and for whom.

No kubectl, environment variable or database write in any of them, and every step is in the audit trail with a diff. The operator copies the Secrets the profile's pods mount into its namespace from troupe-system (configuration.md Part D); watch kubectl -n troupe-system get wp -w, where SecretMissing names any it found in neither.

The clients people install

People install troupe and the desktop app from the release page with the release's installers (quick start), or from where plane.cliUrl points. The apps carry no Apple or Microsoft signature yet (#656), so macOS and Windows warn the first time one downloaded in a browser runs; how to open it, why that is safe and how to check a first download against the release key are in security.md §6. From then on troupe update and the desktop app check every update against that key themselves, and follow the versions the console's Client versions sets (routine-tasks.md).

6. A worked example on managed services

charts/troupe/values.example.yaml is this page on a managed Kubernetes cluster, with every value that has to be yours marked CHANGE ME. It assumes this shape:

Need How Why
Kubernetes a managed cluster, with Cilium as its CNI where that is offered with Cilium the operator writes FQDN egress rules per profile; without it, egress to a named host is a wide CIDR rule
PostgreSQL the provider's managed PostgreSQL, point-in-time recovery on the session index, ledger, audit trail and identity mirror
Sealed session data an S3-compatible object store, versioning on erasure needs versioning
Worker disks a block-storage class, ReadWriteOnce one volume per pod
Team volumes a file-storage class, ReadWriteMany the only thing that needs it
Images ghcr.io/it-minds/troupe-*, public a mirror of your own names a pull secret
Ingress and TLS ingress-nginx behind the provider's load balancer, and cert-manager in the cluster
Keys OpenBao, in the cluster below
Identity your existing provider integrations.md §1
Models your gateway, or the providers' own hosts named in policy.allowedEgress

Two DNS names, both pointing at the load balancer: the plane's (troupe.example.com) and a wildcard for *.workers.example.com, because a client dials each pod directly at <ordinal>-<profile>.workers.<domain>. The hyphen is what lets one wildcard record cover every profile.

ingress-nginx needs longer timeouts and a larger body than its defaults, set in its chart's values:

controller:
  config:
    proxy-read-timeout: "3600"   # a session is a WebSocket, idle while a model thinks
    proxy-send-timeout: "3600"
    proxy-body-size: 16m         # the protocol's frame cap, so the worker refuses, not the proxy

Where the load balancer can send PROXY protocol, turn it on at both ends (use-proxy-protocol: "true" here, and the provider's own annotation on the controller's Service): without it every request arrives from the balancer's address, and the plane's per-client rate limit becomes one bucket for everybody. Then label the controller's namespace, as in section 0.

cert-manager issues every certificate from one ClusterIssuer, the letsencrypt both certIssuer values in the example name:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com     # CHANGE ME
    privateKeySecretRef:
      name: letsencrypt-account-key
    solvers:
      - http01:
          ingress:
            ingressClassName: nginx

HTTP-01 needs no API token for whoever hosts the zone, so it works on any DNS host. It costs one ACME order per pod hostname, against Let's Encrypt's fifty certificates per registered domain per week; at a size where that matters, a wildcard in operator.tlsSecretName, issued over DNS-01, is the answer.

OpenBao stays yours. Troupe needs KV v2 and a transit engine, which a cloud's key manager rarely is, and OpenBao auto-unseals only through a few cloud KMS APIs or another transit engine. One replica with Raft storage is enough: there is no quorum to lose, a restart is a window in which no session can be opened, and live sessions keep their keys in memory. Without a KMS it can use, the seal is Shamir and something unseals it after every restart: a person, or a share kept in a Secret, which protects a stolen volume and not against a cluster admin. Set server.enableServiceLinks: false in its chart, so the <SERVICE>_PORT variables Kubernetes injects cannot collide with its own. Then set up what integrations.md §2 lists, with the plane's role and policies as openbao.md writes them; the plane says on its Overview which grant its role still lacks, with the lines that add it. dev/kind/dependencies.yaml has the rest as working commands, in development shape.

Every worker namespace gets its Secrets from the operator, copied from troupe-system at every pass (configuration.md Part D): keep each once there, External Secrets Operator or by hand, and a new profile has them.

Sizing. The example's plane is two replicas at 1 vCPU and 1 GiB each; the rest is profiles-and-policy.md §7.

Not covered by CI. The chart runs on kind there, not on a cloud's network, storage or load balancer. Every worker pod is internet-facing by design, protected by a pod-bound token, the ACL and the NetworkPolicy; a load-balancer ACL or a VPN in front of *.workers deserves a look before production. The PITR drill (scripts/pitr-drill) is a script, not a schedule.

After the first install, an upgrade is upgrading.md, one release at a time.

7. Uninstalling

helm uninstall troupe -n troupe-system takes away the plane, the operator and the rest of what the chart rendered, and leaves on purpose:

  • The namespace and everything else in it: OpenBao, the Secrets, volume claims, the WorkerProfile resources. The chart marks the namespace helm.sh/resource-policy: keep, so neither an uninstall nor a GitOps controller's remediation that uninstalls takes the rest with it; one made with createNamespace: false was never the chart's to delete.
  • The CRDs, which Helm never deletes, and the TroupePolicy, which is also marked keep.
  • The workers: their namespaces are the operator's, and with the operator gone nothing removes them.

To take everything away, decommission each profile while the operator still runs, which removes its namespace, then uninstall, look at what is left (kubectl -n troupe-system get all,secrets,pvc), and kubectl delete namespace troupe-system. The database and the bucket are yours and are not touched.