Installing Troupe on a cluster that has nothing on it¶
For a Kubernetes cluster and nothing else: no CNI that enforces policy, no ingress, no
database, no object store, no key manager, no identity provider. On a laptop,
scripts/remote-up does all of this into kind and prints where everything is; it is the
same steps automated for one target, kept working by mix troupe.e2e. On a managed
cluster with a cloud's services beside it, section 6
is a worked example.
No cluster at all, one box in a corner for a small team: plane-light.md runs the plane and its own worker from one compose file, and names what that gives up.
0. Before the first command¶
| why | without it | |
|---|---|---|
| Kubernetes ≥ 1.30 | ValidatingAdmissionPolicy |
the operator still refuses a bad profile; the cluster no longer does |
kubectl, helm |
everything below | — |
| A CNI that enforces NetworkPolicy | a worker's isolation is a NetworkPolicy | policies are accepted and enforced by nothing (kind's default CNI) |
| Cilium, optionally | egress by hostname: a worker reaches its allowlist and nothing else | a worker reaches any public host on 443 and 80, and the allowlist is checked at admission, not on the wire (profiles-and-policy.md §4) |
| An ingress controller | how a client reaches a worker | sessions are created and cannot be attached to from outside |
| Nodes that allow unprivileged user namespaces, and the chart's seccomp profile on them | every command on a pod runs in bubblewrap, which builds its sandbox from a user namespace (the workers' sandbox) | a pod refuses every command, saying bubblewrap could not start |
| Where nodes enforce AppArmor (Ubuntu's), the chart's AppArmor profile loaded on them | the runtime's default AppArmor profile, and Ubuntu's restriction on unprivileged user namespaces, stop bubblewrap (the workers' sandbox) | a pod refuses every command, saying bubblewrap could not start; on nodes without AppArmor, worker.appArmor.enabled: false |
| Kubernetes ≥ 1.33, containerd ≥ 2.0 (runc ≥ 1.2), Linux ≥ 6.3, storage that takes idmapped mounts | each worker pod runs in a user namespace of its own with its /proc unmasked, which bubblewrap's own /proc needs; the operator labels each worker namespace with the Pod Security level that takes (the workers' sandbox) |
a worker pod is refused or does not start; with worker.userNamespaces: false it starts and refuses every command |
Every worker admits HTTP only from a namespace labelled troupe.dev/ingress=true. Name
the ingress controller's namespace in operator.ingressNamespace (ingress-nginx in the
example values) and the operator puts the label there and keeps it. Left empty, the label
is yours to put on, and until some namespace that runs a pod carries it each profile's
IngressNamespaceMissing condition and the console's Overview say so, with the
kubectl label namespace line.
1. What Troupe does not run for you¶
The chart installs none of these, deliberately: each is something an organisation already
has opinions about. dev/kind/dependencies.yaml installs a development-grade one of each —
read it as a worked example, not a production install.
- PostgreSQL: one database and a role that owns it. A Helm hook Job migrates it.
- An object store: one bucket, with versioning on — the one requirement here that is not a preference, because erasure destroys prior versions.
- OpenBao (or Vault): a transit mount for signing, a KV v2 mount, and a Kubernetes auth role or a token, carrying the policies in openbao.md. The plane cannot write a policy itself; it checks what its role holds when it starts and on the console's Overview, look-only, and prints the lines for a grant it lacks. A plane with neither a role nor a token says so the same way.
- An identity provider: any OIDC provider that issues a group claim and the device grant. Troupe assigns no roles of its own; a platform admin is a member of a group you name. Requirements in detail: integrations.md.
Create the Secrets the chart expects before installing (configuration.md Part D).
2. CRDs, then the chart¶
version=X.Y.Z-beta # the release you install: https://github.com/it-minds/troupe/releases
chart=oci://ghcr.io/it-minds/charts/troupe
helm show crds "$chart" --version "$version" | kubectl apply --server-side -f -
helm upgrade --install troupe "$chart" --version "$version" \
--namespace troupe-system --create-namespace --values my-values.yaml
Every release publishes the chart twice, as oci://ghcr.io/it-minds/charts/troupe and as
troupe-<version>.tgz on its release page, and either works as $chart; so does
charts/troupe in a checkout of the release's tag. The chart's image tags default to its
version, and the images it names are public on ghcr.io/it-minds (troupe-plane,
troupe-operator, troupe-worker, troupe-a2a, troupe-gui), so nothing needs a pull
secret. A cluster that pulls through a registry of its own copies them there and sets
each *.image.repository, policy.allowedImageRepositories and imagePullSecrets.
The chart makes troupe-system itself unless createNamespace: false. Set that where the
namespace exists before the chart (made for the Secrets of section 1, or for OpenBao):
Helm does not take over a namespace it did not make, and the install stops, saying so.
Either way, uninstalling leaves the namespace and everything in it (section 7).
The CRDs go first because Helm does not upgrade CRDs it installed. Start from
values.small.yaml or values.example.yaml (the overlays);
what you cannot leave unset:
plane:
host: troupe.example.com
baseUrl: https://troupe.example.com # where a client reaches this plane
platformAdminGroup: <group id> # whose members administer this
oidc:
issuer: https://login.example.com
clientId: troupe
deviceUrl: https://login.example.com/device
tokenUrl: https://login.example.com/token
# no client secret: the console and the GUI sign in as a public client
objectStore:
endpoint: https://s3.example.com
bucket: troupe-sessions
bao:
address: https://bao.example.com
policy:
workersDomain: workers.example.com # where a worker pod is reached
allowedEgress: ["*.anthropic.com", api.openai.com, github.com]
For artifacts, the pages people and agents publish, give them a
host of their own: plane.artifactsHost (best a domain apart from the plane's, such as
artifacts.example.org; never plane.host, which the chart refuses) and
plane.artifactsTlsSecretName. The chart routes it, through an Ingress of its own, to the
plane's artifacts port and nothing else. Left empty, artifacts are published and listed,
and opening one says the plane has no artifacts host.
The database URL is the troupe-plane-database Secret. ../egress-allowlist.md
is generated from what each component declares it dials: read it before narrowing
allowedEgress, because a host the code needs and the policy refuses is a tool that fails
the first time somebody uses it.
Every command a session runs on a pod runs in bubblewrap, and three things on the nodes decide whether it can start there. The workers' sandbox says what each needs, how to check a cluster before installing, and what bubblewrap's refusals mean.
3. Prove it is up¶
kubectl -n troupe-system get pods
curl -sS https://troupe.example.com/healthz
curl -sS https://troupe.example.com/.well-known/troupe | jq .plane.build
The build names the commit, time and version; if it is not what you deployed, the rollout has not finished or the tag did not move.
Besides the API, the plane's host serves /, the page a person gets when they are handed
the URL, which says what this host is and the ways in; /docs, the concepts; /console, the
console (console.md); /admin, which sends an old bookmark to the console; and /healthz, for Kubernetes. plane.appUrl says where the graphical client is
(the chart's own at /app by default; a team with its own client sets gui.enabled: false
and points it there), and plane.cliUrl where the terminal client is published (empty,
the page says to ask an administrator).
4. The first administrator¶
Sign in at https://troupe.example.com/console. You are a platform admin because you are in
the group platformAdminGroup names, so a wrong group locks everybody out, including
you. The console checks a group before saving it — how many people would administer the
platform afterwards, and whether you are one — and TROUPE_BREAKGLASS_TOKEN, if you set
one, is Break glass on the console's sign-in screen
(roles-and-permissions.md §4).
5. The rest is the console¶
Troupe.Plane.ConsoleWalkthroughTest drives this sequence on every build:
- Identity: check the admin group.
- Policy and settings: save it, and the ceilings every team inherits.
- Teams: turn a provider group into a team.
- Profiles: write a worker profile.
- Configuration bundles: publish what a session on it gets, reading the diff first.
- Teams: grant the profile to the team.
- Identity: a service principal for work nobody starts by hand.
- Triggers: a schedule that fires as it.
- Budgets: which ceiling would refuse, and for whom.
No kubectl, environment variable or database write in any of them, and every step is in
the audit trail with a diff. The operator copies the Secrets the profile's pods mount into
its namespace from troupe-system (configuration.md Part D);
watch kubectl -n troupe-system get wp -w, where SecretMissing names any it found in
neither.
The clients people install¶
People install troupe and the desktop app from the release page with the release's
installers (quick start), or from where plane.cliUrl
points. The apps carry no Apple or Microsoft signature yet (#656), so macOS and Windows
warn the first time one downloaded in a browser runs; how to open it, why that is safe and
how to check a first download against the release key are in
security.md §6. From then on troupe
update and the desktop app check every update against that key themselves, and follow the
versions the console's Client versions sets (routine-tasks.md).
6. A worked example on managed services¶
charts/troupe/values.example.yaml is this page on a managed Kubernetes cluster, with
every value that has to be yours marked CHANGE ME. It assumes this shape:
| Need | How | Why |
|---|---|---|
| Kubernetes | a managed cluster, with Cilium as its CNI where that is offered | with Cilium the operator writes FQDN egress rules per profile; without it, egress to a named host is a wide CIDR rule |
| PostgreSQL | the provider's managed PostgreSQL, point-in-time recovery on | the session index, ledger, audit trail and identity mirror |
| Sealed session data | an S3-compatible object store, versioning on | erasure needs versioning |
| Worker disks | a block-storage class, ReadWriteOnce |
one volume per pod |
| Team volumes | a file-storage class, ReadWriteMany |
the only thing that needs it |
| Images | ghcr.io/it-minds/troupe-*, public |
a mirror of your own names a pull secret |
| Ingress and TLS | ingress-nginx behind the provider's load balancer, and cert-manager | in the cluster |
| Keys | OpenBao, in the cluster | below |
| Identity | your existing provider | integrations.md §1 |
| Models | your gateway, or the providers' own hosts | named in policy.allowedEgress |
Two DNS names, both pointing at the load balancer: the plane's (troupe.example.com)
and a wildcard for *.workers.example.com, because a client dials each pod directly at
<ordinal>-<profile>.workers.<domain>. The hyphen is what lets one wildcard record cover
every profile.
ingress-nginx needs longer timeouts and a larger body than its defaults, set in its chart's values:
controller:
config:
proxy-read-timeout: "3600" # a session is a WebSocket, idle while a model thinks
proxy-send-timeout: "3600"
proxy-body-size: 16m # the protocol's frame cap, so the worker refuses, not the proxy
Where the load balancer can send PROXY protocol, turn it on at both ends
(use-proxy-protocol: "true" here, and the provider's own annotation on the controller's
Service): without it every request arrives from the balancer's address, and the plane's
per-client rate limit becomes one bucket for everybody. Then label the controller's
namespace, as in section 0.
cert-manager issues every certificate from one ClusterIssuer, the letsencrypt both
certIssuer values in the example name:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: admin@example.com # CHANGE ME
privateKeySecretRef:
name: letsencrypt-account-key
solvers:
- http01:
ingress:
ingressClassName: nginx
HTTP-01 needs no API token for whoever hosts the zone, so it works on any DNS host. It
costs one ACME order per pod hostname, against Let's Encrypt's fifty certificates per
registered domain per week; at a size where that matters, a wildcard in
operator.tlsSecretName, issued over DNS-01, is the answer.
OpenBao stays yours. Troupe needs KV v2 and a transit engine, which a cloud's key
manager rarely is, and OpenBao auto-unseals only through a few cloud KMS APIs or another
transit engine. One replica with Raft storage is enough: there is no quorum to lose, a
restart is a window in which no session can be opened, and live sessions keep their keys
in memory. Without a KMS it can use, the seal is Shamir and something unseals it after
every restart: a person, or a share kept in a Secret, which protects a stolen volume and
not against a cluster admin. Set server.enableServiceLinks: false in its chart, so the
<SERVICE>_PORT variables Kubernetes injects cannot collide with its own. Then set up what
integrations.md §2 lists, with the plane's role and policies
as openbao.md writes them; the plane says on its Overview which grant its
role still lacks, with the lines that add it. dev/kind/dependencies.yaml has the rest as
working commands, in development shape.
Every worker namespace gets its Secrets from the operator, copied from troupe-system
at every pass (configuration.md Part D):
keep each once there, External Secrets Operator or by hand, and a new profile has them.
Sizing. The example's plane is two replicas at 1 vCPU and 1 GiB each; the rest is profiles-and-policy.md §7.
Not covered by CI. The chart runs on kind there, not on a cloud's network, storage or
load balancer. Every worker pod is internet-facing by design, protected by a pod-bound
token, the ACL and the NetworkPolicy; a load-balancer ACL or a VPN in front of *.workers
deserves a look before production. The PITR drill (scripts/pitr-drill) is a script, not
a schedule.
After the first install, an upgrade is upgrading.md, one release at a time.
7. Uninstalling¶
helm uninstall troupe -n troupe-system takes away the plane, the operator and the rest of
what the chart rendered, and leaves on purpose:
- The namespace and everything else in it: OpenBao, the Secrets, volume claims, the
WorkerProfileresources. The chart marks the namespacehelm.sh/resource-policy: keep, so neither an uninstall nor a GitOps controller's remediation that uninstalls takes the rest with it; one made withcreateNamespace: falsewas never the chart's to delete. - The CRDs, which Helm never deletes, and the
TroupePolicy, which is also marked keep. - The workers: their namespaces are the operator's, and with the operator gone nothing removes them.
To take everything away, decommission each profile
while the operator still runs, which removes its namespace, then uninstall, look at what is
left (kubectl -n troupe-system get all,secrets,pvc), and kubectl delete namespace
troupe-system. The database and the bucket are yours and are not touched.