Skip to content

Routine tasks

Step lists for the common changes. Each administrative step names the admin.* method it is; run it from its page in the console at /console (console.md), as the MCP tool of the same name with underscores (admin_team_enable) on /mcp, or as JSON-RPC on /rpc with a plane token. Arguments and their meanings are in the method table (roles-and-permissions.md §9). kubectl and helm assume the release troupe in troupe-system. A fresh install is installing.md.

Upgrade

The whole of it, with the backup before, what to watch, going back and the restore, is upgrading.md; one release at a time. A release that needs a step of its own before the upgrade says so in its release notes, under Before you upgrade a plane. A release publishes the chart and the images and deploys nothing; every deployment rolls it this way, from wherever it keeps its own values.

Teams

  • Add: a group must have been seen — in somebody's token at sign-in, or pushed by SCIM — before admin.team.enable can make it a team; admin.groups.list lists every group seen, whether a team draws on it and when it was last seen, and admin.teams.list the teams, with their grants, budgets, volumes, retention and spend. Edit budget, period (monthly or never), idle timeout and retention with admin.team.update. A team can draw its members from more groups: admin.team.link, admin.team.unlink (with admin.team.unlink.preview).
  • Remove: admin.team.disable.preview says what goes with it — grants, admins, principals, triggers and group links — and what stays: people, groups, and the sessions, which lose their team and become read-only. Then admin.team.disable.
  • Grant a profile: admin.team.grant, effective for sessions started afterwards. A person in two teams that hold it is asked which team a new session is for, once per profile on each app, since it decides whose budget pays. admin.team.revoke makes the team's sessions on it read-only: the running ones are put to sleep on their pods, and the ones waiting for a worker are never started.
  • Team admins: admin.team.admin.add and admin.team.admin.remove, by subject — the string the provider issues as the person. Removed, they stay a member of the team.

Service principals

admin.principal.create with the team, a name and the profiles it may use; the secret is in the answer once. admin.principal.rotate (the old secret stops at once) and admin.principal.disable. Tell every trigger and A2A caller that used the old secret. admin.principals.list {team} shows a team's principals: what each may use, when it was last used and whether it still works.

Bundles and MCP servers

  1. Check the server's host against the policy: admin.mcp.check {url}. If it is refused, add the host to policy.allowedEgress and helm upgrade.
  2. Create the token Secret once in troupe-system, labelled for workers; the operator copies it into every worker namespace whose profile follows the channel: kubectl -n troupe-system create secret generic troupe-mcp-<server> --from-literal=token=…, then kubectl -n troupe-system label secret troupe-mcp-<server> troupe.dev/worker-secret=true
  3. Add the server to the bundle with a credential_ref, then admin.bundle.validate and admin.bundle.publish {channel, content}. Pods on the channel are told at once; admin.bundle.get shows adoption per pod, and admin.bundles.list {channel} every version of the channel, newest first. admin.bundle.retire takes a version out of service for new sessions. The new MCP variable is a new StatefulSet revision, which waits for the restart below.

Drain, restart, scale

admin.profiles.list names the pods; admin.pod.drain {worker_id} sends their sessions to dormancy, to be placed elsewhere. The operator replaces a drained pod on an older revision (UpgradePending: True) by itself. Follow a drain of any other pod with

kubectl -n troupe-w-<profile> delete pod troupe-w-<profile>-<ordinal>

because nothing else restarts a drained pod that is current. Scale with max_sessions and warm_workers on admin.profile.put; the plane computes replicas (profiles-and-policy.md §3).

Settings

admin.settings.list, admin.setting.put {key, value}, admin.setting.reset {key}, visible on the writing replica at once and everywhere within 5 s. Before changing platform_admin_group, admin.identity.check {group} with the candidate; the console will not save it otherwise. What people's own machines start from and may not go past is the client_* settings and client_enforced (client-config.md). Whether people may offer their own signed-in tools to a profile's pods is the profile's personalTools (admin.profile.put, or the profile editor's switch).

Client versions

Which versions of troupe, troupe-daemon and the desktop app this plane serves, per component (tui, daemon, desktop): a recommended version, a minimum and a blocked list (Decision 868). The console's Client versions screen is admin.clients.get and admin.clients.put {component, policy: {recommended, minimum, blocked}}; a platform admin's to change, a team admin's to read, audited as clients.put. Nothing set serves every version.

  • Hold the fleet to a release: set minimum. A client below it is refused when it connects (update_required) with one sentence that says to run troupe update; nothing it is running is stopped, and the refusal is at its next connection.
  • Move the fleet: set recommended. troupe update on a machine signed in here installs that version, rather than the latest, and only if the release key signed it. After an upgrade, Recommend this plane's version on the screen sets all three to the version the plane runs, showing first each one's change and how many clients it leaves behind, as the plane counts them in a dry run that writes nothing. A plane too old to say which version it runs gets one sentence saying so in place of the button.
  • Retire a bad release: put its version in blocked, and recommend a good one. A client on it is refused whatever its number. troupe update installs a newer recommended version by itself; to go back to an older one it names the --version and leaves the choice to the person, so a plane cannot walk machines back to an old release. The recommended version may not be blocked, and the minimum may not be newer than it.

The desktop app follows the same policy (Decision 874). Signed in here, it calls the plane's initialize as troupe-gui with its version at start and once a day: blocked or below minimum, it says the refusal's sentence, keeps to the sessions on its own computer and offers Update now; below recommended, it offers the update quietly. It updates itself and the troupe-daemon beside it to the desktop component's recommended version (the latest release when there is none), never to a blocked one, below the minimum or backwards, only from a release the release key signed, and asks before it restarts.

Write versions as releases are numbered (0.13.0-beta): they are ordered as SemVer orders them, which puts 0.13.0-beta before 0.13.0. A client that does not say its version is not held to any of this, nor is anything that calls /rpc without saying which client it is.

A change here reaches machines already signed in within a minute: their troupe-daemon asks the plane again every thirty seconds while it holds a token, is judged again, and passes a changed policy to the desktop app; a changed organisation's document reaches their running sessions at the next turn. Each refusal is signed by the plane's key, and a machine acts on one only as that key signed it.

Machines that cannot reach GitHub

Mirror releases into the plane and point machines at it:

  1. On the console's Integrations page, under Releases kept on this plane, type the version and press Mirror the release (admin.releases.mirror {version}, a platform admin's). The plane fetches the release's manifest.json and its signature from GitHub's release downloads, checks them against the release keys it was built with, and refuses a manifest they did not sign with nothing copied; then it copies every file the manifest names into its object store under releases/v<version>/, each held to the manifest's size and sha256, and the manifest last. The release is listed as mirrored once its manifest is there. Audited as release.mirror, then release.mirrored or release.mirror_failed. The plane needs to reach GitHub for this; the machines do not. admin.releases.list lists what is mirrored, newest first, with any copy still under way.
  2. Under Policy, set Where people's machines get releases (client_release_url) to this plane's https://<plane>/downloads, and recommend the mirrored version here.

/downloads hands out what is mirrored to anybody, as GitHub does, and answers a bare 404 for anything else, so it says nothing about which versions are blocked. Every machine checks each file against the release key again before installing it.

Locked out of the console

Nobody is a platform admin because platform_admin_group or groups_claim is wrong:

  1. Turn break-glass on if it is not: a Secret with key token (openssl rand -base64 48), plane.breakglass.secretName naming it, helm upgrade.
  2. Open https://<plane>/console, choose Break glass and enter the token. You are a platform admin with no teams for the token's lifetime, marked on every screen and in the audit log; Sign-in shows the identity check and sign-in as the plane holds it.
  3. Where the wrong value was saved in the console, put it back to the deployment's there (Policy and settings, back to the deployment), or save the right one behind the same check. Where the deployment's own value is wrong, set plane.platformAdminGroup or plane.groupsClaim and roll.
  4. Sign in normally to confirm; turn break-glass off (secretName: "", upgrade, delete the Secret).

On a plane deployed without the console, POST /auth/breakglass with {"token": "..."} answers a plane token, and troupe admin or a /rpc call makes the same change with it (roles-and-permissions.md §4).

SCIM

On the console's Sign-in screen, make a token (admin.scim.rotate): shown once. Give the provider the token and the base URL https://<plane>/scim/v2, keyed on externalId = the subject. admin.scim.get shows the last request; admin.scim.update {attrs: {teams_from_groups: true}} makes pushed groups teams; admin.scim.delete removes the token. The deployment's troupe-plane-scim Secret (plane.scim.enabled) opens the door too. Authentik, step by step: authentik.md §4.

admin.identity.duplicates lists people held twice. A set by: "external_id" is an adoption the plane refused: a sign-in or a push under a value SCIM gave another row as its externalId, whose email and userName both differ from that row's, or a push the unique externalId refused because a sign-in had made a row under the value first. If they are one person, make the email or userName agree at the provider: the next sign-in under the value folds the row it made into SCIM's, and the next push adopts SCIM's row. Where the provider cannot say so (a token that carries neither, a person whose email and username both changed), admin.identity.adopt {row, subject}, the console's Settle on the Identity screen, typed by the subject, does it now: the sign-in's row is folded into SCIM's, which takes the subject, keeping SCIM's id; with no such row, the next push or sign-in adopts SCIM's row. If they are two people, the provider's externalId mapping gives two people one value, and is what to fix.

The identity provider

The console's Sign-in screen (admin.provider.get, admin.provider.check, admin.provider.put, admin.provider.reset) changes the issuer, client, endpoints and scopes behind a check against what the provider publishes; it overrides the deployment until reset. No secret is involved any more: every client signs in as a public client, so there is nothing to rotate. The client_secret setting and the troupe-plane-oidc Secret are still accepted this release and read by nothing; the next release drops them.

Rotate object-store credentials

Update troupe-object-store in troupe-system, which the operator copies into every worker namespace within half a minute, restart the plane, then drain and delete each worker pod, one profile at a time. Revoke the old key only after the last pod runs on the new one.

Erase a session

admin.session.erase.preview, then admin.session.erase {session_id}. Irreversible: the plane destroys the key first, then every object version goes; the owner is not told; spend stays in the ledger; the audit row keeps your name. The answer's state says whether the key went: erased, or erasure_pending when OpenBao refused or could not be reached, which the session list shows too. Erase it again to retry; the plane also retries every five minutes (Decision 811).

A team session's objects are deleted by a pod of its profile: with no healthy pod they wait until one enrols. A private session has no pod: the owner's daemon retries the key too the next time it is signed in, and the objects go once it has connected and dropped its own copy, unreadable until then because the key is gone (Decision 756). Objects the store would not delete are tried again at the next enrolment or connection, and the log says how many were left and why (Decision 804).

Decommission a profile

  1. admin.team.revoke it from every team: no new sessions, existing ones read-only.
  2. Drain every pod.
  3. admin.profile.delete: the operator deletes the troupe-w-<profile> namespace, volumes and Secrets included.
  4. Its sessions stay in object storage and the index, read-only; erase any that should go, and drop its hosts from allowedEgress if nothing else uses them.

Machines and hosts

admin.host.register (the secret is shown once, with the CA the machine pins), admin.host.rotate, admin.host.enabled, admin.hosts.list, admin.provisioners.list: single-machine.md.