Operations

Day-to-day tasks for a running install: upgrading, backing up, switching features off, reading the audit log, watching health and removing Eddy.

Full reference: docs/install.md, backup, upgrade, uninstall in the repository.

Upgrade

Upgrade the hub first, then each agent to the same chart version. The hub runs its database migrations itself when it starts, under a PostgreSQL advisory lock, so replicas that start together are fine. Migrations only move forward, and the first new replica applies them.

1. Read the release notes for the version, then apply CRD changes first. Helm installs the Cluster resource definition from a chart's crds/ directory on first install but never upgrades it:

   $ helm pull oci://ghcr.io/idestis/charts/eddy-hub --version <new> --untar --untardir /tmp/eddy$ kubectl apply -f /tmp/eddy/eddy-hub/crds/

   Copy
2. Upgrade the hub with the same values file:

   $ helm upgrade eddy-hub oci://ghcr.io/idestis/charts/eddy-hub --version <new> \--namespace eddy -f hub-values.yaml

   Copy
3. Upgrade each agent with the same chart version, and the values you installed it with:

   $ helm upgrade eddy-agent oci://ghcr.io/idestis/charts/eddy-agent --version <new> \--kube-context prod-eu --namespace eddy-system --reuse-values

   Copy

The hub rolls one replica at a time, so the UI stays up. Agents of the replica being replaced reconnect to another within seconds, and with two agent replicas the standby keeps the cluster connected. Sessions and tokens survive, because they are in PostgreSQL.

Verify

$ kubectl -n eddy rollout status deploy/eddy-hub$ kubectl --context mgmt get clusters.gitops.eddy.dev$ kubectl -n eddy logs deploy/eddy-hub

Backups

All hub state is in the PostgreSQL database: personal access tokens, threads, audit events and preferences. Sessions and rate-limit counters are there too, but they are throwaway. Cluster definitions and agent tokens live in Kubernetes, which in practice means your Helm values and the token Secrets. The database holds emails and message text, and no credentials: token secrets are stored only as HMACs.

• CloudNativePG: a ScheduledBackup with barman to object storage, or volume snapshots.
• RDS, Aurora, Cloud SQL: their automated backups and point-in-time recovery.
• Anywhere:

  $ pg_dump --format=custom "$DSN" > eddy.dump

  Copy
  Restore with pg_restore. The UNLOGGED tables are dumped too, and restoring them is harmless.

Backups contain personal data (emails, message text), so encrypt them and restrict access. Also keep a copy of your Helm values and any Secrets you create by hand, such as the credentials Secret and the session key.

Runtime kill switches

Four flags switch a feature off without a restart. They live in the eddy-runtime ConfigMap, under the key flags.yaml, which is mounted into the hub. The hub re-reads the file every 30 seconds and checks the flags on every request. Kubernetes takes up to a minute to propagate ConfigMap edits, so expect about 90 seconds in total.

1. Flip a switch now:

   $ kubectl -n eddy patch configmap eddy-runtime --type merge \-p '{"data":{"flags.yaml":"aiEnabled: false\nmcpEnabled: true\nmcpWrites: false\nmcpAllowLogs: false\n"}}'

   Copy
2. Make it stick: the chart owns this ConfigMap, so the next helm upgrade resets a manual patch. Set the same values in hub-values.yaml:

   runtimeFlags:aiEnabled: falsemcpEnabled: truemcpWrites: falsemcpAllowLogs: false

   Copy

A flag can only turn something off. It cannot enable what config.* disables (for example config.ai.enabled: false stays off whatever aiEnabled says). A missing file or key keeps the default.

Audit log

• Every write, MCP call and Ask AI step is logged as a JSON line on the hub's standard output, with user, groups, via (web, mcp or askai), cluster, target and result. Ship the hub logs to your log store.
• A copy is kept in PostgreSQL for 90 days by default (store.retention.auditDays).
• In the UI, press g a for your own events. Users in a group listed in config.auth.auditViewerGroups (prefixed, for example eddy:platform) can see other people's.
• Because actions are impersonated, each cluster's own audit log also records the real user. Alert on impersonation by the agent's ServiceAccount in those logs if you want a second view.
• To cut off a person at once, end every session and token they hold:

  $ kubectl -n eddy exec deploy/eddy-hub -- /eddy-hub admin revoke --user alice@example.com

  Copy

Monitoring

Cluster state is visible too: kubectl get clusters.gitops.eddy.dev (short name ecl) shows each cluster's phase. Logs from both components are JSON on standard output. Set networkPolicy.metrics.from to your Prometheus namespace if you turn the hub's network policy on.

Uninstall

1. Remove each agent, then the hub:

   $ helm uninstall eddy-agent -n eddy-system --kube-context prod-eu$ helm uninstall eddy-hub -n eddy

   Copy
2. Some things remain on purpose: the database, the signing-key Secret, the CRD, the clusters added in the UI with their token Secrets, and, on agents installed with a join token, the agent's token Secret. To remove everything, drop the database (or delete the CloudNativePG Cluster), then:

   $ kubectl -n eddy-system delete secret eddy-agent-token --context prod-eu$ kubectl -n eddy delete secret -l gitops.eddy.dev/cluster$ kubectl -n eddy delete secret eddy-hub-key$ kubectl delete crd clusters.gitops.eddy.dev

   Copy
   The label selects every eddy-agent-<name> token Secret, from the chart and the wizard. Deleting the CRD removes every Cluster resource with it.

Next: review the security model.