diff --git a/content/en/docs/v1.6/applications/openbao.md b/content/en/docs/v1.6/applications/openbao.md index 5af98b269..e2d1986ff 100644 --- a/content/en/docs/v1.6/applications/openbao.md +++ b/content/en/docs/v1.6/applications/openbao.md @@ -16,6 +16,70 @@ It provides identity-based secrets and encryption management for cloud infrastru > `storageClass` is annotated as immutable in the chart schema — see [`docs/storage-immutability.md`](../../../docs/storage-immutability.md) for the contract and which consumers enforce it. +## Auto-unseal + +By default OpenBAO uses Shamir key shares, so after every pod restart an operator has to unseal each replica by hand. Setting `seal.type: static` switches the instance to OpenBAO's [`seal "static"`](https://openbao.org/docs/configuration/seal/static/), which unseals the barrier from a 32-byte key read from a Secret in the tenant namespace. The chart never generates, stores or rotates the key. It only mounts the Secret you name. + +The choice is a trade-off. With `static` the key sits in a Secret in the same cluster as the data it unwraps, so the barrier is only as strong as access to Secrets in that namespace, to etcd and to backups. Shamir keeps the key material out of the cluster at the price of a manual unseal after every restart. Pick `static` when availability after a restart matters more than that boundary. + +1. A cluster administrator creates the key Secret in the tenant namespace. Tenant roles grant no access to Secrets, so tenant users can neither create the key nor read it back. The pods mount the Secret and do not start until it exists, so have it created before you set `seal.type: static`. The value is the base64 text of 32 random bytes, stored under the key `key`. + + ```bash + kubectl -n tenant- create secret generic openbao-unseal \ + --from-literal=key="$(head -c 32 /dev/urandom | base64 | tr -d '\n')" + ``` + +2. Point the instance at it: + + ```yaml + seal: + type: static + secretName: openbao-unseal + keyId: key-1 + ``` + + `keyId` is a permanent identifier for that key material. OpenBAO stores it with the data the key wraps and picks the key by it, so change `keyId` whenever you change the key. + +To rotate the key: + +1. Have a cluster administrator create a second Secret with the new key material, and pick a new `keyId`. +2. Move the current pair to `previousSecretName`/`previousKeyId`, put the new pair in `secretName`/`keyId` and upgrade the release. The upgrade changes the config and the StatefulSet template, not the running pods. The upstream chart uses the `OnDelete` update strategy and the server copies its HCL to `/tmp/storageconfig.hcl` at startup, so running pods keep the old key and the old config until they are replaced. +3. Replace the pods. In standalone mode delete the single pod. In HA mode delete the standby pods one at a time, waiting for each to rejoin and unseal, then delete the active pod so that leadership moves to a pod that already runs the new config. Do not delete the active pod first. Once a node with the new config has taken over, a standby still on the old template cannot unseal after a restart. + + ```bash + kubectl -n get pods -l openbao-active=true # the active pod + kubectl -n delete pod # each standby, one at a time + kubectl -n delete pod # last + ``` + +4. Confirm the rotation before touching the old key. The node that becomes active with the new config re-wraps the stored barrier and recovery keys with `current_key` and logs it. Tenant roles do not grant `pods/log`, so a cluster administrator reads that log. + + ```bash + kubectl -n logs | grep -E 'upgrading (stored keys|recovery key)' + ``` + + Until those lines have appeared, storage is still wrapped with the previous key. The static seal decrypts by the key identifier stored with the data and refuses an identifier it does not know (`unknown key id for data`), which is why both keys have to stay configured until the re-wrap is done. +5. Drop the `previous*` values, upgrade, and replace the pods once more in the same order. A pod that unseals with the new key alone proves the re-wrap. The cluster administrator deletes the old Secret last. + +### Upgrading an existing instance + +The chart labels the server pods with `policy.cozystack.io/allow-to-apiserver: "true"`, which the tenant network policy needs before `service_registration "kubernetes"` can reach the API server (#2793). The StatefulSet uses the `OnDelete` strategy, so pods created before the label was added keep running without it after an upgrade. Delete them once, in HA mode one at a time, so that they come back labelled. With the Shamir seal every replaced pod has to be unsealed again. An HA instance that hangs at start because of #2793 stays hung until its pods are replaced. + +### Migrating an existing instance from Shamir + +Switching a running Shamir-sealed instance to `static` is a seal migration, see the OpenBAO [seal migration](https://openbao.org/docs/concepts/seal/#seal-migration) documentation. The chart does not run it for you. It refuses a changed `seal.type` on an instance that already runs with another seal, because a plain config change leaves an instance that cannot unseal. OpenBAO documents the migration as a procedure with downtime, so plan a window. + +1. Take a restorable backup first, as OpenBAO recommends before any seal migration. In HA mode `bao operator raft snapshot save ` writes it. In standalone mode the file storage backend is not transactional, so a copy of the data volume taken while the server runs can be inconsistent. Copy the volume while OpenBAO is stopped, or take a volume snapshot with a storage class that snapshots atomically. +2. A cluster administrator creates the key Secret as described above. +3. Set `seal.type: static` with `secretName` and `keyId`, add `seal.allowMigration: true` and upgrade the release. The StatefulSet uses the `OnDelete` strategy, so the running pods keep the Shamir configuration until they are replaced. This upgrade is the point of no return for the chart. The server config now names the static seal while every pod still runs Shamir, and from here on the chart refuses `seal.type: shamir`. If you abort before any pod is replaced, a cluster administrator has to remove the `seal "static"` stanza from the `openbao--config` ConfigMap before the release accepts `shamir` again. +4. Standalone. Delete the pod. When it is back, run `bao operator unseal -migrate` with each Shamir unseal key. The server performs the migration and the Shamir keys become the recovery keys. +5. HA. Delete one standby pod, wait for it to come back and run `bao operator unseal -migrate` on it with each Shamir unseal key. Repeat for the other standby pods one at a time. Then run `bao operator step-down` on the active node and wait until another node has become active. The new active node performs the migration and logs its completion. Give that a little time to replicate to the other Raft nodes, then delete the old active pod, so that it restarts with the static configuration and unseals on its own. +6. Remove `seal.allowMigration` and upgrade the release once more. + +Migrating away from `static` needs the old stanza kept with `disabled = "true"` and its key still mounted, which this chart does not render. That direction is refused regardless of `allowMigration`. + +Keep a copy of the key outside the cluster. A Raft snapshot or a data PVC is unreadable without it. + ## Parameters ### Common parameters @@ -38,3 +102,16 @@ It provides identity-based secrets and encryption management for cloud infrastru | ---- | -------------------------- | ------ | ------ | | `ui` | Enable the OpenBAO web UI. | `bool` | `true` | + +### Seal + +| Name | Description | Type | Value | +| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -------- | +| `seal` | Seal configuration. The default is Shamir key shares. Set `type: static` for auto-unseal. | `object` | `{}` | +| `seal.type` | Seal type. `shamir` keeps the current behaviour, `static` enables auto-unseal from `secretName`. | `string` | `shamir` | +| `seal.secretName` | Existing Secret in the tenant namespace whose key `key` holds the base64 text of 32 random bytes (for example `openssl rand -base64 32`). A cluster administrator creates it, because tenant roles cannot create Secrets. Lowercase alphanumerics and hyphens, at most 52 characters, because the pod volume is named `userconfig-` and a volume name is a DNS label. Required when `type` is `static`. | `string` | `""` | +| `seal.keyId` | Permanent identifier of the current key. OpenBAO stores it with the data the key wraps and picks the key by it, so change it whenever the key material changes. Lowercase alphanumerics, `-`, `.` and `_`, starting and ending with an alphanumeric, because the upstream chart evaluates the server config as a template and rewrites upper case placeholders such as `HOSTNAME` in it. Required when `type` is `static`. | `string` | `""` | +| `seal.previousSecretName` | Secret holding the previous key during an n-1 key rotation, with the same name constraints as `secretName`. Keep it until the pods have been replaced and the active node has logged `upgrading stored keys`, see the README. | `string` | `""` | +| `seal.previousKeyId` | Identifier of the previous key, with the same character constraints as `keyId`. Required when `previousSecretName` is set. | `string` | `""` | +| `seal.allowMigration` | Acknowledge a seal migration. The chart refuses to change `type` on an existing instance unless this is `true`. Set it only while running `bao operator unseal -migrate`, then remove it. | `bool` | `false` | + diff --git a/content/en/docs/v1.6/networking/tcp-balancer.md b/content/en/docs/v1.6/networking/tcp-balancer.md index fa1f02053..8e36fc738 100644 --- a/content/en/docs/v1.6/networking/tcp-balancer.md +++ b/content/en/docs/v1.6/networking/tcp-balancer.md @@ -39,16 +39,16 @@ Managed TCP Load Balancer Service efficiently utilizes HAProxy for load balancin ### Application-specific parameters -| Name | Description | Type | Value | -| -------------------------------- | ------------------------------------------------------------- | ---------- | ------- | -| `httpAndHttps` | HTTP and HTTPS configuration. | `object` | `{}` | -| `httpAndHttps.mode` | Mode for balancer. | `string` | `tcp` | -| `httpAndHttps.targetPorts` | Target ports configuration. | `object` | `{}` | -| `httpAndHttps.targetPorts.http` | HTTP port number. | `int` | `80` | -| `httpAndHttps.targetPorts.https` | HTTPS port number. | `int` | `443` | -| `httpAndHttps.endpoints` | Endpoint addresses list. | `[]string` | `[]` | -| `whitelistHTTP` | Secure HTTP by whitelisting client networks (default: false). | `bool` | `false` | -| `whitelist` | List of allowed client networks. | `[]string` | `[]` | +| Name | Description | Type | Value | +| -------------------------------- | --------------------------------------------------------------------------------------------------------- | ---------- | ------- | +| `httpAndHttps` | HTTP and HTTPS configuration. | `object` | `{}` | +| `httpAndHttps.mode` | Mode for balancer. | `string` | `tcp` | +| `httpAndHttps.targetPorts` | Target ports configuration. | `object` | `{}` | +| `httpAndHttps.targetPorts.http` | HTTP port number. | `int` | `80` | +| `httpAndHttps.targetPorts.https` | HTTPS port number. | `int` | `443` | +| `httpAndHttps.endpoints` | Endpoint addresses list. | `[]string` | `[]` | +| `whitelistHTTP` | Secure HTTP and HTTPS by whitelisting client networks. Requires a non-empty `whitelist` (default: false). | `bool` | `false` | +| `whitelist` | List of allowed client networks. | `[]string` | `[]` | ## Parameter examples and reference diff --git a/content/en/docs/v1.6/operations/services/monitoring/parameters.md b/content/en/docs/v1.6/operations/services/monitoring/parameters.md index af4e8a2b8..15e871eae 100644 --- a/content/en/docs/v1.6/operations/services/monitoring/parameters.md +++ b/content/en/docs/v1.6/operations/services/monitoring/parameters.md @@ -66,26 +66,34 @@ source: https://github.com/cozystack/cozystack/blob/release-1.6/packages/extra/m ### Alerta configuration -| Name | Description | Type | Value | -| ----------------------------------------- | --------------------------------------------------------------------- | ---------- | ------- | -| `alerta` | Configuration for the Alerta service. | `object` | `{}` | -| `alerta.storage` | Persistent volume size for the database. | `string` | `10Gi` | -| `alerta.storageClassName` | StorageClass used for the database. | `string` | `""` | -| `alerta.resources` | Resource configuration. | `object` | `{}` | -| `alerta.resources.requests` | Resource requests. | `object` | `{}` | -| `alerta.resources.requests.cpu` | CPU request. | `quantity` | `100m` | -| `alerta.resources.requests.memory` | Memory request. | `quantity` | `256Mi` | -| `alerta.resources.limits` | Resource limits. | `object` | `{}` | -| `alerta.resources.limits.cpu` | CPU limit. | `quantity` | `1` | -| `alerta.resources.limits.memory` | Memory limit. | `quantity` | `1Gi` | -| `alerta.alerts` | Alert routing configuration. | `object` | `{}` | -| `alerta.alerts.telegram` | Configuration for Telegram alerts. | `object` | `{}` | -| `alerta.alerts.telegram.token` | Telegram bot token. | `string` | `""` | -| `alerta.alerts.telegram.chatID` | Telegram chat ID(s), separated by commas. | `string` | `""` | -| `alerta.alerts.telegram.disabledSeverity` | List of severities without alerts (e.g. ["informational","warning"]). | `[]string` | `[]` | -| `alerta.alerts.slack` | Configuration for Slack alerts. | `object` | `{}` | -| `alerta.alerts.slack.url` | Configuration uri for Slack alerts. | `string` | `""` | -| `alerta.alerts.slack.disabledSeverity` | List of severities without alerts (e.g. ["informational","warning"]). | `[]string` | `[]` | +| Name | Description | Type | Value | +| ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ------- | +| `alerta` | Configuration for the Alerta service. | `object` | `{}` | +| `alerta.storage` | Persistent volume size for the database. | `string` | `10Gi` | +| `alerta.storageClassName` | StorageClass used for the database. | `string` | `""` | +| `alerta.resources` | Resource configuration. | `object` | `{}` | +| `alerta.resources.requests` | Resource requests. | `object` | `{}` | +| `alerta.resources.requests.cpu` | CPU request. | `quantity` | `100m` | +| `alerta.resources.requests.memory` | Memory request. | `quantity` | `256Mi` | +| `alerta.resources.limits` | Resource limits. | `object` | `{}` | +| `alerta.resources.limits.cpu` | CPU limit. | `quantity` | `1` | +| `alerta.resources.limits.memory` | Memory limit. | `quantity` | `1Gi` | +| `alerta.alerts` | Alert routing configuration. | `object` | `{}` | +| `alerta.alerts.telegram` | Configuration for Telegram alerts. | `object` | `{}` | +| `alerta.alerts.telegram.token` | Telegram bot token. | `string` | `""` | +| `alerta.alerts.telegram.chatID` | Telegram chat ID(s), separated by commas. | `string` | `""` | +| `alerta.alerts.telegram.disabledSeverity` | List of severities without alerts (e.g. ["informational","warning"]). | `[]string` | `[]` | +| `alerta.alerts.slack` | Configuration for Slack alerts. | `object` | `{}` | +| `alerta.alerts.slack.url` | Configuration uri for Slack alerts. | `string` | `""` | +| `alerta.alerts.slack.disabledSeverity` | List of severities without alerts (e.g. ["informational","warning"]). | `[]string` | `[]` | +| `alerta.alerts.email` | Configuration for email alerts. Alerts carrying the label notify_email with the string value true are emailed when their severity is in the severities list (default critical and major). A bare YAML boolean does not match. Alerts listed in alertnames are emailed unless matched by an earlier platform route. | `object` | `{}` | +| `alerta.alerts.email.to` | Recipient address, or several separated by commas. A display name is accepted, such as `Oncall `, as long as it carries no comma. Quote display names containing punctuation such as `:`, `@`, parentheses or `?`. RFC 2047 encoded words are not accepted. IP domain literals accept dotted-quad IPv4 only. IPv6 literals are rejected because Go mail parsers disagree on their syntax across versions. ASCII control characters are rejected. Addresses must be literal, so Alertmanager template actions are rejected. The accepted form is deliberately narrower than RFC 5322. The local part must be a dot-atom, with no leading, trailing or doubled dot, and quoted local parts, comments and a `mailto:` prefix are rejected. An address outside that form fails the render, because Alertmanager parses addresses only when it sends and drops the notification with no record a tenant can read. | `string` | `""` | +| `alerta.alerts.email.from` | Sender address, one only. A display name is accepted. It takes the same narrower form as `to`, and an address outside it fails the render. | `string` | `""` | +| `alerta.alerts.email.smarthost` | SMTP server as `host:port` (e.g. "smtp.example.com:465"). A value without a port, an invalid bracketed IP address or a port with leading zeros fails the render. Port 465 uses implicit TLS, other ports use STARTTLS. | `string` | `""` | +| `alerta.alerts.email.authUsername` | Username for SMTP LOGIN/PLAIN authentication. The password is read from the `password` key of a Secret named `alertmanager-email-password` in the release namespace, which the platform operator creates. The name is fixed and the Secret is mounted into Alertmanager, so a missing Secret blocks the whole Alertmanager rollout, not only email delivery. The credential in it must belong to this tenant alone. `smarthost` is chosen by whoever writes the Monitoring, so one relay credential provisioned under the same name in every namespace can be collected by pointing `smarthost` at a host the tenant controls. | `string` | `""` | +| `alerta.alerts.email.repeatInterval` | Minimum time before a still-firing alert is emailed again, as an Alertmanager duration with units `y`, `w`, `d`, `h`, `m`, `s`, `ms` in descending order, each at most once, such as `5m`, `12h` or `1h30m`. Delivery is evaluated every 5m, so shorter intervals take effect at 5m and other intervals round up to a multiple of 5m. Zero, a unit out of order and a total beyond what Alertmanager accepts fail the render. Empty keeps the 12h default. | `string` | `""` | +| `alerta.alerts.email.alertnames` | Alert names emailed whether or not the rule carries the notify_email label, with no severity filter. Earlier platform routes still take precedence. Alerts with `event="Heartbeat"`, including `Watchdog`, go only to Alerta, and alerts with severity `none`, `CDIDefaultStorageClassDegraded` and `CDINoDefaultStorageClass` are blackholed and never reach email. This is the way to opt in alerts the platform ships, since their helm-owned rule definitions cannot be labelled per cluster. Entries must be plain Prometheus alert names such as `KubeAPIDown`, and anything else fails the render. Empty adds no extra route. | `[]string` | `[]` | +| `alerta.alerts.email.severities` | Severities the notify_email route accepts (e.g. ["critical","major","warning"]). Empty keeps the default of critical and major. List every severity in use to email opted-in alerts regardless of severity. Entries must be plain severity names, and anything else fails the render. | `[]string` | `[]` | ### Grafana configuration