add terraform-backend host to hosting-provider #882

Merged
kiara merged 2 commits from kiara/fediversity:dev-terraform-backend into main 2026-05-05 21:31:42 +02:00
Owner

Adds a long-running shared terraform-backend (nimbolus) deployed via the dev-CD path so production TF deployments can point at it as their state HTTP backend, as used in the tests.

part of #515.

note that this still hard-codes various values, raising bootstrapping questions if infra like this was intended to hold such state (along with openbao, see #493).

bootstrap layering: seed → openbao-via-dev → SPIRE → workloads

Track the dependency chain that lets us remove deployment-specific data (e.g. KMS_KEY_FILE, agenix recipients) from the codebase.

Layering

  1. Seed state via examples/hosting-env-config.nix (gitignorable). Schema gains a bootstrap/seed sub-attrset, explicitly marked bootstrap-only so we can audit which entries are still load-bearing as we climb the ladder. Prefer derived sources (TPM-sealed, generated-on-first-boot) where possible so seed entries stay minimal.
  2. openbao: Its unseal key is the one irreducible seed entry until TPM-sealed auto-unseal is wired.
  3. SPIRE issues workload identity to services on dev/operator hosts.
  4. Workloads (terraform-backend KMS, panel secrets, forgejo-runner tokens, …) source secrets from openbao via SPIRE-attested identity. Self-heal on first run: migrate any seed-supplied secret into openbao, after which the seed entry becomes vestigial and removable.

Concrete follow-ups blocked on this

  • #882 terraform-backend: drop KMS_KEY_FILE once openbao KMS is reachable (related: #515, #493).
  • agenix usage in repo: replace with SPIRE+openbao retrieval so external teams can deploy without owning Fediversity-team recipients.

Out of scope here

Implementation of any single layer — this issue only captures the chain so each piece has a documented predecessor.

Adds a long-running shared `terraform-backend` (nimbolus) deployed via the dev-CD path so production TF deployments can point at it as their state HTTP backend, as used in the tests. part of #515. note that this still hard-codes various values, raising bootstrapping questions if infra like this was intended to hold such state (along with openbao, see #493). <details> <summary> bootstrap layering: seed → openbao-via-dev → SPIRE → workloads </summary> Track the dependency chain that lets us remove deployment-specific data (e.g. `KMS_KEY_FILE`, agenix recipients) from the codebase. ## Layering 1. **Seed state** via `examples/hosting-env-config.nix` (gitignorable). Schema gains a `bootstrap`/`seed` sub-attrset, explicitly marked bootstrap-only so we can audit which entries are still load-bearing as we climb the ladder. Prefer derived sources (TPM-sealed, generated-on-first-boot) where possible so seed entries stay minimal. 2. **openbao**: Its unseal key is the one irreducible seed entry until TPM-sealed auto-unseal is wired. 3. **SPIRE** issues workload identity to services on dev/operator hosts. 4. **Workloads** (terraform-backend KMS, panel secrets, forgejo-runner tokens, …) source secrets from openbao via SPIRE-attested identity. Self-heal on first run: migrate any seed-supplied secret into openbao, after which the seed entry becomes vestigial and removable. ## Concrete follow-ups blocked on this - #882 terraform-backend: drop `KMS_KEY_FILE` once openbao KMS is reachable (related: #515, #493). - agenix usage in repo: replace with SPIRE+openbao retrieval so external teams can deploy without owning Fediversity-team recipients. ## Out of scope here Implementation of any single layer — this issue only captures the chain so each piece has a documented predecessor. </details>
kiara force-pushed dev-terraform-backend from 56158c1fca
All checks were successful
Nix flake completeness check / _complete (pull_request) Successful in 13s
checks-deployment-nixos-hosts / deployment-nixos-hosts (pull_request) Successful in 12s
checks-deployment-ssh-hosts / deployment-ssh-hosts (pull_request) Successful in 1m42s
checks-deployment-tf-hosts / deployment-tf-hosts (pull_request) Successful in 2m47s
checks-apps-panel / apps-panel (pull_request) Successful in 6m28s
checks-pre-commit / pre-commit (pull_request) Successful in 5s
checks-test-mastodon-service / test-mastodon-service (pull_request) Successful in 8s
checks-test-peertube-service / test-peertube-service (pull_request) Successful in 8s
checks-test-pixelfed-service / test-pixelfed-service (pull_request) Successful in 8s
devShells-default / default (pull_request) Successful in 5s
nix-unit-effects-common-lib / effects-common-lib (pull_request) Successful in 4s
flake-show / apps-ssh (pull_request) Successful in 10s
nix-unit-effects-tf-common-conversion / effects-tf-common-conversion (pull_request) Successful in 4s
nix-unit-lib-data-model / lib-data-model (pull_request) Successful in 7s
nix-unit-lib-function / lib-function (pull_request) Successful in 7s
nix-unit-lib-lib / lib-lib (pull_request) Successful in 4s
nixosConfigurations-dev-fedipanel / dev-fedipanel (pull_request) Successful in 2s
nixosConfigurations-dev-forgejo-actions-runner / dev-forgejo-actions-runner (pull_request) Successful in 2s
nixosConfigurations-dev-forgejo / dev-forgejo (pull_request) Successful in 3s
nixosConfigurations-dev-terraform-backend / dev-terraform-backend (pull_request) Successful in 2s
nixosConfigurations-operator-combined / operator-combined (pull_request) Successful in 2s
nixosConfigurations-operator-garage / operator-garage (pull_request) Successful in 2s
nixosConfigurations-operator-mastodon / operator-mastodon (pull_request) Successful in 3s
nixosConfigurations-operator-peertube / operator-peertube (pull_request) Successful in 3s
nixosConfigurations-operator-pixelfed / operator-pixelfed (pull_request) Successful in 2s
checks-apps-tf / apps-tf (pull_request) Successful in 10m39s
checks-deployment-tf-incus-hosts / deployment-tf-incus-hosts (pull_request) Successful in 9m9s
checks-apps-tf-incus / apps-tf-incus (pull_request) Successful in 16m24s
to 14385b8b6f
All checks were successful
Nix flake completeness check / _complete (pull_request) Successful in 12s
checks-apps-tf-incus / apps-tf-incus (pull_request) Successful in 7m5s
checks-deployment-nixos-hosts / deployment-nixos-hosts (pull_request) Successful in 10s
checks-deployment-ssh-hosts / deployment-ssh-hosts (pull_request) Successful in 1m29s
checks-apps-tf / apps-tf (pull_request) Successful in 9m24s
checks-pre-commit / pre-commit (pull_request) Successful in 5s
devShells-default / default (pull_request) Successful in 5s
flake-show / apps-ssh (pull_request) Successful in 8s
nix-unit-effects-common-lib / effects-common-lib (pull_request) Successful in 15s
nix-unit-effects-tf-common-conversion / effects-tf-common-conversion (pull_request) Successful in 7s
nix-unit-lib-data-model / lib-data-model (pull_request) Successful in 6s
checks-deployment-tf-hosts / deployment-tf-hosts (pull_request) Successful in 2m13s
nix-unit-lib-function / lib-function (pull_request) Successful in 6s
nix-unit-lib-lib / lib-lib (pull_request) Successful in 6s
nixosConfigurations-dev-fedipanel / dev-fedipanel (pull_request) Successful in 2s
nixosConfigurations-dev-forgejo-actions-runner / dev-forgejo-actions-runner (pull_request) Successful in 2s
nixosConfigurations-dev-forgejo / dev-forgejo (pull_request) Successful in 2s
nixosConfigurations-dev-terraform-backend / dev-terraform-backend (pull_request) Successful in 2s
nixosConfigurations-operator-combined / operator-combined (pull_request) Successful in 2s
nixosConfigurations-operator-garage / operator-garage (pull_request) Successful in 2s
nixosConfigurations-operator-mastodon / operator-mastodon (pull_request) Successful in 2s
nixosConfigurations-operator-peertube / operator-peertube (pull_request) Successful in 2s
nixosConfigurations-operator-pixelfed / operator-pixelfed (pull_request) Successful in 2s
checks-apps-panel / apps-panel (pull_request) Successful in 17m21s
checks-deployment-tf-incus-hosts / deployment-tf-incus-hosts (pull_request) Successful in 8m37s
2026-05-01 21:50:37 +02:00
Compare
kiara force-pushed dev-terraform-backend from 14385b8b6f
All checks were successful
Nix flake completeness check / _complete (pull_request) Successful in 12s
checks-apps-tf-incus / apps-tf-incus (pull_request) Successful in 7m5s
checks-deployment-nixos-hosts / deployment-nixos-hosts (pull_request) Successful in 10s
checks-deployment-ssh-hosts / deployment-ssh-hosts (pull_request) Successful in 1m29s
checks-apps-tf / apps-tf (pull_request) Successful in 9m24s
checks-pre-commit / pre-commit (pull_request) Successful in 5s
devShells-default / default (pull_request) Successful in 5s
flake-show / apps-ssh (pull_request) Successful in 8s
nix-unit-effects-common-lib / effects-common-lib (pull_request) Successful in 15s
nix-unit-effects-tf-common-conversion / effects-tf-common-conversion (pull_request) Successful in 7s
nix-unit-lib-data-model / lib-data-model (pull_request) Successful in 6s
checks-deployment-tf-hosts / deployment-tf-hosts (pull_request) Successful in 2m13s
nix-unit-lib-function / lib-function (pull_request) Successful in 6s
nix-unit-lib-lib / lib-lib (pull_request) Successful in 6s
nixosConfigurations-dev-fedipanel / dev-fedipanel (pull_request) Successful in 2s
nixosConfigurations-dev-forgejo-actions-runner / dev-forgejo-actions-runner (pull_request) Successful in 2s
nixosConfigurations-dev-forgejo / dev-forgejo (pull_request) Successful in 2s
nixosConfigurations-dev-terraform-backend / dev-terraform-backend (pull_request) Successful in 2s
nixosConfigurations-operator-combined / operator-combined (pull_request) Successful in 2s
nixosConfigurations-operator-garage / operator-garage (pull_request) Successful in 2s
nixosConfigurations-operator-mastodon / operator-mastodon (pull_request) Successful in 2s
nixosConfigurations-operator-peertube / operator-peertube (pull_request) Successful in 2s
nixosConfigurations-operator-pixelfed / operator-pixelfed (pull_request) Successful in 2s
checks-apps-panel / apps-panel (pull_request) Successful in 17m21s
checks-deployment-tf-incus-hosts / deployment-tf-incus-hosts (pull_request) Successful in 8m37s
to 371b99f14c
Some checks failed
Nix flake completeness check / _complete (pull_request) Successful in 11s
checks-deployment-nixos-hosts / deployment-nixos-hosts (pull_request) Successful in 11s
checks-deployment-ssh-hosts / deployment-ssh-hosts (pull_request) Successful in 1m31s
checks-deployment-tf-hosts / deployment-tf-hosts (pull_request) Successful in 2m32s
checks-pre-commit / pre-commit (pull_request) Successful in 5s
devShells-default / default (pull_request) Successful in 5s
flake-show / apps-ssh (pull_request) Successful in 27s
nix-unit-effects-common-lib / effects-common-lib (pull_request) Successful in 8s
nix-unit-effects-tf-common-conversion / effects-tf-common-conversion (pull_request) Successful in 7s
nix-unit-lib-data-model / lib-data-model (pull_request) Successful in 7s
nix-unit-lib-function / lib-function (pull_request) Successful in 8s
nix-unit-lib-lib / lib-lib (pull_request) Successful in 7s
nixosConfigurations-dev-combined / dev-combined (pull_request) Successful in 2s
nixosConfigurations-dev-forgejo-actions-runner / dev-forgejo-actions-runner (pull_request) Successful in 2s
nixosConfigurations-dev-forgejo / dev-forgejo (pull_request) Successful in 2s
nixosConfigurations-hosting-provider-combined / hosting-provider-combined (pull_request) Failing after 1s
nixosConfigurations-hosting-provider-panel / hosting-provider-panel (pull_request) Successful in 2s
nixosConfigurations-hosting-provider-terraform-backend / hosting-provider-terraform-backend (pull_request) Failing after 1s
nixosConfigurations-operator-combined / operator-combined (pull_request) Successful in 2s
nixosConfigurations-operator-garage / operator-garage (pull_request) Successful in 2s
nixosConfigurations-operator-mastodon / operator-mastodon (pull_request) Successful in 2s
nixosConfigurations-operator-peertube / operator-peertube (pull_request) Successful in 2s
nixosConfigurations-operator-pixelfed / operator-pixelfed (pull_request) Successful in 2s
checks-apps-tf-incus / apps-tf-incus (pull_request) Successful in 8m3s
checks-apps-panel / apps-panel (pull_request) Successful in 10m54s
checks-apps-tf / apps-tf (pull_request) Successful in 11m22s
checks-deployment-tf-incus-hosts / deployment-tf-incus-hosts (pull_request) Successful in 11m7s
2026-05-05 19:51:28 +02:00
Compare
decouple cd-paths from dev to extend to hosting-provider
All checks were successful
Nix flake completeness check / _complete (pull_request) Successful in 10s
checks-deployment-nixos-hosts / deployment-nixos-hosts (pull_request) Successful in 11s
checks-deployment-ssh-hosts / deployment-ssh-hosts (pull_request) Successful in 1m30s
checks-pre-commit / pre-commit (pull_request) Successful in 5s
devShells-default / default (pull_request) Successful in 6s
flake-show / apps-ssh (pull_request) Successful in 29s
nix-unit-effects-common-lib / effects-common-lib (pull_request) Successful in 8s
nix-unit-effects-tf-common-conversion / effects-tf-common-conversion (pull_request) Successful in 7s
nix-unit-lib-data-model / lib-data-model (pull_request) Successful in 8s
checks-deployment-tf-hosts / deployment-tf-hosts (pull_request) Successful in 2m29s
nix-unit-lib-function / lib-function (pull_request) Successful in 7s
nix-unit-lib-lib / lib-lib (pull_request) Successful in 6s
nixosConfigurations-dev-combined / dev-combined (pull_request) Successful in 2s
nixosConfigurations-dev-forgejo-actions-runner / dev-forgejo-actions-runner (pull_request) Successful in 2s
nixosConfigurations-dev-forgejo / dev-forgejo (pull_request) Successful in 2s
nixosConfigurations-hosting-provider-combined / hosting-provider-combined (pull_request) Successful in 1s
nixosConfigurations-hosting-provider-panel / hosting-provider-panel (pull_request) Successful in 2s
nixosConfigurations-hosting-provider-terraform-backend / hosting-provider-terraform-backend (pull_request) Successful in 2s
nixosConfigurations-operator-combined / operator-combined (pull_request) Successful in 2s
nixosConfigurations-operator-garage / operator-garage (pull_request) Successful in 2s
nixosConfigurations-operator-mastodon / operator-mastodon (pull_request) Successful in 2s
nixosConfigurations-operator-peertube / operator-peertube (pull_request) Successful in 2s
nixosConfigurations-operator-pixelfed / operator-pixelfed (pull_request) Successful in 2s
checks-apps-tf-incus / apps-tf-incus (pull_request) Successful in 7m33s
checks-deployment-tf-incus-hosts / deployment-tf-incus-hosts (pull_request) Successful in 9m38s
checks-apps-tf / apps-tf (pull_request) Successful in 10m3s
checks-apps-panel / apps-panel (pull_request) Successful in 24m57s
0aee824e8b
Signed-off-by: Kiara Grouwstra <kiara@procolix.eu>
kiara changed title from WIP: add terraform-backend dev host to add terraform-backend dev host 2026-05-05 21:30:58 +02:00
kiara changed title from add terraform-backend dev host to add terraform-backend host to hosting-provider 2026-05-05 21:31:33 +02:00
kiara merged commit 805b8e7d44 into main 2026-05-05 21:31:42 +02:00
Sign in to join this conversation.
No reviewers
fediversity/developers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
fediversity/fediversity!882
No description provided.