orchestrator state persisted #515

New issue

Open

opened 2025-09-11 20:24:49 +02:00 by kiara · 0 comments

kiara commented

2025-09-11 20:24:49 +02:00

Owner

for deployment, we will need to be able to transfer relevant orchestrator state, so that we may update (or migrate data between - #100) instances.

implementation notes

c.f. #493

examples:

IPs (c.f. #314)
garage credentials (or by #493 if single machine for services and garage - see #339)

the best backend seems http's nimbolus (nix example) on openbao.

backend comparison

back-ends: open-source version of comparison (c.f. earlier notes):

Backend	Locking	Encryption	Auth	Storage
`s3`	Native	server-side?	?	garage
`http`: nimbolus TF backend (nix example)	local map / Redis / Postgres	local AES key / AES from openbao key-value store (v2) / openbao transit engine	~~basic~~/JWT	S3/postgres/files/openbao
`http`: lynx	yes	?	SSO	postgres
`http`: terraform-vault-backend	?	?	?	openbao
`http`: vault-backend	?	?	?	openbao
opencredo blog	?	?	?	openbao

custom: integrations with other back-ends possible, e.g.:
- TF state - for our purposes might help keep vars state in the same place as orchestrator state
- (openbao - also unofficially available as TF back-end, see below)
- ~~valkey (fwiw, afaict not available thru TF back-ends)~~

c.f. clan's exports as a means of handling dependencies between nodes

for deployment, we will need to be able to transfer relevant orchestrator state, so that we may update (or migrate data between - #100) instances. ## implementation notes c.f. #493 examples: - [ ] IPs (c.f. #314) - [ ] garage credentials (or by #493 if single machine for services and garage - see #339) the best backend seems `http`'s [nimbolus](https://github.com/nimbolus/terraform-backend) ([nix example](https://git.dgnum.eu/DGNum/infrastructure/search/branch/main?mode=exact&q=terraform)) on [openbao](https://openbao.org/). <details> <summary> backend comparison </summary> back-ends: open-source version of [comparison](https://scalr.com/learning-center/terraform-backend-configuration-guide-choosing-the-right-state-management-solution/#backend-comparison-matrix) (c.f. [earlier notes](https://codeberg.org/kiara/e2ed-hetzner/issues/13)): | Backend | Locking | Encryption | Auth | Storage | |-|-|-|-|-| | `s3` | Native | server-side? | ? | garage | | `http`: [nimbolus TF backend](https://github.com/nimbolus/terraform-backend) ([nix example](https://git.dgnum.eu/DGNum/infrastructure/search/branch/main?mode=exact&q=terraform)) | local map / Redis / Postgres | local AES key / AES from openbao key-value store (v2) / openbao transit engine | ~~basic~~/JWT |S3/postgres/files/openbao | | `http`: [lynx](https://github.com/Clivern/Lynx) | yes | ? | SSO | postgres | | `http`: [terraform-vault-backend](https://github.com/volvo-cars/terraform-vault-backend) | ? | ? | ? | openbao | | `http`: [vault-backend](https://github.com/gherynos/vault-backend) | ? | ? | ? | openbao | | [opencredo blog](https://www.opencredo.com/blogs/securing-terraform-state-with-vault) | ? | ? | ? | openbao | ~~| `consul` | Session-based | TLS | ACL tokens | Service discovery | consul (closed-source) | Service mesh users |~~ - custom: integrations with other back-ends possible, e.g.: - TF state - for our purposes might help keep vars state in the same place as orchestrator state - ([openbao](https://openbao.org/) - also unofficially available as TF back-end, see below) - ~~[valkey](https://valkey.io/) (fwiw, afaict not available thru TF back-ends)~~ </details> - c.f. [clan's `exports`](https://docs.clan.lol/guides/services/exports/) as a means of handling dependencies between nodes