History

Nicolas “Niols” Jeannerod ee5c2b90b7 Introduce test for deploying all services with `nixops4 apply` (#329 ) Closes Fediversity/Fediversity#276 This PR adds a CLI deployment test. It builds on top of Fediversity/Fediversity#323. This test features a deployer node and four target nodes. The deployer node runs `nixops4 apply` on a deployment built with our actual code in `deployment/default.nix`, which pushes onto the four target machines combinations of Garage/Mastodon/Peertube/Pixelfed depending on a JSON payload. We check that the expected services are indeed deployed on the machines. Getting there involved reworking the existing basic test to extract common patterns, and adding support for ACME certificates negotiation inside the NixOS test. What works: - deployer successfully runs `nixops4 apply` with various payloads - target machines indeed get the right services pushed onto them and removed - services on target machines successfully negotiate ACME certificates What does not work: the services themselves depend a lot on DNS and that is not taken care of at all, so they are probably very broken. Still, this is a good milestone. Test it yourself by running `nix build .#checks.x86_64-linux.deployment-basic -vL` and `nix build .#checks.x86_64-linux.deployment-cli -vL`. On the very beefy machine that I am using, the basic test runs in ~4 minutes and the CLI test in ~17 minutes. We know from Fediversity/Fediversity#323 that the basic test runs in ~12 minutes on the CI runner, so maybe about an hour for the CLI test? Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> Reviewed-on: Fediversity/Fediversity#329 Reviewed-by: kiara Grouwstra <kiara@procolix.eu> Reviewed-by: Valentin Gagarin <valentin.gagarin@tweag.io> Co-authored-by: Nicolas “Niols” Jeannerod <nicolas.jeannerod@moduscreate.com> Co-committed-by: Nicolas “Niols” Jeannerod <nicolas.jeannerod@moduscreate.com>		2025-05-19 02:18:54 +02:00
..
common	unify versioning (minus around nixops, for now)	2025-05-01 13:24:45 +02:00
machines	allow accessing test vms from fedi201, closes #286 (#297 )	2025-04-09 16:58:50 +02:00
test-machines	Introduce test for deploying all services with `nixops4 apply` (#329 )	2025-05-19 02:18:54 +02:00
flake-part.nix	Introduce test for deploying all services with `nixops4 apply` (#329 )	2025-05-19 02:18:54 +02:00
machines.md	Generate list of machines from config	2025-02-27 16:45:11 +01:00
machines.md.sh	Make ShellCheck happy	2025-02-27 16:45:11 +01:00
makeInstallerIso.nix
old-deployment-readme.md
proxmox-provision.sh	unify versioning (minus around nixops, for now)	2025-05-01 13:24:45 +02:00
proxmox-remove.sh
README.md	Document regenerating the list of machines	2025-02-27 16:45:11 +01:00

README.md

Infra

This directory contains the definition of the VMs that host our infrastructure.

Provisioning VMs with an initial configuration

NOTE[Niols]: This is very manual and clunky. Two things will happen. In the near future, I will improve the provisioning script to make this a bit less clunky. In the far future, NixOps4 will be able to communicate with Proxmox directly and everything will become much cleaner.

Choose names for your VMs. It is recommended to choose fediXXX, with XXX above 100. For instance, fedi117.
Add a basic configuration for the machine. These typically go in infra/machines/<name>/default.nix. You can look at other fediXXX VMs to find inspiration. You probably do not need a nixos.module option at this point.
Add a file for each of those VM's public keys, eg.
```
touch keys/systems/fedi117.pub
```
Those files need to exist during provisioning, but their content matters only when updating the machines' configuration.

FIXME: Remove this step by making the provisioning script not fail with the public key does not exist yet.
Run the provisioning script:
```
sh infra/proxmox-provision.sh fedi117
```
The script can take several ids at the same time. It requires some authentication options and provides several more. See --help.
(Optional) Add a DNS entry for the machine; for instance fedi117.abundos.eu A 95.215.187.117.
Grab the public host keys for the machines in question, and add it to the repository. For instance:
```
ssh fedi117.abundos.eu 'sudo cat /etc/ssh/ssh_host_ed25519_key.pub' > keys/systems/fedi117.pub
```
FIXME: Make the provisioning script do that for us.
Regenerate the list of machines:
```
sh infra/machines.md.sh
```
Commit it with the machine's configuration, public key, etc.
At this point, the machine contains a very basic configuration that contains just enough for it to boot and be reachable. Go on to the next section to update the machine and put an actual configuration.

FIXME: Figure out why the full configuration isn't on the machine at this point and fix it.

Updating existing VM configurations

Their configuration can be updated via NixOps4. Run

nixops4 deployments list

to see the available deployments. This should be done from the root of the repository, otherwise NixOps4 will fail with something like:

    nixops4 error: evaluation: error:
           … while calling the 'getFlake' builtin

           error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist, evaluation: error:
           … while calling the 'getFlake' builtin

           error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist

Then, given a deployment (eg. fedi200), run

nixops4 apply <deployment>

Alternatively, to run the default deployment, which contains all the VMs, run

nixops4 apply

Removing an existing VM

See infra/proxmox-remove.sh --help.