Closes Fediversity/Fediversity#276
This PR adds a CLI deployment test. It builds on top of Fediversity/Fediversity#323. This test features a deployer node and four target nodes. The deployer node runs `nixops4 apply` on a deployment built with our actual code in `deployment/default.nix`, which pushes onto the four target machines combinations of Garage/Mastodon/Peertube/Pixelfed depending on a JSON payload. We check that the expected services are indeed deployed on the machines. Getting there involved reworking the existing basic test to extract common patterns, and adding support for ACME certificates negotiation inside the NixOS test.
What works:
- deployer successfully runs `nixops4 apply` with various payloads
- target machines indeed get the right services pushed onto them and removed
- services on target machines successfully negotiate ACME certificates
What does not work: the services themselves depend a lot on DNS and that is not taken care of at all, so they are probably very broken. Still, this is a good milestone.
Test it yourself by running `nix build .#checks.x86_64-linux.deployment-basic -vL` and `nix build .#checks.x86_64-linux.deployment-cli -vL`. On the very beefy machine that I am using, the basic test runs in ~4 minutes and the CLI test in ~17 minutes. We know from Fediversity/Fediversity#323 that the basic test runs in ~12 minutes on the CI runner, so maybe about an hour for the CLI test?
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Reviewed-on: Fediversity/Fediversity#329
Reviewed-by: kiara Grouwstra <kiara@procolix.eu>
Reviewed-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Nicolas “Niols” Jeannerod <nicolas.jeannerod@moduscreate.com>
Co-committed-by: Nicolas “Niols” Jeannerod <nicolas.jeannerod@moduscreate.com>
this shows a proof of concept for generating Django forms from NixOS modules
note that the form behavior is still rather clumsy and doesn't exactly map to the module semantics:
- since forms can only be sent wholesale, empty form fields will show up as empty strings
and break validation without additional cleanup (not done here)
- it's not possible to faithfully translate `type = submodule { /* ... */}; default = {};`, since the default
is translated to an empty dict `{}`. this is because the JSON schema converter does not preserve type information.
this can be added by making it use `$defs` [1], but that would likely amount to half a rewrite
- there's a glitch in enum default values that needs to be fixed in `datamodel-code-generator` [0]
[0]: dd44480359/src/datamodel_code_generator/parser/base.py (L1015)
[1]: https://json-schema.org/understanding-json-schema/structuring#defs
a generated file will be placed into the source (by the development shell and the package respectively)
that declares Pydantic types from which to render the form. it looks something like this:
```python
from __future__ import annotations
from enum import Enum
from typing import Optional
from pydantic import BaseModel, Extra, Field
from drf_pydantic import BaseModel
class Domain(Enum):
fediversity_net = 'fediversity.net'
# ...
class Model(BaseModel):
class Config:
extra = Extra.forbid
domain: Optional[Domain] = Field(
'fediversity.net',
description='Apex domain under which the services will be deployed.\n',
)
# ...
```
This PR adds a basic deployment test to the repository. This test will, in a NixOS test, run a deployer VM and a target VM, and check that we manage to run `nixops4 apply` on the deployer VM to change things on the target VM. The ideas are all @roberth's and this test has been extremely heavily inspired by https://github.com/nixops4/nixops4-nixos/blob/main/test/default/nixosTest.nix.
Reviewed-on: Fediversity/Fediversity#323
Reviewed-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Nicolas “Niols” Jeannerod <nicolas.jeannerod@moduscreate.com>
Co-committed-by: Nicolas “Niols” Jeannerod <nicolas.jeannerod@moduscreate.com>
Rotate in a new SSH key, fediversity-specific (and not password protected).
Reviewed-on: Fediversity/Fediversity#316
Co-authored-by: Nicolas “Niols” Jeannerod <nicolas.jeannerod@moduscreate.com>
Co-committed-by: Nicolas “Niols” Jeannerod <nicolas.jeannerod@moduscreate.com>
this change is a no-op (it merely indents the option definitions by one,
by setting `config` explicitly) and prepares an addition of option
declarations that would otherwise be lost in the huge diff.
Reviewed-on: Fediversity/Fediversity#315
Reviewed-by: kiara Grouwstra <kiara@procolix.eu>
closes#264
Added a clickable link to the services that deployed.
Reviewed-on: Fediversity/Fediversity#287
Reviewed-by: kiara Grouwstra <kiara@procolix.eu>
Co-authored-by: lois <lois@procolix.eu>
Co-committed-by: lois <lois@procolix.eu>
Show which services deployed and if so, the urls
Co-authored-by: kevin <kevin@procolix.com>
Reviewed-on: Fediversity/Fediversity#283
Reviewed-by: kiara Grouwstra <kiara@procolix.eu>
Co-authored-by: lois <lois@procolix.eu>
Co-committed-by: lois <lois@procolix.eu>
closes#74
Show progress indicator to track deployment
- Disable deploy button when deployment is in progress.
Co-authored-by: kevin <kevin@procolix.com>
Reviewed-on: Fediversity/Fediversity#259
Reviewed-by: kiara Grouwstra <kiara@procolix.eu>
removing fediversity.eu from form options as its subdomains are running live services
Reviewed-on: Fediversity/Fediversity#257
Reviewed-by: Kevin Muller <kevin@procolix.com>
Co-authored-by: Kiara Grouwstra <kiara@procolix.eu>
Co-committed-by: Kiara Grouwstra <kiara@procolix.eu>
adds a deploy button to the panel form - covers the local part of #76.
As a workaround to pass info (from our user form) into nixops4 uses
environment variable `DEPLOYMENT` thru nix's
`--extra-experimental-features configurable-impure-env`.
- simplify the configuration module
the `package` attribute makes little sense to be user-configurable,
since it will always need to be the derivation defined in this very
repository. for debugging one may as well change the original code itself.
- unbreak deployment
setting `CREDENTIALS_DIRECTORY` disabled the systemd mechanism set up
in the configuration module.
- remove unneeded configuration for deployment
- unbreak integration tests
before that missed waiting for the service to create some
state before running the application-level tests.
- test the form interaction for a fixed schema version
- also add a database migration missed in the last commit
Closes#73
Reviewed-on: Fediversity/Fediversity#236
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-committed-by: Valentin Gagarin <valentin.gagarin@tweag.io>
