Compare commits

...

49 commits

Author SHA1 Message Date
b294149af7
use a nixpkgs version unbreaking both TF and peertube
see 318 about the regression breaking the peertube test
2025-04-22 20:08:03 +02:00
b1bf597e94
fix authorized keys 2025-04-22 19:19:22 +02:00
95f9dc48bb
nixpkgs version that makes peertube test pass again 2025-04-22 14:29:31 +02:00
baa9b8871b
update hosts 2025-04-21 21:12:53 +02:00
19fe0af24f
import home-manager, deploy to fedipanel from tf working 2025-04-21 21:12:22 +02:00
dbd45b7027
panel working again locally 2025-04-21 20:55:48 +02:00
f0f0a7988d
infra tf-validate tests working 2025-04-21 20:31:10 +02:00
effad34170
fix root test/hook 2025-04-21 17:03:27 +02:00
d397c7dfea
services tests run, and fail 2025-04-21 16:30:46 +02:00
7b985c0b81
move stuff around, utterly untested 2025-04-21 16:00:59 +02:00
0919db08f7 wip: handle infra by TF 2025-04-18 19:09:03 +02:00
754bf4d8f9 switch dependency management to npins, closes #279 2025-04-18 17:41:11 +02:00
ad65fb944d clean out nixops bits 2025-04-18 17:41:11 +02:00
09899c7f3a move gitignore to apply to infra dir 2025-04-18 17:41:11 +02:00
65393abf9c clean documentation 2025-04-18 17:41:11 +02:00
173518ed90 setup: reset lock file, prevents Error: Failed to install provider
error example:

```
Error while installing hashicorp/external v2.3.4: the local package for
registry.opentofu.org/hashicorp/external 2.3.4 doesn't match any of the
checksums previously recorded in the dependency lock file (this might be
because the available checksums are for packages targeting different
platforms); for more information:
https://opentofu.org/docs/language/files/dependency-lock/#checksum-verification
```
2025-04-18 17:35:43 +02:00
84e51745f8 fix nix-hash 2025-04-18 16:51:38 +02:00
2fef5c42aa further purge test machines from nixops 2025-04-18 13:42:49 +02:00
cdd39543bf rm corresponding nixops deployments 2025-04-18 13:36:27 +02:00
f2e294e77c add unit test: TF validate 2025-04-18 13:21:17 +02:00
9090269e55 add further comments 2025-04-18 10:53:08 +02:00
6843e84657 consolidate setup in script 2025-04-17 20:42:38 +02:00
e6b06c86a6 move info passed to facilitate passing extra config 2025-04-17 17:52:07 +02:00
3716de6409 fix returncode 2025-04-17 17:01:03 +02:00
8f2571ac70 better document TF file 2025-04-17 16:59:11 +02:00
90eae7c336 split out vars 2025-04-17 16:58:27 +02:00
5ce0987640 retain nix_path for debugging 2025-04-17 16:07:16 +02:00
69b6cac387 ditch nixpkgs link from /run/current-system 2025-04-17 16:04:38 +02:00
4f93303ef8 fix key path 2025-04-17 15:03:52 +02:00
376d003e28 add more comments 2025-04-17 14:23:08 +02:00
1aa9e959d3 Revert "update nixpkgs to ditch opentofu patch"
This reverts commit 7bab876d1b.
2025-04-17 14:22:51 +02:00
7bab876d1b update nixpkgs to ditch opentofu patch 2025-04-17 14:10:27 +02:00
a5b74452c8 rm unused import 2025-04-17 13:59:52 +02:00
428b1d4e3a more comments 2025-04-17 13:52:46 +02:00
4c6fa980f7 ignore generated tf lock 2025-04-17 13:52:09 +02:00
d26b3502ab reinstate user group, fixes application-tests 2025-04-17 13:27:43 +02:00
31fce3acd2 rm unused file 2025-04-17 13:22:20 +02:00
3f516da411 rm some unused code 2025-04-17 13:22:20 +02:00
a4a2c8c4c1 implicit ssh 2025-04-17 13:22:20 +02:00
c649802896 set env var specific to deployed setting 2025-04-17 13:22:20 +02:00
b3767d30ce Revert "kill git hook"
This reverts commit f7d050a3a6.
2025-04-17 13:22:20 +02:00
cbc4937427 filter for git-tracked files 2025-04-17 13:22:20 +02:00
d409c75476 rm tf lock 2025-04-17 13:22:20 +02:00
be8dec1f86 kill git hook 2025-04-17 13:22:20 +02:00
44dfef8cb4 mv env var 2025-04-17 13:22:20 +02:00
44711d52ec rm null from tf 2025-04-17 13:22:20 +02:00
dcd87a3f76 rm unused provider from lock 2025-04-17 13:22:20 +02:00
38c12e82e9 buttons works deployed 2025-04-17 13:22:20 +02:00
3581ab5af1 fix npins 2025-04-17 13:21:23 +02:00
107 changed files with 1192 additions and 2213 deletions

4
.envrc
View file

@ -3,8 +3,8 @@
# shellcheck shell=bash
if type -P lorri &>/dev/null; then
eval "$(lorri direnv --flake .)"
eval "$(lorri direnv)"
else
echo 'while direnv evaluated .envrc, could not find the command "lorri" [https://github.com/nix-community/lorri]'
use flake
use_nix
fi

View file

@ -13,16 +13,22 @@ jobs:
runs-on: native
steps:
- uses: actions/checkout@v4
- run: nix build .#checks.x86_64-linux.pre-commit -L
- run: nix-build -A tests
check-peertube:
check-services:
runs-on: native
steps:
- uses: actions/checkout@v4
- run: nix build .#checks.x86_64-linux.peertube -L
- run: cd services && nix-build -A tests.peertube
check-panel:
runs-on: native
steps:
- uses: actions/checkout@v4
- run: cd panel && nix-build -A tests
check-infra:
runs-on: native
steps:
- uses: actions/checkout@v4
- run: cd infra && nix-build -A tests

6
.gitignore vendored
View file

@ -1,3 +1,9 @@
.npins.json
.terraform/
.terraform.lock.hcl
.terraform.tfstate.lock.info
terraform.tfstate*
.auto.tfvars.json
.DS_Store
.idea
*.log

View file

@ -1,8 +1,7 @@
# The Fediversity project
This repository contains all the code and code-related files having to do with
[the Fediversity project](https://fediversity.eu/), with the notable exception
of [NixOps4 that is hosted on GitHub](https://github.com/nixops4/nixops4).
[the Fediversity project](https://fediversity.eu/).
## Goals
@ -81,27 +80,15 @@ Not everyone has the expertise and time to run their own server.
The software includes technical configuration that links software components.
Most user-facing configuration remains untouched by the deployment process.
> Example: NixOps4 is used to deploy [Pixelfed](https://pixelfed.org).
> Example: OpenTofu is used to deploy [Pixelfed](https://pixelfed.org).
- Migrate
Move service configurations and user data to a different hosting provider.
- [NixOps4](https://github.com/nixops4/nixops4)
- [OpenTofu](https://opentofu.org/)
A tool for deploying and managing resources through the Nix language.
NixOps4 development is supported by the Fediversity project
- Resource
A [resource for NixOps4](https://nixops.dev/manual/development/concept/resource.html) is any external entity that can be declared with NixOps4 expressions and manipulated with NixOps4, such as a virtual machine, an active NixOS configuration, a DNS entry, or customer database.
- Resource provider
A resource provider for NixOps4 is an executable that communicates between a resource and NixOps4 using a standardised protocol, allowing [CRUD operations](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) on the resources to be performed by NixOps4.
Refer to the [NixOps4 manual](https://nixops.dev/manual/development/resource-provider/index.html) for details.
> Example: We need a resource provider for obtaining deployment secrets from a database.
An infrastructure-as-code tool, and open-source (MPL 2.0) fork of Terraform.
## Development
@ -118,9 +105,6 @@ Contact the project team if you have questions or suggestions, or if you're inte
Most of the directories in this repository have their own README going into more
details as to what they are for. As an overview:
- [`deployment/`](./deployment) contains work to generate a full Fediversity
deployment from a minimal configuration.
- [`infra/`](./infra) contains the configurations for the various VMs that are
in production for the project, for instance the Git instances or the Wiki, as
well as means to provision and set up new ones.
@ -128,14 +112,8 @@ details as to what they are for. As an overview:
- [`keys/`](./keys) contains the public keys of the contributors to this project
as well as the systems that we administrate.
- [`matrix/`](./matrix) contains everything having to do with setting up a
fully-featured Matrix server.
- [`secrets/`](./secrets) contains the secrets that need to get injected into
machine configurations.
- [`services/`](./services) contains our effort to make Fediverse applications
work seemlessly together in our specific setting.
- [`website/`](./website) contains the framework and the content of [the
Fediversity website](https://fediversity.eu/)

57
default.nix Normal file
View file

@ -0,0 +1,57 @@
{
system ? builtins.currentSystem,
sources ? import ./npins,
pkgs ? import sources.nixpkgs { inherit system; },
}:
let
inherit (sources) nixpkgs git-hooks gitignore;
inherit (pkgs) lib;
pre-commit-check =
(import "${git-hooks}/nix" {
inherit nixpkgs system;
gitignore-nix-src = {
lib = import gitignore { inherit lib; };
};
}).run
{
src = ./.;
hooks =
let
## Add a directory here if pre-commit hooks shouldn't apply to it.
optout = [
"npins"
".terraform"
];
excludes = map (dir: "^${dir}/") optout;
addExcludes = lib.mapAttrs (_: c: c // { inherit excludes; });
in
addExcludes {
nixfmt-rfc-style.enable = true;
deadnix.enable = true;
trim-trailing-whitespace.enable = true;
shellcheck.enable = true;
};
};
in
{
# shell for testing TF directly
shell = pkgs.mkShellNoCC {
inherit (pre-commit-check) shellHook;
buildInputs = pre-commit-check.enabledPackages;
packages = [
pkgs.nixfmt-rfc-style
];
};
tests = {
inherit pre-commit-check;
};
# re-export inputs so they can be overridden granularly
# (they can't be accessed from the outside any other way)
inherit
sources
system
pkgs
;
}

View file

@ -1,6 +0,0 @@
# Deployment
This repository contains work to generate a full Fediversity deployment from a
minimal configuration. This is different from [`../services/`](../services) that
focuses on one machine, providing a polished and unified interface to different
Fediverse services.

View file

@ -1,181 +0,0 @@
## `makeMakeDeployment` -- Function to help hosting providers make a
## `makeDeployment` function.
##
## https://factoryfactoryfactory.net/
## Generic utilities used in this function, eg. nixpkgs, NixOps4 providers, etc.
## REVIEW: We should maybe be more specific than just `inputs`.
{
lib,
nixops4,
nixops4-nixos,
fediversity,
}:
## Information on the hosting provider's infrastructure. This is where we inform
## this function of where it can find eg. Proxmox.
{
## Four NixOS configuration resource modules for four services. Those are VMs
## that are already deployed and on which we will push our configurations.
##
## - Ultimately, we just want a pool of VMs, or even just a Proxmox.
## - Each machine is flagged for a certain use case until we control DNS.
garageConfigurationResource,
mastodonConfigurationResource,
peertubeConfigurationResource,
pixelfedConfigurationResource,
}:
## From the hosting provider's perspective, the function is meant to be
## partially applied only until here.
## Information on the specific deployment that we request. This is the
## information coming from the FediPanel.
##
## FIXME: lock step the interface with the definitions in the FediPanel
panelConfig:
let
inherit (lib) mkIf;
in
## Regular arguments of a NixOps4 deployment module.
{ providers, ... }:
{
providers = { inherit (nixops4.modules.nixops4Provider) local; };
resources =
let
## NOTE: All of these secrets are publicly available in this source file
## and will end up in the Nix store. We don't care as they are only ever
## used for testing anyway.
##
## FIXME: Generate and store in NixOps4's state.
mastodonS3KeyConfig =
{ pkgs, ... }:
{
s3AccessKeyFile = pkgs.writeText "s3AccessKey" "GK3515373e4c851ebaad366558";
s3SecretKeyFile = pkgs.writeText "s3SecretKey" "7d37d093435a41f2aab8f13c19ba067d9776c90215f56614adad6ece597dbb34";
};
peertubeS3KeyConfig =
{ pkgs, ... }:
{
s3AccessKeyFile = pkgs.writeText "s3AccessKey" "GK1f9feea9960f6f95ff404c9b";
s3SecretKeyFile = pkgs.writeText "s3SecretKey" "7295c4201966a02c2c3d25b5cea4a5ff782966a2415e3a196f91924631191395";
};
pixelfedS3KeyConfig =
{ pkgs, ... }:
{
s3AccessKeyFile = pkgs.writeText "s3AccessKey" "GKb5615457d44214411e673b7b";
s3SecretKeyFile = pkgs.writeText "s3SecretKey" "5be6799a88ca9b9d813d1a806b64f15efa49482dbe15339ddfaf7f19cf434987";
};
makeConfigurationResource = resourceModule: config: {
type = providers.local.exec;
imports = [
nixops4-nixos.modules.nixops4Resource.nixos
resourceModule
{
## NOTE: With NixOps4, there are several levels and all of them live
## in the NixOS module system:
##
## 1. Each NixOps4 deployment is a module.
## 2. Each NixOps4 resource is a module. This very comment is
## inside an attrset imported as a module in a resource.
## 3. Each NixOps4 'configuration' resource contains an attribute
## 'nixos.module', itself a NixOS configuration module.
nixos.module =
{ ... }:
{
imports = [
config
fediversity
];
};
}
];
};
in
{
garage-configuration = makeConfigurationResource garageConfigurationResource (
{ pkgs, ... }:
mkIf (panelConfig.mastodon.enable || panelConfig.peertube.enable || panelConfig.pixelfed.enable) {
fediversity = {
inherit (panelConfig) domain;
garage.enable = true;
pixelfed = pixelfedS3KeyConfig { inherit pkgs; };
mastodon = mastodonS3KeyConfig { inherit pkgs; };
peertube = peertubeS3KeyConfig { inherit pkgs; };
};
}
);
mastodon-configuration = makeConfigurationResource mastodonConfigurationResource (
{ pkgs, ... }:
mkIf panelConfig.mastodon.enable {
fediversity = {
inherit (panelConfig) domain;
temp.initialUser = {
inherit (panelConfig.initialUser) username email displayName;
# FIXME: disgusting, but nvm, this is going to be replaced by
# proper central authentication at some point
passwordFile = pkgs.writeText "password" panelConfig.initialUser.password;
};
mastodon = mastodonS3KeyConfig { inherit pkgs; } // {
enable = true;
};
temp.cores = 1; # FIXME: should come from NixOps4 eventually
};
}
);
peertube-configuration = makeConfigurationResource peertubeConfigurationResource (
{ pkgs, ... }:
mkIf panelConfig.peertube.enable {
fediversity = {
inherit (panelConfig) domain;
temp.initialUser = {
inherit (panelConfig.initialUser) username email displayName;
# FIXME: disgusting, but nvm, this is going to be replaced by
# proper central authentication at some point
passwordFile = pkgs.writeText "password" panelConfig.initialUser.password;
};
peertube = peertubeS3KeyConfig { inherit pkgs; } // {
enable = true;
## NOTE: Only ever used for testing anyway.
##
## FIXME: Generate and store in NixOps4's state.
secretsFile = pkgs.writeText "secret" "574e093907d1157ac0f8e760a6deb1035402003af5763135bae9cbd6abe32b24";
};
};
}
);
pixelfed-configuration = makeConfigurationResource pixelfedConfigurationResource (
{ pkgs, ... }:
mkIf panelConfig.pixelfed.enable {
fediversity = {
inherit (panelConfig) domain;
temp.initialUser = {
inherit (panelConfig.initialUser) username email displayName;
# FIXME: disgusting, but nvm, this is going to be replaced by
# proper central authentication at some point
passwordFile = pkgs.writeText "password" panelConfig.initialUser.password;
};
pixelfed = pixelfedS3KeyConfig { inherit pkgs; } // {
enable = true;
};
};
}
);
};
}

1410
flake.lock generated

File diff suppressed because it is too large Load diff

View file

@ -1,73 +0,0 @@
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-24.11";
flake-parts.url = "github:hercules-ci/flake-parts";
git-hooks.url = "github:cachix/git-hooks.nix";
agenix.url = "github:ryantm/agenix";
disko.url = "github:nix-community/disko";
nixops4.url = "github:nixops4/nixops4";
nixops4-nixos.url = "github:nixops4/nixops4-nixos";
};
outputs =
inputs@{ flake-parts, ... }:
flake-parts.lib.mkFlake { inherit inputs; } {
systems = [
"x86_64-linux"
"aarch64-linux"
"x86_64-darwin"
"aarch64-darwin"
];
imports = [
inputs.git-hooks.flakeModule
inputs.nixops4.modules.flake.default
./infra/flake-part.nix
./services/flake-part.nix
];
perSystem =
{
config,
pkgs,
lib,
inputs',
...
}:
{
formatter = pkgs.nixfmt-rfc-style;
pre-commit.settings.hooks =
let
## Add a directory here if pre-commit hooks shouldn't apply to it.
optout = [ "npins" ];
excludes = map (dir: "^${dir}/") optout;
addExcludes = lib.mapAttrs (_: c: c // { inherit excludes; });
in
addExcludes {
nixfmt-rfc-style.enable = true;
deadnix.enable = true;
trim-trailing-whitespace.enable = true;
shellcheck.enable = true;
};
devShells.default = pkgs.mkShell {
packages = [
pkgs.nil
inputs'.agenix.packages.default
pkgs.openssh
pkgs.httpie
pkgs.jq
# exposing this env var as a hack to pass info in from form
(inputs'.nixops4.packages.default.overrideAttrs {
impureEnvVars = [ "DEPLOYMENT" ];
})
];
shellHook = config.pre-commit.installationScript;
};
};
};
}

10
infra/.envrc Normal file
View file

@ -0,0 +1,10 @@
#!/usr/bin/env bash
# the shebang is ignored, but nice for editors
# shellcheck shell=bash
if type -P lorri &>/dev/null; then
eval "$(lorri direnv)"
else
echo 'while direnv evaluated .envrc, could not find the command "lorri" [https://github.com/nix-community/lorri]'
use_nix
fi

View file

@ -1,98 +1,30 @@
# Infra
# service deployment
This directory contains the definition of [the VMs](machines.md) that host our
infrastructure.
deploys [NixOS](https://nixos.org/) templates using [OpenTofu](https://opentofu.org/).
## Provisioning VMs with an initial configuration
## requirements
NOTE[Niols]: This is very manual and clunky. Two things will happen. In the near
future, I will improve the provisioning script to make this a bit less clunky.
In the far future, NixOps4 will be able to communicate with Proxmox directly and
everything will become much cleaner.
- [nix](https://nix.dev/)
1. Choose names for your VMs. It is recommended to choose `fediXXX`, with `XXX`
above 100. For instance, `fedi117`.
## usage
2. Add a basic configuration for the machine. These typically go in
`infra/machines/<name>/default.nix`. You can look at other `fediXXX` VMs to
find inspiration. You probably do not need a `nixos.module` option at this
point.
### development
2. Add a file for each of those VM's public keys, eg.
```
touch keys/systems/fedi117.pub
```
Those files need to exist during provisioning, but their content matters only
when updating the machines' configuration.
FIXME: Remove this step by making the provisioning script not fail with the
public key does not exist yet.
3. Run the provisioning script:
```
sh infra/proxmox-provision.sh fedi117
```
The script can take several ids at the same time. It requires some
authentication options and provides several more. See `--help`.
4. (Optional) Add a DNS entry for the machine; for instance `fedi117.abundos.eu
A 95.215.187.117`.
5. Grab the public host keys for the machines in question, and add it to the
repository. For instance:
```
ssh fedi117.abundos.eu 'sudo cat /etc/ssh/ssh_host_ed25519_key.pub' > keys/systems/fedi117.pub
```
FIXME: Make the provisioning script do that for us.
7. Regenerate the list of machines:
```
sh infra/machines.md.sh
```
Commit it with the machine's configuration, public key, etc.
8. At this point, the machine contains a very basic configuration that contains
just enough for it to boot and be reachable. Go on to the next section to
update the machine and put an actual configuration.
FIXME: Figure out why the full configuration isn't on the machine at this
point and fix it.
## Updating existing VM configurations
Their configuration can be updated via NixOps4. Run
before using other commands, if not using direnv:
```sh
nixops4 deployments list
nix-shell
```
to see the available deployments.
This should be done from the root of the repository,
otherwise NixOps4 will fail with something like:
```
nixops4 error: evaluation: error:
… while calling the 'getFlake' builtin
error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist, evaluation: error:
… while calling the 'getFlake' builtin
error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist
```
Then, given a deployment (eg. `fedi200`), run
then to initialize, or after updating pins or TF providers:
```sh
nixops4 apply <deployment>
setup
```
Alternatively, to run the `default` deployment, which contains all the VMs, run
then, one can use the `tofu` CLI in the sub-folders.
```sh
nixops4 apply
```
## implementing
## Removing an existing VM
See `infra/proxmox-remove.sh --help`.
proper documentation TODO.
until then, a reference implementation may be found in [`panel/`](https://git.fediversity.eu/Fediversity/Fediversity/src/branch/main/panel).

View file

@ -2,7 +2,6 @@
let
inherit (lib) mkDefault;
in
{
imports = [

View file

@ -93,7 +93,7 @@ in
description = ''
The IP address of the machine, version 4. It will be injected as a
value in `networking.interfaces.eth0`, but it will also be used to
communicate with the machine via NixOps4.
communicate with the machine.
'';
};
@ -118,7 +118,7 @@ in
description = ''
The IP address of the machine, version 6. It will be injected as a
value in `networking.interfaces.eth0`, but it will also be used to
communicate with the machine via NixOps4.
communicate with the machine.
'';
};
@ -141,7 +141,7 @@ in
hostPublicKey = mkOption {
description = ''
The ed25519 host public key of the machine. It is used to filter Age
secrets and only keep the relevant ones, and to feed to NixOps4.
secrets and only keep the relevant ones, and to feed to TF.
'';
};

View file

@ -1,5 +1,4 @@
{
inputs,
lib,
config,
...
@ -16,48 +15,27 @@ let
in
{
imports = [ ./options.nix ];
fediversityVm.hostPublicKey = mkDefault keys.systems.${config.fediversityVm.name};
ssh = {
host = config.fediversityVm.ipv4.address;
hostPublicKey = config.fediversityVm.hostPublicKey;
};
nixpkgs = inputs.nixpkgs;
## The configuration of the machine. We strive to keep in this file only the
## options that really need to be injected from the resource. Everything else
## should go into the `./nixos` subdirectory.
nixos.module = {
imports = [
inputs.agenix.nixosModules.default
inputs.disko.nixosModules.default
./options.nix
./nixos
];
imports = [
./options.nix
./nixos
];
## Inject the shared options from the resource's `config` into the NixOS
## configuration.
fediversityVm = config.fediversityVm;
## Read all the secrets, filter the ones that are supposed to be readable
## with this host's public key, and add them correctly to the configuration
## as `age.secrets.<name>.file`.
age.secrets = concatMapAttrs (
name: secret:
optionalAttrs (elem config.fediversityVm.hostPublicKey secret.publicKeys) {
${removeSuffix ".age" name}.file = secretsPrefix + "/${name}";
}
) secrets;
## Read all the secrets, filter the ones that are supposed to be readable
## with this host's public key, and add them correctly to the configuration
## as `age.secrets.<name>.file`.
age.secrets = concatMapAttrs (
name: secret:
optionalAttrs (elem config.fediversityVm.hostPublicKey secret.publicKeys) ({
${removeSuffix ".age" name}.file = secretsPrefix + "/${name}";
})
) secrets;
## FIXME: Remove direct root authentication once the NixOps4 NixOS provider
## supports users with password-less sudo.
users.users.root.openssh.authorizedKeys.keys = attrValues keys.contributors ++ [
# allow our panel vm access to the test machines
keys.panel
];
};
## FIXME: Remove direct root authentication once the NixOps4 NixOS provider
## supports users with password-less sudo.
users.users.root.openssh.authorizedKeys.keys = attrValues keys.contributors;
}

26
infra/common/shared.nix Normal file
View file

@ -0,0 +1,26 @@
{
pkgs,
config,
...
}:
let
inherit (config.terraform) hostname domain initialUser;
in
{
imports = [
<disko/module.nix>
<agenix/modules/age.nix>
../../services/fediversity
./resource.nix
];
fediversityVm.name = hostname;
fediversity = {
inherit domain;
temp.initialUser = {
inherit (initialUser) username email displayName;
# FIXME: disgusting, but nvm, this is going to be replaced by
# proper central authentication at some point
passwordFile = pkgs.writeText "password" initialUser.password;
};
};
}

29
infra/default.nix Normal file
View file

@ -0,0 +1,29 @@
{
system ? builtins.currentSystem,
sources ? import ../npins,
pkgs ? import sources.nixpkgs { inherit system; },
}:
let
inherit (pkgs) lib;
setup = import ./setup.nix { inherit lib pkgs sources; };
in
{
# shell for testing TF directly
shell = pkgs.mkShellNoCC {
packages = [
(import ./tf.nix { inherit lib pkgs; })
pkgs.jaq
setup
];
};
tests = pkgs.callPackage ./tests.nix { };
# re-export inputs so they can be overridden granularly
# (they can't be accessed from the outside any other way)
inherit
sources
system
pkgs
;
}

44
infra/dev/main.tf Normal file
View file

@ -0,0 +1,44 @@
locals {
vm_domain = "abundos.eu"
}
module "nixos" {
source = "../sync-nix"
vm_domain = local.vm_domain
hostname = each.value.hostname
config_nix = each.value.config_nix
config_tf = each.value.config_tf
for_each = { for name, inst in {
# wiki = "vm02187" # does not resolve
# forgejo = "vm02116" # does not resolve
# TODO: move these to a separate `host` dir
dns = "fedi200"
fedipanel = "fedi201"
} : name => {
hostname = inst
config_tf = {
terraform = {
domain = local.vm_domain
hostname = inst
}
}
config_nix = <<-EOF
{
# note interpolations here TF ones
imports = [
# shared NixOS config
${path.root}/../common/shared.nix
# FIXME: separate template options by service
${path.root}/options.nix
# for service `forgejo` import `forgejo.nix`
${path.root}/../../machines/dev/${inst}/${name}.nix
# FIXME: get VM details from TF
${path.root}/../../machines/dev/${inst}
];
}
EOF
}
}
}

28
infra/dev/options.nix Normal file
View file

@ -0,0 +1,28 @@
# TODO: could (part of) this be generated somehow? c.f #275
{
lib,
...
}:
let
inherit (lib) types mkOption;
inherit (types) str enum;
in
{
options.terraform = {
domain = mkOption {
type = enum [
"fediversity.net"
];
description = ''
Apex domain under which the services will be deployed.
'';
default = "fediversity.net";
};
hostname = mkOption {
type = str;
description = ''
Internal name of the host, e.g. test01
'';
};
};
}

1
infra/dev/variables.tf Normal file
View file

@ -0,0 +1 @@

View file

@ -1,180 +0,0 @@
{
self,
inputs,
lib,
...
}:
let
inherit (builtins) readDir readFile fromJSON;
inherit (lib)
attrNames
mkOption
evalModules
filterAttrs
;
inherit (lib.attrsets) genAttrs;
## Given a machine's name and whether it is a test VM, make a resource module,
## except for its missing provider. (Depending on the use of that resource, we
## will provide a different one.)
makeResourceModule =
{ vmName, isTestVm }:
{
_module.args = { inherit inputs; };
imports =
[
./common/resource.nix
]
++ (
if isTestVm then
[
./test-machines/${vmName}
{
nixos.module.users.users.root.openssh.authorizedKeys.keys = [
# allow our panel vm access to the test machines
(import ../keys).panel
];
}
]
else
[
./machines/${vmName}
]
);
fediversityVm.name = vmName;
};
## Given a list of machine names, make a deployment with those machines'
## configurations as resources.
makeDeployment =
vmNames:
{ providers, ... }:
{
providers.local = inputs.nixops4.modules.nixops4Provider.local;
resources = genAttrs vmNames (vmName: {
type = providers.local.exec;
imports = [
inputs.nixops4-nixos.modules.nixops4Resource.nixos
(makeResourceModule {
inherit vmName;
isTestVm = false;
})
];
});
};
makeDeployment' = vmName: makeDeployment [ vmName ];
## Given an attrset of test configurations (key = test machine name, value =
## NixOS configuration module), make a deployment with those machines'
## configurations as resources.
makeTestDeployment =
(import ../deployment)
{
inherit lib;
inherit (inputs) nixops4 nixops4-nixos;
inherit (self.nixosModules) fediversity;
}
{
garageConfigurationResource = makeResourceModule {
vmName = "test01";
isTestVm = true;
};
mastodonConfigurationResource = makeResourceModule {
vmName = "test06"; # somehow `test02` has a problem - use test06 instead
isTestVm = true;
};
peertubeConfigurationResource = makeResourceModule {
vmName = "test05";
isTestVm = true;
};
pixelfedConfigurationResource = makeResourceModule {
vmName = "test04";
isTestVm = true;
};
};
nixops4ResourceNixosMockOptions = {
## NOTE: We allow the use of a few options from
## `inputs.nixops4-nixos.modules.nixops4Resource.nixos` such that we can
## reuse modules that make use of them.
##
## REVIEW: We can probably do much better and cleaner. On the other hand,
## this is only needed to expose NixOS configurations for provisioning
## purposes, and eventually all of this should be handled by NixOps4.
options = {
nixos.module = mkOption { }; # NOTE: not just `nixos` otherwise merging will go wrong
nixpkgs = mkOption { };
ssh = mkOption { };
};
};
makeResourceConfig =
vm:
(evalModules {
modules = [
nixops4ResourceNixosMockOptions
(makeResourceModule vm)
];
}).config;
## Given a VM name, make a NixOS configuration for this machine.
makeConfiguration =
isTestVm: vmName:
inputs.nixpkgs.lib.nixosSystem {
modules = [
(makeResourceConfig { inherit vmName isTestVm; }).nixos.module
];
};
makeVmOptions = isTestVm: vmName: {
inherit ((makeResourceConfig { inherit vmName isTestVm; }).fediversityVm)
proxmox
vmId
description
sockets
cores
memory
diskSize
hostPublicKey
unsafeHostPrivateKey
;
};
listSubdirectories = path: attrNames (filterAttrs (_: type: type == "directory") (readDir path));
machines = listSubdirectories ./machines;
testMachines = listSubdirectories ./test-machines;
in
{
flake.lib.makeInstallerIso = import ./makeInstallerIso.nix;
## - Each normal or test machine gets a NixOS configuration.
## - Each normal or test machine gets a VM options entry.
## - Each normal machine gets a deployment.
## - We add a “default” deployment with all normal machines.
## - We add a “test” deployment with all test machines.
nixops4Deployments = genAttrs machines makeDeployment' // {
default = makeDeployment machines;
test = makeTestDeployment (
fromJSON (
let
env = builtins.getEnv "DEPLOYMENT";
in
if env != "" then
env
else
builtins.trace "env var DEPLOYMENT not set, falling back to ./test-machines/configuration.json!" (readFile ./test-machines/configuration.json)
)
);
};
flake.nixosConfigurations =
genAttrs machines (makeConfiguration false)
// genAttrs testMachines (makeConfiguration true);
flake.vmOptions =
genAttrs machines (makeVmOptions false)
// genAttrs testMachines (makeVmOptions true);
}

View file

@ -1,15 +0,0 @@
<!-- This file is auto-generated by `machines.md.sh` from the machines'
configuration. -->
# Machines
Currently, this repository keeps track of the following VMs:
Machine | Proxmox | Description
--------|---------|-------------
[`fedi200`](./fedi200) | fediversity | Testing machine for Hans
[`fedi201`](./fedi201) | fediversity | FediPanel
[`vm02116`](./vm02116) | procolix | Forgejo
[`vm02187`](./vm02187) | procolix | Wiki
This table excludes all machines with names starting with `test`.

View file

@ -1,43 +0,0 @@
#!/usr/bin/env sh
set -euC
cd "$(dirname "$0")"
{
cat <<\EOF
<!-- This file is auto-generated by `machines.md.sh` from the machines'
configuration. -->
# Machines
Currently, this repository keeps track of the following VMs:
Machine | Proxmox | Description
--------|---------|-------------
EOF
vmOptions=$(
cd ..
nix eval \
--impure --raw --expr "
builtins.toJSON (builtins.getFlake (builtins.toString ./.)).vmOptions
" \
--log-format raw --quiet
)
## NOTE: `jq`'s `keys` is alphabetically sorted, just what we want here.
for machine in $(echo "$vmOptions" | jq -r 'keys[]'); do
if [ "${machine#test}" = "$machine" ]; then
proxmox=$(echo "$vmOptions" | jq -r ".$machine.proxmox")
description=$(echo "$vmOptions" | jq -r ".$machine.description" | head -n 1)
# shellcheck disable=SC2016
printf '[`%s`](./%s) | %s | %s\n' "$machine" "$machine" "$proxmox" "$description"
fi
done
cat <<\EOF
This table excludes all machines with names starting with `test`.
EOF
} >| machines.md

View file

@ -1,38 +0,0 @@
{
fediversityVm = {
vmId = 2116;
proxmox = "procolix";
description = "Forgejo";
ipv4.address = "185.206.232.34";
ipv6.address = "2a00:51c0:12:1201::20";
};
nixos.module =
{ lib, ... }:
{
imports = [
./forgejo.nix
];
## vm02116 is running on old hardware based on a Xen VM environment, so it
## needs these extra options. Once the VM gets moved to a newer node, these
## two options can safely be removed.
boot.initrd.availableKernelModules = [ "xen_blkfront" ];
services.xe-guest-utilities.enable = true;
## NOTE: This VM was created manually, which requires us to override the
## default disko-based `fileSystems` definition.
fileSystems = lib.mkForce {
"/" = {
device = "/dev/disk/by-uuid/3802a66d-e31a-4650-86f3-b51b11918853";
fsType = "ext4";
};
"/boot" = {
device = "/dev/disk/by-uuid/2CE2-1173";
fsType = "vfat";
};
};
};
}

View file

@ -1,36 +0,0 @@
{
fediversityVm = {
vmId = 2187;
proxmox = "procolix";
description = "Wiki";
ipv4.address = "185.206.232.187";
ipv6.address = "2a00:51c0:12:1201::187";
};
nixos.module =
{ lib, ... }:
{
imports = [
./wiki.nix
];
## NOTE: This VM was created manually, which requires us to override the
## default disko-based `fileSystems` definition.
fileSystems = lib.mkForce {
"/" = {
device = "/dev/disk/by-uuid/a46a9c46-e32b-4216-a4aa-8819b2cd0d49";
fsType = "ext4";
};
"/boot" = {
device = "/dev/disk/by-uuid/6AB5-4FA8";
fsType = "vfat";
options = [
"fmask=0022"
"dmask=0022"
];
};
};
};
}

74
infra/operator/main.tf Normal file
View file

@ -0,0 +1,74 @@
locals {
vm_domain = "abundos.eu"
# user-facing applications
application_configs = {
# FIXME: wrap applications at the interface to grab them in one go?
mastodon = {
cfg = var.mastodon
hostname = "test06"
}
pixelfed = {
cfg = var.pixelfed
hostname = "test04"
}
peertube = {
cfg = var.peertube
hostname = "test05"
}
}
# services shared between applications
peripherals = { for name, inst in {
garage = "test01"
} : name => {
hostname = inst
cfg = {
# enable if any user applications are enabled
enable = anytrue([for _, app in local.application_configs: app.cfg.enable])
}
}
}
}
module "nixos" {
source = "../sync-nix"
vm_domain = local.vm_domain
hostname = each.value.hostname
config_nix = each.value.config_nix
config_tf = each.value.config_tf
for_each = {for name, inst in merge(
local.peripherals,
local.application_configs,
) : name => merge(inst, {
config_tf = {
terraform = {
domain = var.domain
hostname = inst.hostname
initialUser = var.initialUser
}
}
config_nix = <<-EOF
{
# note interpolations here TF ones
imports = [
# shared NixOS config
${path.root}/../common/shared.nix
# FIXME: separate template options by service
${path.root}/options.nix
# for service `mastodon` import `mastodon.nix`
${path.root}/../../machines/operator/${inst.hostname}/${name}.nix
# FIXME: get VM details from TF
${path.root}/../../machines/operator/${inst.hostname}
];
## FIXME: switch root authentication to users with password-less sudo, see #24
users.users.root.openssh.authorizedKeys.keys = let
keys = import ../../keys;
in [
# allow our panel vm access to the test machines
keys.panel
];
}
EOF
}) if inst.cfg.enable}
}

View file

@ -0,0 +1,54 @@
# TODO: could (part of) this be generated somehow? c.f #275
{
lib,
...
}:
let
inherit (lib) types mkOption;
inherit (types) str enum submodule;
in
{
options.terraform = {
domain = mkOption {
type = enum [
"fediversity.net"
];
description = ''
Apex domain under which the services will be deployed.
'';
default = "fediversity.net";
};
hostname = mkOption {
type = str;
description = ''
Internal name of the host, e.g. test01
'';
};
initialUser = mkOption {
description = ''
Some services require an initial user to access them.
This option sets the credentials for such an initial user.
'';
type = submodule {
options = {
displayName = mkOption {
type = str;
description = "Display name of the user";
};
username = mkOption {
type = str;
description = "Username for login";
};
email = mkOption {
type = str;
description = "User's email address";
};
password = mkOption {
type = str;
description = "Password for login";
};
};
};
};
};
}

View file

@ -0,0 +1,51 @@
# TODO: (partially) generate, say from nix modules, c.f. #275
variable "domain" {
type = string
default = "fediversity.net"
}
variable "mastodon" {
type = object({
enable = bool
})
default = {
enable = false
}
}
variable "pixelfed" {
type = object({
enable = bool
})
default = {
enable = false
}
}
variable "peertube" {
type = object({
enable = bool
})
default = {
enable = false
}
}
variable "initialUser" {
type = object({
displayName = string
username = string
email = string
# TODO: mark (nested) credentials as sensitive
# https://discuss.hashicorp.com/t/is-it-possible-to-mark-an-attribute-of-an-object-as-sensitive/24649/2
password = string
})
# FIXME: remove default when the form provides this value, see #285
default = {
displayName = "Testy McTestface"
username = "test"
email = "test@test.com"
password = "testtest"
}
}

20
infra/setup.nix Normal file
View file

@ -0,0 +1,20 @@
{
pkgs,
lib,
sources,
...
}:
pkgs.writeScriptBin "setup" ''
# calculated pins
echo '${lib.strings.toJSON sources}' > sync-nix/.npins.json
# generate TF lock for nix's TF providers
for category in dev operator; do
pushd "$category"
rm -rf .terraform/
rm -f .terraform.lock.hcl
# suppress warning on architecture-specific generated lock file:
# `Warning: Incomplete lock file information for providers`.
tofu init -input=false 1>/dev/null
popd
done
''

1
infra/shell.nix Normal file
View file

@ -0,0 +1 @@
(import ./. { }).shell

102
infra/sync-nix/main.tf Normal file
View file

@ -0,0 +1,102 @@
locals {
system = "x86_64-linux"
# dependency paths pre-calculated from npins
pins = jsondecode(file("${path.module}/.npins.json"))
# nix path: expose pins, use nixpkgs in flake commands (`nix run`)
nix_path = "${join(":", [for name, dir in local.pins : "${name}=${dir}"])}:flake=${local.pins["nixpkgs"]}:flake"
}
# hash of our code directory, used to trigger re-deploy
# FIXME calculate separately to reduce false positives
data "external" "hash" {
program = ["sh", "-c", "echo \"{\\\"hash\\\":\\\"$(nix-hash ..)\\\"}\""]
}
# TF resource to build and deploy NixOS instances.
resource "terraform_data" "nixos" {
# trigger rebuild/deploy if (FIXME?) any potentially used config/code changed,
# preventing these (20+s, build being bottleneck) when nothing changed.
# terraform-nixos separates these to only deploy if instantiate changed,
# yet building even then - which may be not as bad using deploy on remote.
# having build/deploy one resource reflects wanting to prevent no-op rebuilds
# over preventing (with less false positives) no-op deployments,
# as i could not find a way to do prevent no-op rebuilds without merging them:
# - generic resources cannot have outputs, while we want info from the instantiation (unless built on host?).
# - `data` always runs, which is slow for deploy and especially build.
triggers_replace = [
data.external.hash.result,
var.hostname,
var.config_nix,
var.config_tf,
]
provisioner "local-exec" {
# directory to run the script from. we use the TF project root dir,
# here as a path relative from where TF is run from,
# matching calling modules' expectations on config_nix locations.
# note that absolute paths can cause false positives in triggers,
# so are generally discouraged in TF.
working_dir = path.root
environment = {
# nix path used on build, lets us refer to e.g. nixpkgs like `<nixpkgs>`
NIX_PATH = local.nix_path
}
# TODO: refactor back to command="ignoreme" interpreter=concat([]) to protect sensitive data from error logs?
# TODO: build on target?
command = <<-EOF
set -euo pipefail
# INSTANTIATE
command=(
nix-instantiate
--expr
'let
os = import <nixpkgs/nixos> {
system = "${local.system}";
configuration =
${var.config_nix} //
# template parameters passed in from TF thru json
builtins.fromJSON "${replace(jsonencode(var.config_tf), "\"", "\\\"")}" //
{
# nix path for debugging
nix.nixPath = [ "${local.nix_path}" ];
};
};
in
# info we want to get back out
{
substituters = builtins.concatStringsSep " " os.config.nix.settings.substituters;
trusted_public_keys = builtins.concatStringsSep " " os.config.nix.settings.trusted-public-keys;
drv_path = os.config.system.build.toplevel.drvPath;
out_path = os.config.system.build.toplevel;
}'
)
# instantiate the config in /nix/store
"$${command[@]}" -A out_path
# get the other info
json="$("$${command[@]}" --eval --strict --json)"
# DEPLOY
declare substituters trusted_public_keys drv_path
# set our variables using the json object
eval "export $(echo $json | jaq -r 'to_entries | map("\(.key)=\(.value)") | @sh')"
host="root@${var.hostname}.${var.vm_domain}" # FIXME: #24
buildArgs=(
--option extra-binary-caches https://cache.nixos.org/
--option substituters $substituters
--option trusted-public-keys $trusted_public_keys
)
sshOpts=(
-o BatchMode=yes
-o StrictHostKeyChecking=no
)
# get the realized derivation to deploy
outPath=$(nix-store --realize "$drv_path" "$${buildArgs[@]}")
# deploy the config by nix-copy-closure
NIX_SSHOPTS="$${sshOpts[*]}" nix-copy-closure --to "$host" "$outPath" --gzip --use-substitutes
# switch the remote host to the config
ssh "$${sshOpts[@]}" "$host" "nix-env --profile /nix/var/nix/profiles/system --set $outPath; $outPath/bin/switch-to-configuration switch"
EOF
}
}

View file

@ -0,0 +1,17 @@
variable "vm_domain" {
type = string
}
variable "hostname" {
type = string
}
variable "config_nix" {
type = string
default = "{}"
}
variable "config_tf" {
type = map(any)
default = {}
}

37
infra/tests.nix Normal file
View file

@ -0,0 +1,37 @@
{ lib, pkgs }:
let
defaults = {
virtualisation = {
memorySize = 2048;
cores = 2;
};
};
tf = pkgs.callPackage ./tf.nix {
inherit lib pkgs;
};
tfEnv = pkgs.callPackage ./tf-env.nix { };
nodes = {
server = {
environment.systemPackages = [
tf
tfEnv
];
};
};
in
lib.mapAttrs (name: test: pkgs.testers.runNixOSTest (test // { inherit name; })) {
tf-validate-dev = {
inherit defaults nodes;
testScript = ''
server.wait_for_unit("multi-user.target")
server.succeed("${lib.getExe tf} -chdir='${tfEnv}/infra/dev' validate")
'';
};
tf-validate-operator = {
inherit defaults nodes;
testScript = ''
server.wait_for_unit("multi-user.target")
server.succeed("${lib.getExe tf} -chdir='${tfEnv}/infra/operator' validate")
'';
};
}

32
infra/tf-env.nix Normal file
View file

@ -0,0 +1,32 @@
{
lib,
pkgs,
sources ? import ../npins,
...
}:
pkgs.stdenv.mkDerivation {
name = "tf-repo";
src =
with lib.fileset;
toSource {
root = ../.;
# don't copy ignored files
fileset = intersection (gitTracked ../.) ../.;
};
buildInputs = [
(import ./tf.nix { inherit lib pkgs; })
(import ./setup.nix { inherit lib pkgs sources; })
];
buildPhase = ''
runHook preBuild
pushd infra
setup
popd
runHook postBuild
'';
installPhase = ''
runHook preInstall
cp -r . $out
runHook postInstall
'';
}

24
infra/tf.nix Normal file
View file

@ -0,0 +1,24 @@
# FIXME: use overlays so this gets imported just once?
{
lib,
pkgs,
...
}:
let
tofuProvider =
provider:
provider.override (oldArgs: {
provider-source-address =
lib.replaceStrings [ "https://registry.terraform.io/providers" ] [ "registry.opentofu.org" ]
oldArgs.homepage;
});
tf = pkgs.opentofu;
tfPlugins = (
p: [
p.external
]
);
in
# tf.withPlugins tfPlugins
# https://github.com/NixOS/nixpkgs/pull/358522
tf.withPlugins (p: pkgs.lib.lists.map tofuProvider (tfPlugins p))

View file

@ -14,7 +14,7 @@ overwrite a secret without knowing its contents.)
In infra management, the systems' keys are used for security reasons; they
identify the machine that we are talking to. The contributor keys are used to
give access to the `root` user on these machines, which allows, among other
things, to deploy their configurations with NixOps4.
things, to deploy their configurations.
## Adding a contributor

4
machines/README.md Normal file
View file

@ -0,0 +1,4 @@
# Machines
This directory contains the definition of [the VMs](machines.md) that host our
infrastructure.

View file

@ -0,0 +1,2 @@
_: {
}

View file

@ -14,10 +14,4 @@
gateway = "2a00:51c0:13:1305::1";
};
};
nixos.module = {
imports = [
./fedipanel.nix
];
};
}

View file

@ -7,12 +7,12 @@ let
in
{
imports = [
<home-manager/nixos>
(import ../../../panel { }).module
];
security.acme = {
acceptTerms = true;
defaults.email = "beheer@procolix.com";
};
age.secrets.panel-ssh-key = {
@ -37,6 +37,24 @@ in
enable = true;
production = true;
domain = "demo.fediversity.eu";
# FIXME: make it work without this duplication
settings =
let
cfg = config.services.${name};
in
{
STATIC_ROOT = "/var/lib/${name}/static";
DEBUG = false;
ALLOWED_HOSTS = [
cfg.domain
cfg.host
"localhost"
"[::1]"
];
CSRF_TRUSTED_ORIGINS = [ "https://${cfg.domain}" ];
COMPRESS_OFFLINE = true;
LIBSASS_OUTPUT_STYLE = "compressed";
};
secrets = {
SECRET_KEY = config.age.secrets.panel-secret-key.path;
};

View file

@ -0,0 +1,31 @@
{ lib, ... }:
{
fediversityVm = {
vmId = 2116;
proxmox = "procolix";
description = "Forgejo";
ipv4.address = "185.206.232.34";
ipv6.address = "2a00:51c0:12:1201::20";
};
## vm02116 is running on old hardware based on a Xen VM environment, so it
## needs these extra options. Once the VM gets moved to a newer node, these
## two options can safely be removed.
boot.initrd.availableKernelModules = [ "xen_blkfront" ];
services.xe-guest-utilities.enable = true;
## NOTE: This VM was created manually, which requires us to override the
## default disko-based `fileSystems` definition.
fileSystems = lib.mkForce {
"/" = {
device = "/dev/disk/by-uuid/3802a66d-e31a-4650-86f3-b51b11918853";
fsType = "ext4";
};
"/boot" = {
device = "/dev/disk/by-uuid/2CE2-1173";
fsType = "vfat";
};
};
}

View file

@ -0,0 +1,29 @@
{ lib, ... }:
{
fediversityVm = {
vmId = 2187;
proxmox = "procolix";
description = "Wiki";
ipv4.address = "185.206.232.187";
ipv6.address = "2a00:51c0:12:1201::187";
};
## NOTE: This VM was created manually, which requires us to override the
## default disko-based `fileSystems` definition.
fileSystems = lib.mkForce {
"/" = {
device = "/dev/disk/by-uuid/a46a9c46-e32b-4216-a4aa-8819b2cd0d49";
fsType = "ext4";
};
"/boot" = {
device = "/dev/disk/by-uuid/6AB5-4FA8";
fsType = "vfat";
options = [
"fmask=0022"
"dmask=0022"
];
};
};
}

View file

@ -0,0 +1,34 @@
{ pkgs, ... }:
let
## NOTE: All of these secrets are publicly available in this source file
## and will end up in the Nix store. We don't care as they are only ever
## used for testing anyway.
##
## FIXME: Generate and store in state.
mastodonS3KeyConfig =
{ pkgs, ... }:
{
s3AccessKeyFile = pkgs.writeText "s3AccessKey" "GK3515373e4c851ebaad366558";
s3SecretKeyFile = pkgs.writeText "s3SecretKey" "7d37d093435a41f2aab8f13c19ba067d9776c90215f56614adad6ece597dbb34";
};
peertubeS3KeyConfig =
{ pkgs, ... }:
{
s3AccessKeyFile = pkgs.writeText "s3AccessKey" "GK1f9feea9960f6f95ff404c9b";
s3SecretKeyFile = pkgs.writeText "s3SecretKey" "7295c4201966a02c2c3d25b5cea4a5ff782966a2415e3a196f91924631191395";
};
pixelfedS3KeyConfig =
{ pkgs, ... }:
{
s3AccessKeyFile = pkgs.writeText "s3AccessKey" "GKb5615457d44214411e673b7b";
s3SecretKeyFile = pkgs.writeText "s3SecretKey" "5be6799a88ca9b9d813d1a806b64f15efa49482dbe15339ddfaf7f19cf434987";
};
in
{
fediversity = {
garage.enable = true;
pixelfed = pixelfedS3KeyConfig { inherit pkgs; };
mastodon = mastodonS3KeyConfig { inherit pkgs; };
peertube = peertubeS3KeyConfig { inherit pkgs; };
};
}

View file

@ -0,0 +1,16 @@
{ pkgs, ... }:
let
pixelfedS3KeyConfig =
{ pkgs, ... }:
{
s3AccessKeyFile = pkgs.writeText "s3AccessKey" "GKb5615457d44214411e673b7b";
s3SecretKeyFile = pkgs.writeText "s3SecretKey" "5be6799a88ca9b9d813d1a806b64f15efa49482dbe15339ddfaf7f19cf434987";
};
in
{
fediversity = {
pixelfed = pixelfedS3KeyConfig { inherit pkgs; } // {
enable = true;
};
};
}

View file

@ -0,0 +1,20 @@
{ pkgs, ... }:
let
peertubeS3KeyConfig =
{ pkgs, ... }:
{
s3AccessKeyFile = pkgs.writeText "s3AccessKey" "GK1f9feea9960f6f95ff404c9b";
s3SecretKeyFile = pkgs.writeText "s3SecretKey" "7295c4201966a02c2c3d25b5cea4a5ff782966a2415e3a196f91924631191395";
};
in
{
fediversity = {
peertube = peertubeS3KeyConfig { inherit pkgs; } // {
enable = true;
## NOTE: Only ever used for testing anyway.
##
## FIXME: Generate and store in state.
secretsFile = pkgs.writeText "secret" "574e093907d1157ac0f8e760a6deb1035402003af5763135bae9cbd6abe32b24";
};
};
}

View file

@ -0,0 +1,17 @@
{ pkgs, ... }:
let
mastodonS3KeyConfig =
{ pkgs, ... }:
{
s3AccessKeyFile = pkgs.writeText "s3AccessKey" "GK3515373e4c851ebaad366558";
s3SecretKeyFile = pkgs.writeText "s3SecretKey" "7d37d093435a41f2aab8f13c19ba067d9776c90215f56614adad6ece597dbb34";
};
in
{
fediversity = {
mastodon = mastodonS3KeyConfig { inherit pkgs; } // {
enable = true;
};
temp.cores = 1; # FIXME: should come from TF eventually
};
}

View file

@ -1,5 +1,86 @@
{
"pins": {
"agenix": {
"type": "Git",
"repository": {
"type": "GitHub",
"owner": "ryantm",
"repo": "agenix"
},
"branch": "main",
"submodules": false,
"revision": "e600439ec4c273cf11e06fe4d9d906fb98fa097c",
"url": "https://github.com/ryantm/agenix/archive/e600439ec4c273cf11e06fe4d9d906fb98fa097c.tar.gz",
"hash": "006ngydiykjgqs85cl19h9klq8kaqm5zs0ng51dnwy7nzgqxzsdr"
},
"disko": {
"type": "GitRelease",
"repository": {
"type": "GitHub",
"owner": "nix-community",
"repo": "disko"
},
"pre_releases": false,
"version_upper_bound": null,
"release_prefix": null,
"submodules": false,
"version": "v1.11.0",
"revision": "cdf8deded8813edfa6e65544f69fdd3a59fa2bb4",
"url": "https://api.github.com/repos/nix-community/disko/tarball/v1.11.0",
"hash": "13brimg7z7k9y36n4jc1pssqyw94nd8qvgfjv53z66lv4xkhin92"
},
"flake-inputs": {
"type": "Git",
"repository": {
"type": "GitHub",
"owner": "fricklerhandwerk",
"repo": "flake-inputs"
},
"branch": "main",
"submodules": false,
"revision": "559574c9cbb8af262f3944b67d60fbf0f6ad03c3",
"url": "https://github.com/fricklerhandwerk/flake-inputs/archive/559574c9cbb8af262f3944b67d60fbf0f6ad03c3.tar.gz",
"hash": "0gbhmp6x2vdzvfnsvqzal3g8f8hx2ia6r73aibc78kazf78m67x6"
},
"git-hooks": {
"type": "Git",
"repository": {
"type": "GitHub",
"owner": "cachix",
"repo": "git-hooks.nix"
},
"branch": "master",
"submodules": false,
"revision": "dcf5072734cb576d2b0c59b2ac44f5050b5eac82",
"url": "https://github.com/cachix/git-hooks.nix/archive/dcf5072734cb576d2b0c59b2ac44f5050b5eac82.tar.gz",
"hash": "1jmdxmx29xghjiaks6f5amnxld8w3kmxb2zv8lk2yzpgp6kr60qg"
},
"gitignore": {
"type": "Git",
"repository": {
"type": "GitHub",
"owner": "hercules-ci",
"repo": "gitignore.nix"
},
"branch": "master",
"submodules": false,
"revision": "637db329424fd7e46cf4185293b9cc8c88c95394",
"url": "https://github.com/hercules-ci/gitignore.nix/archive/637db329424fd7e46cf4185293b9cc8c88c95394.tar.gz",
"hash": "02wxkdpbhlm3yk5mhkhsp3kwakc16xpmsf2baw57nz1dg459qv8w"
},
"home-manager": {
"type": "Git",
"repository": {
"type": "GitHub",
"owner": "nix-community",
"repo": "home-manager"
},
"branch": "master",
"submodules": false,
"revision": "22b326b42bf42973d5e4fe1044591fb459e6aeac",
"url": "https://github.com/nix-community/home-manager/archive/22b326b42bf42973d5e4fe1044591fb459e6aeac.tar.gz",
"hash": "0hwllnym5mrrxinjsq0p9zn39i110c1xixp4x64svl7jjm5zb4c4"
},
"htmx": {
"type": "GitRelease",
"repository": {
@ -30,10 +111,17 @@
"hash": "1wms0wxwvxac1r1daihj5wsx1nghfk5hwdvy5cpgq481bp9x4cjn"
},
"nixpkgs": {
"type": "Channel",
"name": "nixpkgs-unstable",
"url": "https://releases.nixos.org/nixpkgs/nixpkgs-25.05pre782598.18dd725c2960/nixexprs.tar.xz",
"hash": "1p7kgyph7xkj57p19nbxpycmbchc6d9gwdznsmxhymrzyzi3if21"
"type": "Git",
"repository": {
"type": "GitHub",
"owner": "nixos",
"repo": "nixpkgs"
},
"branch": "nixpkgs-unstable",
"submodules": false,
"revision": "f33a4d26226c05d501b9d4d3e5e60a3a59991921",
"url": "https://github.com/nixos/nixpkgs/archive/f33a4d26226c05d501b9d4d3e5e60a3a59991921.tar.gz",
"hash": "1b6dm1sn0bdpcsmxna0zzspjaixa2dald08005fry5jrbjvwafdj"
}
},
"version": 5

View file

@ -20,11 +20,18 @@ in
pkgs.npins
manage
];
env = import ./env.nix { inherit lib pkgs; } // {
NPINS_DIRECTORY = toString ../npins;
CREDENTIALS_DIRECTORY = toString ./.credentials;
DATABASE_URL = "sqlite:///${toString ./src}/db.sqlite3";
};
env =
let
inherit (builtins) toString;
in
import ./env.nix { inherit lib pkgs; }
// {
NPINS_DIRECTORY = toString ../npins;
CREDENTIALS_DIRECTORY = toString ./.credentials;
DATABASE_URL = "sqlite:///${toString ./src}/db.sqlite3";
# locally: use a fixed relative reference, so we can use our newest files without copying to the store
REPO_DIR = toString ../.;
};
shellHook = ''
ln -sf ${sources.htmx}/dist/htmx.js src/panel/static/htmx.min.js
# in production, secrets are passed via CREDENTIALS_DIRECTORY by systemd.

View file

@ -3,16 +3,14 @@
pkgs,
...
}:
let
inherit (builtins) toString;
in
{
REPO_DIR = toString ../.;
# explicitly use nix, as e.g. lix does not have configurable-impure-env
BIN_PATH = lib.makeBinPath [
# explicitly use nix, as e.g. lix does not have configurable-impure-env
pkgs.nix
# nixops error maybe due to our flake git hook: executing 'git': No such file or directory
pkgs.lix
pkgs.bash
pkgs.coreutils
pkgs.openssh
pkgs.git
pkgs.jaq # tf
(import ../infra/tf.nix { inherit lib pkgs; })
];
}

View file

@ -29,6 +29,9 @@ let
((pkgs.formats.pythonVars { }).generate "settings.py" cfg.settings)
(builtins.toFile "extra-settings.py" cfg.extra-settings)
];
REPO_DIR = import ../../infra/tf-env.nix {
inherit lib pkgs;
};
};
python-environment = pkgs.python3.withPackages (
@ -157,9 +160,7 @@ in
};
};
users.users.${name} = {
isNormalUser = true;
};
users.users.${name}.isNormalUser = true;
users.groups.${name} = { };
systemd.services.${name} = {
@ -167,6 +168,7 @@ in
after = [ "network.target" ];
wantedBy = [ "multi-user.target" ];
path = [
pkgs.openssh
python-environment
manage-service
];

View file

@ -1,5 +1,6 @@
{
lib,
pkgs,
sqlite,
python3,
sources ? import ../../npins,
@ -11,7 +12,7 @@ let
root = ../src;
fileset = intersection (gitTracked ../../.) ../src;
};
pyproject = with lib; fromTOML pyproject-toml;
pyproject = fromTOML pyproject-toml;
# TODO: define this globally
name = "panel";
# TODO: we may want this in a file so it's easier to read statically
@ -58,7 +59,13 @@ python3.pkgs.buildPythonPackage {
mkdir -p $out/bin
cp -v ${src}/manage.py $out/bin/manage.py
chmod +x $out/bin/manage.py
wrapProgram $out/bin/manage.py --prefix PYTHONPATH : "$PYTHONPATH"
wrapProgram $out/bin/manage.py \
--set REPO_DIR "${
import ../../infra/tf-env.nix {
inherit lib pkgs;
}
}" \
--prefix PYTHONPATH : "$PYTHONPATH"
cp ${sources.htmx}/dist/htmx.min.js* $out/${python3.sitePackages}/panel/static/
'';
}

View file

@ -1,3 +1,4 @@
# TODO upstream, see #248
{
lib,
buildPythonPackage,

View file

@ -171,6 +171,53 @@ COMPRESS_PRECOMPILERS = [
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"filters": {
"require_debug_false": {
"()": "django.utils.log.RequireDebugFalse",
},
"require_debug_true": {
"()": "django.utils.log.RequireDebugTrue",
},
},
"formatters": {
"django.server": {
"()": "django.utils.log.ServerFormatter",
"format": "[{server_time}] {message}",
"style": "{",
},
"standard": {
"format": "%(asctime)s [%(levelname)s] %(name)s: %(message)s"
},
},
"handlers": {
"console": {
"level": "INFO",
# "filters": ["require_debug_true"],
"class": "logging.StreamHandler",
"formatter": "standard",
},
"django.server": {
"level": "INFO",
"class": "logging.StreamHandler",
"formatter": "django.server",
},
"mail_admins": {
"level": "ERROR",
"filters": ["require_debug_false"],
"class": "django.utils.log.AdminEmailHandler",
},
},
"loggers": {
"": {
"handlers": ["console"],
"level": "DEBUG" if DEBUG else "INFO",
},
},
}
# Customization via user settings
# This must be at the end, as it must be able to override the above
# TODO(@fricklerhandwerk):
@ -194,6 +241,6 @@ if user_settings_file is not None:
# PATH to expose to launch button
bin_path=env['BIN_PATH']
# path of the root flake to trigger nixops from, see #94.
# path of the root flake to deploy from
# to deploy this should be specified, for dev just use a relative path.
repo_dir = env["REPO_DIR"]

View file

@ -1,6 +1,7 @@
from enum import Enum
import json
import subprocess
import logging
import os
from django.urls import reverse_lazy
@ -13,6 +14,8 @@ from django.shortcuts import render
from panel import models, settings
from panel.configuration import forms
logger = logging.getLogger(__name__)
class Index(TemplateView):
template_name = 'index.html'
@ -134,25 +137,26 @@ class DeploymentStatus(ConfigurationForm):
},
}
# serialize back and forth now we still need to manually inject the dummy user
deployment_params = json.dumps(dummy_user | json.loads(submission))
deployment_params = dummy_user | json.loads(submission)
env = {
"PATH": settings.bin_path,
# "TF_LOG": "info",
} | {
# pass in form info to our deployment
"DEPLOYMENT": deployment_params,
# FIXME: ensure sensitive info is protected
f"TF_VAR_{k}": v if isinstance(v, str) else json.dumps(v) for k, v in deployment_params.items()
}
logger.info("env: %s", env)
cwd = f"{settings.repo_dir}/infra/operator"
cmd = [
"nix",
"develop",
"--extra-experimental-features",
"configurable-impure-env",
"--command",
"nixops4",
"tofu",
# f"-chdir={cwd}",
"apply",
"test",
f"-state={cwd}/terraform.tfstate", # FIXME: separate users' state, see #313
"--auto-approve",
"-lock=false",
"-parallelism=1" # limit OOM risk
]
deployment_result = subprocess.run(
cmd,
cwd=settings.repo_dir,
env=env,
)
return deployment_result, json.loads(deployment_params)
deployment_result = subprocess.run(cmd, cwd=cwd, env=env)
logger.debug("deployment_result: %s", deployment_result)
return deployment_result, deployment_params

10
proxmox/.envrc Normal file
View file

@ -0,0 +1,10 @@
#!/usr/bin/env bash
# the shebang is ignored, but nice for editors
# shellcheck shell=bash
if type -P lorri &>/dev/null; then
eval "$(lorri direnv)"
else
echo 'while direnv evaluated .envrc, could not find the command "lorri" [https://github.com/nix-community/lorri]'
use_nix
fi

60
proxmox/README.md Normal file
View file

@ -0,0 +1,60 @@
# Infra
## Provisioning VMs with an initial configuration
NOTE[Niols]: This is very manual and clunky. Two things will happen. In the near
future, I will improve the provisioning script to make this a bit less clunky.
In the future, orchestration will be able to communicate with Proxmox directly and
everything will become much cleaner.
1. Choose names for your VMs. It is recommended to choose `fediXXX`, with `XXX`
above 100. For instance, `fedi117`.
2. Add a basic configuration for the machine. These typically go in
`infra/machines/<name>/default.nix`. You can look at other `fediXXX` VMs to
find inspiration.
2. Add a file for each of those VM's public keys, eg.
```
touch keys/systems/fedi117.pub
```
Those files need to exist during provisioning, but their content matters only
when updating the machines' configuration.
FIXME: Remove this step by making the provisioning script not fail with the
public key does not exist yet.
3. Run the provisioning script:
```
sh infra/proxmox-provision.sh fedi117
```
The script can take several ids at the same time. It requires some
authentication options and provides several more. See `--help`.
4. (Optional) Add a DNS entry for the machine; for instance `fedi117.abundos.eu
A 95.215.187.117`.
5. Grab the public host keys for the machines in question, and add it to the
repository. For instance:
```
ssh fedi117.abundos.eu 'sudo cat /etc/ssh/ssh_host_ed25519_key.pub' > keys/systems/fedi117.pub
```
FIXME: Make the provisioning script do that for us.
7. Regenerate the list of machines:
```
sh infra/machines.md.sh
```
Commit it with the machine's configuration, public key, etc.
8. At this point, the machine contains a very basic configuration that contains
just enough for it to boot and be reachable. Go on to the next section to
update the machine and put an actual configuration.
FIXME: Figure out why the full configuration isn't on the machine at this
point and fix it.
## Removing an existing VM
See `infra/proxmox-remove.sh --help`.

23
proxmox/default.nix Normal file
View file

@ -0,0 +1,23 @@
{
system ? builtins.currentSystem,
sources ? import ../npins,
pkgs ? import sources.nixpkgs { inherit system; },
}:
{
# shell for testing TF directly
shell = pkgs.mkShellNoCC {
packages = [
pkgs.openssh
pkgs.httpie
pkgs.jq
];
};
# re-export inputs so they can be overridden granularly
# (they can't be accessed from the outside any other way)
inherit
sources
system
pkgs
;
}

View file

@ -15,7 +15,6 @@ let
installer =
{
config,
pkgs,
lib,
...

View file

@ -228,8 +228,7 @@ build_iso () {
nix build \
--impure --expr "
let flake = builtins.getFlake (builtins.toString ./.); in
flake.lib.makeInstallerIso {
import ./makeInstallerIso.nix {
nixosConfiguration = flake.nixosConfigurations.$vm_name;
nixpkgs = flake.inputs.nixpkgs;
$nix_host_keys

10
secrets/.envrc Normal file
View file

@ -0,0 +1,10 @@
#!/usr/bin/env bash
# the shebang is ignored, but nice for editors
# shellcheck shell=bash
if type -P lorri &>/dev/null; then
eval "$(lorri direnv)"
else
echo 'while direnv evaluated .envrc, could not find the command "lorri" [https://github.com/nix-community/lorri]'
use_nix
fi

View file

@ -22,7 +22,7 @@ As an example, let us add a secret in a file “cheeses” whose content should
extension); this will open your `$EDITOR` ; enter “best ones come
unpasteurised”, save and close.
3. If you are doing something flake-related such as NixOps4, remember to commit
3. If you are doing something flake-related, remember to commit
or at least stage the secret.
4. In the machine's configuration, load our `ageSecrets` NixOS module, declare the machine's host key and start using your secrets, eg.:

24
secrets/default.nix Normal file
View file

@ -0,0 +1,24 @@
{
system ? builtins.currentSystem,
sources ? import ../npins,
pkgs ? import sources.nixpkgs { inherit system; },
}:
let
inherit (sources) agenix;
in
{
# shell for testing TF directly
shell = pkgs.mkShellNoCC {
packages = [
(pkgs.callPackage "${agenix}/pkgs/agenix.nix" { })
];
};
# re-export inputs so they can be overridden granularly
# (they can't be accessed from the outside any other way)
inherit
sources
system
pkgs
;
}

1
secrets/shell.nix Normal file
View file

@ -0,0 +1 @@
(import ./. { }).shell

10
services/.envrc Normal file
View file

@ -0,0 +1,10 @@
#!/usr/bin/env bash
# the shebang is ignored, but nice for editors
# shellcheck shell=bash
if type -P lorri &>/dev/null; then
eval "$(lorri direnv)"
else
echo 'while direnv evaluated .envrc, could not find the command "lorri" [https://github.com/nix-community/lorri]'
use_nix
fi

14
services/default.nix Normal file
View file

@ -0,0 +1,14 @@
{
system ? builtins.currentSystem,
sources ? import ../npins,
pkgs ? import sources.nixpkgs {
inherit system;
},
}:
{
tests = {
mastodon = import ./tests/mastodon.nix { inherit pkgs; };
pixelfed-garage = import ./tests/pixelfed-garage.nix { inherit pkgs; };
peertube = import ./tests/peertube.nix { inherit pkgs; };
};
}

Some files were not shown because too many files have changed in this diff Show more