move files in line with tf-infra

.gitignore

@@ -1,3 +1,9 @@
+.npins.json
+.terraform/
+.terraform.lock.hcl
+.terraform.tfstate.lock.info
+terraform.tfstate*
+.auto.tfvars.json
 .DS_Store
 .idea
 *.log

default.nix

@@ -27,7 +27,7 @@ let
             ## Add a directory here if pre-commit hooks shouldn't apply to it.
             optout = [
               "npins"
-              "launch/.terraform"
+              "**/.terraform"
             ];
             excludes = map (dir: "^${dir}/") optout;
             addExcludes = lib.mapAttrs (_: c: c // { inherit excludes; });

infra/README.md

@@ -1,96 +1,28 @@
-# Infra
+# service deployment
 
-This directory contains the definition of [the VMs](../machines/machines.md) that host our
+deploys [NixOS](https://nixos.org/) templates using [OpenTofu](https://opentofu.org/).
-infrastructure.
 
-## Provisioning VMs with an initial configuration
+## requirements
 
-> NOTE[Niols]: This is still very manual and clunky. Two things will happen:
+- [nix](https://nix.dev/)
-> 1. In the near future, I will improve the provisioning script to make this a bit less clunky.
-> 2. In the far future, NixOps4 will be able to communicate with Proxmox directly and everything will become much cleaner.
 
-1. Choose names for your VMs. It is recommended to choose `fediXXX`, with `XXX`
+## usage
-   above 100. For instance, `fedi117`.
 
-2. Add a basic configuration for the machine. These typically go in
+### development
-   `machines/dev/<name>/default.nix`. You can look at other `fediXXX` VMs to
-   find inspiration. You probably do not need a `nixos.module` option at this
-   point.
 
-2. Add a file for each of those VM's public keys, eg.
+before using other commands, if not using direnv:
-   ```
-   touch keys/systems/fedi117.pub
-   ```
-   Those files need to exist during provisioning, but their content matters only
-   when updating the machines' configuration.
-
-   > FIXME: Remove this step by making the provisioning script not fail with the public key does not exist yet.
-
-3. Run the provisioning script:
-   ```
-   sh infra/proxmox-provision.sh fedi117
-   ```
-   The script can take several ids at the same time. It requires some
-   authentication options and provides several more. See `--help`.
-
-4. (Optional) Add a DNS entry for the machine; for instance `fedi117.abundos.eu
-   A 95.215.187.117`.
-
-5. Grab the public host keys for the machines in question, and add it to the
-   repository. For instance:
-   ```
-   ssh fedi117.abundos.eu 'sudo cat /etc/ssh/ssh_host_ed25519_key.pub' > keys/systems/fedi117.pub
-   ```
-
-   > FIXME: Make the provisioning script do that for us.
-
-7. Regenerate the list of machines:
-   ```
-   sh machines/machines.md.sh
-   ```
-   Commit it with the machine's configuration, public key, etc.
-
-8. At this point, the machine contains a very basic configuration that contains
-   just enough for it to boot and be reachable. Go on to the next section to
-   update the machine and put an actual configuration.
-
-   > FIXME: Figure out why the full configuration isn't on the machine at this
-   point and fix it.
-
-## Updating existing VM configurations
-
-Their configuration can be updated via NixOps4. Run
 
 ```sh
-nixops4 deployments list
+nix-shell
 ```
 
-to see the available deployments.
+then to initialize, or after updating pins or TF providers:
-This should be done from the root of the repository,
-otherwise NixOps4 will fail with something like:
-
-```
-    nixops4 error: evaluation: error:
-           … while calling the 'getFlake' builtin
-
-           error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist, evaluation: error:
-           … while calling the 'getFlake' builtin
-
-           error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist
-```
-
-Then, given a deployment (eg. `fedi200`), run
 
 ```sh
-nixops4 apply <deployment>
+setup
 ```
 
-Alternatively, to run the `default` deployment, which contains all the VMs, run
+## implementing
 
-```sh
+proper documentation TODO.
-nixops4 apply
+until then, a reference implementation may be found in [`panel/`](https://git.fediversity.eu/Fediversity/Fediversity/src/branch/main/panel).
-```
-
-## Removing an existing VM
-
-See `infra/proxmox-remove.sh --help`.

infra/common/shared.nix

@@ -11,7 +11,7 @@ in
     <disko/module.nix>
     <agenix/modules/age.nix>
     ../services/fediversity
-    ./resource.nix
+    ./node.nix
   ];
   fediversityVm.name = hostname;
   fediversity = {

infra/operator/main.tf

@@ -96,9 +96,9 @@ resource "terraform_data" "nixos" {
                 # FIXME: separate template options by service
                 ${path.root}/options.nix
                 # for service `mastodon` import `mastodon.nix`
-                ${path.root}/${each.key}.nix
+                ${path.root}/../../machines/operator/${inst.hostname}/${each.key}.nix
                 # FIXME: get VM details from TF
-                ${path.root}/../infra/test-machines/${each.value.hostname}
+                ${path.root}/../../machines/operator/${inst.hostname}
               ];
               # nix path for debugging
               nix.nixPath = [ "${local.nix_path}" ];

infra/tf-env.nix

@@ -18,7 +18,7 @@ pkgs.stdenv.mkDerivation {
   ];
   buildPhase = ''
     runHook preBuild
-    pushd launch/
+    pushd infra
     # calculated pins
     echo '${lib.strings.toJSON sources}' > .npins.json
     # generate TF lock for nix's TF providers

launch/.gitignore

@@ -1,7 +0,0 @@
-# generated
-.auto.tfvars.json
-.npins.json
-.terraform/
-.terraform.lock.hcl
-.terraform.tfstate.lock.info
-terraform.tfstate*

launch/README.md

@@ -1,28 +0,0 @@
-# service deployment
-
-deploys [NixOS](https://nixos.org/) templates using [OpenTofu](https://opentofu.org/).
-
-## requirements
-
-- [nix](https://nix.dev/)
-
-## usage
-
-### development
-
-before using other commands, if not using direnv:
-
-```sh
-nix-shell
-```
-
-then to initialize, or after updating pins or TF providers:
-
-```sh
-setup
-```
-
-## implementing
-
-proper documentation TODO.
-until then, a reference implementation may be found in [`panel/`](https://git.fediversity.eu/Fediversity/Fediversity/src/branch/main/panel).

panel/env.nix

@@ -11,6 +11,6 @@
     pkgs.openssh
     pkgs.git
     pkgs.jaq # tf
-    (import ../launch/tf.nix { inherit lib pkgs; })
+    (import ../infra/tf.nix { inherit lib pkgs; })
   ];
 }

panel/nix/configuration.nix

@@ -29,7 +29,7 @@ let
       ((pkgs.formats.pythonVars { }).generate "settings.py" cfg.settings)
       (builtins.toFile "extra-settings.py" cfg.extra-settings)
     ];
-    REPO_DIR = import ../../launch/tf-env.nix { inherit lib pkgs; };
+    REPO_DIR = import ../../infra/tf-env.nix { inherit lib pkgs; };
   };
 
   python-environment = pkgs.python3.withPackages (

panel/nix/package.nix

@@ -100,7 +100,7 @@ python3.pkgs.buildPythonPackage {
     cp -v ${src}/manage.py $out/bin/manage.py
     chmod +x $out/bin/manage.py
     wrapProgram $out/bin/manage.py \
-      --set REPO_DIR "${import ../../launch/tf-env.nix { inherit lib pkgs; }}" \
+      --set REPO_DIR "${import ../../infra/tf-env.nix { inherit lib pkgs; }}" \
       --prefix PYTHONPATH : "$PYTHONPATH"
     ${lib.concatStringsSep "\n" (
       map (file: "cp ${file.from} $out/${python3.sitePackages}/${file.to}") generated

panel/src/panel/configuration/schema.json

@@ -0,0 +1 @@
+/nix/store/cgfzw0ddm69ypfmxpfzbifzswipa8pq9-schema.json

proxmox/.envrc

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+# the shebang is ignored, but nice for editors
+
+# shellcheck shell=bash
+if type -P lorri &>/dev/null; then
+  eval "$(lorri direnv)"
+else
+  echo 'while direnv evaluated .envrc, could not find the command "lorri" [https://github.com/nix-community/lorri]'
+  use_nix
+fi

proxmox/README.md

@@ -0,0 +1,98 @@
+# Infra
+
+This directory contains the definition of [the VMs](machines.md) that host our
+infrastructure.
+
+## Provisioning VMs with an initial configuration
+
+NOTE[Niols]: This is very manual and clunky. Two things will happen. In the near
+future, I will improve the provisioning script to make this a bit less clunky.
+In the far future, NixOps4 will be able to communicate with Proxmox directly and
+everything will become much cleaner.
+
+1. Choose names for your VMs. It is recommended to choose `fediXXX`, with `XXX`
+   above 100. For instance, `fedi117`.
+
+2. Add a basic configuration for the machine. These typically go in
+   `infra/machines/<name>/default.nix`. You can look at other `fediXXX` VMs to
+   find inspiration. You probably do not need a `nixos.module` option at this
+   point.
+
+2. Add a file for each of those VM's public keys, eg.
+   ```
+   touch keys/systems/fedi117.pub
+   ```
+   Those files need to exist during provisioning, but their content matters only
+   when updating the machines' configuration.
+
+   FIXME: Remove this step by making the provisioning script not fail with the
+   public key does not exist yet.
+
+3. Run the provisioning script:
+   ```
+   sh infra/proxmox-provision.sh fedi117
+   ```
+   The script can take several ids at the same time. It requires some
+   authentication options and provides several more. See `--help`.
+
+4. (Optional) Add a DNS entry for the machine; for instance `fedi117.abundos.eu
+   A 95.215.187.117`.
+
+5. Grab the public host keys for the machines in question, and add it to the
+   repository. For instance:
+   ```
+   ssh fedi117.abundos.eu 'sudo cat /etc/ssh/ssh_host_ed25519_key.pub' > keys/systems/fedi117.pub
+   ```
+
+   FIXME: Make the provisioning script do that for us.
+
+7. Regenerate the list of machines:
+   ```
+   sh infra/machines.md.sh
+   ```
+   Commit it with the machine's configuration, public key, etc.
+
+8. At this point, the machine contains a very basic configuration that contains
+   just enough for it to boot and be reachable. Go on to the next section to
+   update the machine and put an actual configuration.
+
+   FIXME: Figure out why the full configuration isn't on the machine at this
+   point and fix it.
+
+## Updating existing VM configurations
+
+Their configuration can be updated via NixOps4. Run
+
+```sh
+nixops4 deployments list
+```
+
+to see the available deployments.
+This should be done from the root of the repository,
+otherwise NixOps4 will fail with something like:
+
+```
+    nixops4 error: evaluation: error:
+           … while calling the 'getFlake' builtin
+
+           error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist, evaluation: error:
+           … while calling the 'getFlake' builtin
+
+           error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist
+```
+
+Then, given a deployment (eg. `fedi200`), run
+
+```sh
+nixops4 apply <deployment>
+```
+
+Alternatively, to run the `default` deployment, which contains all the VMs, run
+
+```sh
+nixops4 apply
+```
+
+## Removing an existing VM
+
+See `infra/proxmox-remove.sh --help`.

proxmox/default.nix

@@ -0,0 +1,23 @@
+{
+  system ? builtins.currentSystem,
+  sources ? import ../npins,
+  pkgs ? import sources.nixpkgs { inherit system; },
+}:
+{
+  # shell for testing TF directly
+  shell = pkgs.mkShellNoCC {
+    packages = [
+      pkgs.openssh
+      pkgs.httpie
+      pkgs.jq
+    ];
+  };
+
+  # re-export inputs so they can be overridden granularly
+  # (they can't be accessed from the outside any other way)
+  inherit
+    sources
+    system
+    pkgs
+    ;
+}