diff --git a/deployment/README.md b/deployment/README.md new file mode 100644 index 0000000..50fddd0 --- /dev/null +++ b/deployment/README.md @@ -0,0 +1,219 @@ +# Provisioning VMs via Proxmox + +## Quick links + +Proxmox API doc +: + +Fediversity Proxmox +: + +## Basic terminology + +Node +: physical host + +## Fediversity Proxmox + +- It is only accessible via Procolix\'s VPN: + - Get credentials for the VPN portal and Proxmox from + [Kevin](https://git.fediversity.eu/kevin). + + - Log in to the [VPN + portal](https://vpn.fediversity.eu/vpn-user-portal/home). + + - Create a **New Configuration**: + - Select **WireGuard (UDP)** + - Enter some name, e.g. `fediversity` + - Click Download + + - Write the WireGuard configuration to a file + `fediversity-vpn.config` next to your NixOS configuration + + - Add that file's path to `.git/info/exclude` and make sure + it doesn't otherwise leak (for example, use + [Agenix](https://github.com/ryantm/agenix) to manage + secrets) + + - To your NixOS configuration, add + + ``` nix + networking.wg-quick.interfaces.fediversity.configFile = toString ./fediversity-vpn.config; + ``` +- Select "Promox VE authentication server". +- Ignore the "You do not have a valid subscription" message. + +## Automatically + +This directory contains scripts that can automatically provision or +remove a Proxmox VM. For now, they are tied to one node in the +Fediversity Proxmox, but it would not be difficult to make them more +generic. Try: + +```sh +bash proxmox/provision.sh --help +bash proxmox/remove.sh --help +``` + +## Preparing the machine configuration + +- It is nicer if the machine is a QEMU guest. On NixOS: + + ``` nix + services.qemuGuest.enable = true + ``` + +- Choose name for your machine. + +- Choose static IPs for your machine. The IPv4 and IPv6 subnets + available for Fediversity testing are: + + - `95.215.187.0/24`. Gateway is `95.215.187.1`. + - `2a00:51c0:13:1305::/64`. Gateway is `2a00:51c0:13:1305::1`. + +- I have been using id `XXX` (starting from `001`), name `fediXXX`, + `95.215.187.XXX` and `2a00:51c0:13:1305::XXX`. + +- Name servers should be `95.215.185.6` and `95.215.185.7`. + +- Check [Netbox](https://netbox.protagio.org) to see which addresses + are free. + +## Manually via the GUI + +### Upload your ISO + +- Go to Fediversity proxmox. +- In the left view, expand under the node that you want and click on + "local". +- Select "ISO Images", then click "Upload". +- Note: You can also download from URL. +- Note: You should click on "local" and not "local-zfs". + +### Creating the VM + +- Click "Create VM" at the top right corner. + +#### General + +Node +: which node will host the VM; has to be the same + +VM ID +: Has to be unique, probably best to use the `xxxx` in `vm0xxxx` + (yet to be decided) + +Name +: Usually `vm` + 5 digits, e.g. `vm02199` + +Resource pool +: Fediversity + +#### OS + +Use CD/DVD disc image file (iso) + +: + + Storage + : local, means storage of the node. + + ISO image + : select the image previously uploaded + +No need to touch anything else + +#### System + +BIOS +: OVMF (UEFI) + +EFI Storage +: `linstor_storage`; this is a storage shared by all of the Proxmox + machines. + +Pre-Enroll keys +: MUST be unchecked + +Qemu Agent +: check + +#### Disks + +- Tick "advanced" at the bottom. +- Disk size (GiB) :: 40 (depending on requirements) +- SSD emulation :: check (only visible if "Advanced" is checked) +- Discard :: check, so that blocks of removed data are cleared + +#### CPU + +Sockets +: 1 (depending on requirements) + +Cores +: 2 (depending on requirements) + +Enable NUMA +: check + +#### Memory + +Memory (MiB) +: choose what you want + +Ballooning Device +: leave checked (only visible if "Advanced" is checked) + +#### Network + +Bridge +: `vnet1306`. This is the provisioning bridge; + we will change it later. + +Firewall +: uncheck, we will handle the firewall on the VM itself + +#### Confirm + +### Install and start the VM + +- Start the VM a first time. + - Select the VM in the left panel. You might have to expand the + node on which it is hosted. + - Select "Console" and start the VM. +- Install the VM as you would any other machine. +- [*Shutdown the VM*]{.spurious-link target="Shutdown the VM"}. +- After the VM has been installed: + - Select the VM again, then go to "Hardware". + - Double click on the CD/DVD Drive line. Select "Do not use any + media" and press OK. + - Double click on Network Device, and change the bridge to + `vnet1305`, the public bridge. +- Start the VM again. + +### Remove the VM + +- [*Shutdown the VM*]{.spurious-link target="Shutdown the VM"}. +- On the top right corner, click "More", then "Remove". +- Enter the ID of the machine. +- Check "Purge from job configurations" +- Check "Destroy unreferenced disks owned by guest" +- Click "Remove". + +### Move the VM to another node + +- Make sure there is no ISO plugged in. +- Click on the VM. Click migrate. Choose target node. Go. +- Since the storage is shared, it should go pretty fast (~1 minute). + +### Shutdown the VM + +- Find the VM in the left panel. +- At the top right corner appears a "Shutdown" button with a submenu. +- Clicking "Shutdown" sends a signal to shutdown the machine. This + might not work if the machine is not listening for that signal. +- Brutal solution: in the submenu, select "Stop". +- The checkbox "Overrule active shutdown tasks" means that the machine + should be stopped even if a shutdown is currently ongoing. This is + particularly important if you have tried to shut the machine down + normally just before. diff --git a/deployment/README.org b/deployment/README.org deleted file mode 100644 index ec8acd1..0000000 --- a/deployment/README.org +++ /dev/null @@ -1,113 +0,0 @@ -#+title: Provisioning VMs via Proxmox - -* Quick links -- Proxmox API doc :: https://pve.proxmox.com/pve-docs/api-viewer -- Fediversity Proxmox :: http://192.168.51.81:8006/ -* Basic terminology -- Node :: physical host -* Fediversity Proxmox -- It is only accessible via Procolix's VPN: - - Get credentials for the VPN portal and Proxmox from [[https://git.fediversity.eu/kevin][Kevin]]. - - Log in to the [[https://vpn.fediversity.eu/vpn-user-portal/home][VPN portal]]. - - Create a *New Configuration*: - - Select *WireGuard (UDP)* - - Enter some name, e.g. ~fediversity~ - - Click Download - - Write the WireGuard configuration to a file ~fediversity-vpn.config~ next to your NixOS configuration - - Add that file's path to ~.git/info/exclude~ and make sure it doesn't otherwise leak (for example, use [[https://github.com/ryantm/agenix][Agenix]] to manage secrets) - - To your NixOS configuration, add - #+begin_src nix - networking.wg-quick.interfaces.fediversity.configFile = toString ./fediversity-vpn.config; - #+end_src -- Select “Promox VE authentication server”. -- Ignore the “You do not have a valid subscription” message. -* Automatically -This directory contains scripts that can automatically provision or remove a -Proxmox VM. For now, they are tied to one node in the Fediversity Proxmox, but -it would not be difficult to make them more generic. Try: -#+begin_src sh -sh proxmox/provision.sh --help -sh proxmox/remove.sh --help -#+end_src -* Preparing the machine configuration -- It is nicer if the machine is a QEMU guest. On NixOS: - #+begin_src nix - services.qemuGuest.enable = true - #+end_src -- Choose name for your machine. -- Choose static IPs for your machine. The IPv4 and IPv6 subnets available for - Fediversity testing are: - - ~95.215.187.0/24~. Gateway is ~95.215.187.1~. - - ~2a00:51c0:13:1305::/64~. Gateway is ~2a00:51c0:13:1305::1~. -- I have been using id ~XXX~ (starting from ~001~), name ~fediXXX~, ~95.215.187.XXX~ and - ~2a00:51c0:13:1305::XXX~. -- Name servers should be ~95.215.185.6~ and ~95.215.185.7~. -- Check [[https://netbox.protagio.org][Netbox]] to see which addresses are free. -* Manually via the GUI -** Upload your ISO -- Go to Fediversity proxmox. -- In the left view, expand under the node that you want and click on “local”. -- Select “ISO Images”, then click “Upload”. -- Note: You can also download from URL. -- Note: You should click on “local” and not “local-zfs”. -** Creating the VM -- Click “Create VM” at the top right corner. -*** General -- Node :: which node will host the VM; has to be the same -- VM ID :: Has to be unique, probably best to use the "xxxx" in "vm0xxxx" (yet to be decided) -- Name :: Usually "vm" + 5 digits, e.g. "vm02199" -- Resource pool :: Fediversity -*** OS -- Use CD/DVD disc image file (iso) :: - - Storage :: local, means storage of the node. - - ISO image :: select the image previously uploaded -No need to touch anything else -*** System -- BIOS :: OVMF (UEFI) -- EFI Storage :: ~linstor_storage~; this is a storage shared by all of the Proxmox machines. -- Pre-Enroll keys :: MUST be unchecked -- Qemu Agent :: check -*** Disks -- Tick “advanced” at the bottom. -- Disk size (GiB) :: 40 (depending on requirements) -- SSD emulation :: check (only visible if “Advanced” is checked) -- Discard :: check, so that blocks of removed data are cleared -*** CPU -- Sockets :: 1 (depending on requirements) -- Cores :: 2 (depending on requirements) -- Enable NUMA :: check -*** Memory -- Memory (MiB) :: choose what you want -- Ballooning Device :: leave checked (only visible if “Advanced” is checked) -*** Network -- Bridge :: ~vnet1306~. This is the provisioning bridge; we will change it later. -- Firewall :: uncheck, we will handle the firewall on the VM itself -*** Confirm -** Install and start the VM -- Start the VM a first time. - - Select the VM in the left panel. You might have to expand the node on which it is hosted. - - Select “Console” and start the VM. -- Install the VM as you would any other machine. -- [[Shutdown the VM]]. -- After the VM has been installed: - - Select the VM again, then go to “Hardware”. - - Double click on the CD/DVD Drive line. Select “Do not use any media” and press OK. - - Double click on Network Device, and change the bridge to ~vnet1305~, the public bridge. -- Start the VM again. -** Remove the VM -- [[Shutdown the VM]]. -- On the top right corner, click “More”, then “Remove”. -- Enter the ID of the machine. -- Check “Purge from job configurations” -- Check “Destroy unreferenced disks owned by guest” -- Click “Remove”. -** Move the VM to another node -- Make sure there is no ISO plugged in. -- Click on the VM. Click migrate. Choose target node. Go. -- Since the storage is shared, it should go pretty fast (~1 minute). -** Shutdown the VM -- Find the VM in the left panel. -- At the top right corner appears a “Shutdown” button with a submenu. -- Clicking “Shutdown” sends a signal to shutdown the machine. This might not work if the machine is not listening for that signal. -- Brutal solution: in the submenu, select “Stop”. -- The checkbox “Overrule active shutdown tasks” means that the machine should be stopped even if a shutdown is currently ongoing. This is particularly important if you have tried to shut the machine down normally just before. diff --git a/infra/README.md b/infra/README.md new file mode 100644 index 0000000..fef93f6 --- /dev/null +++ b/infra/README.md @@ -0,0 +1,65 @@ +# Infra + +This directory contains the definition of the VMs that host our infrastructure. + +## NixOps4 + +Their configuration can be updated via NixOps4. Run + +```sh +nixops4 deployments list +``` + +to see the available deployments. +This should be done from the root of the repository, +otherwise NixOps4 will fail with something like: + +``` + nixops4 error: evaluation: error: + … while calling the 'getFlake' builtin + + error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist, evaluation: error: + … while calling the 'getFlake' builtin + + error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist +``` + +Then, given a deployment (eg. `git`), run + +```sh +nixops4 apply +``` + +Alternatively, to run the `default` deployment, run + +```sh +nixops4 apply +``` + +## Deployments + +default +: Contains everything + +`git` +: Machines hosting our Git infrastructure, eg. Forgejo and its actions runners + +`web` +: Machines hosting our online content, eg. the website or the wiki + +`other` +: Machines without a specific purpose + +## Machines + +These machines are hosted on the Procolix Proxmox instance, +to which non-Procolix members of the project do not have access. +They host our stable infrastructure. + + Machine Proxmox Description Deployment + --------- ------------- ------------------------ ------------ + vm02116 Procolix Forgejo `git` + vm02179 Procolix *unused* `other` + vm02186 Procolix *unused* `other` + vm02187 Procolix Wiki `web` + fedi300 Fediversity Forgejo actions runner `git` diff --git a/infra/README.org b/infra/README.org deleted file mode 100644 index 8029f4b..0000000 --- a/infra/README.org +++ /dev/null @@ -1,58 +0,0 @@ -#+title: Infra - -This directory contains the definition of the VMs that host our infrastructure. - -* NixOps4 - -Their configuration can be updated via NixOps4. Run - -#+begin_src sh -nixops4 deployments list -#+end_src - -to see the available deployments. This should be done from the root of the -repository, otherwise NixOps4 will fail with something like: - -#+begin_src -nixops4 error: evaluation: error: - … while calling the 'getFlake' builtin - - error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist, evaluation: error: - … while calling the 'getFlake' builtin - - error: path '/nix/store/05nn7krhvi8wkcyl6bsysznlv60g5rrf-source/flake.nix' does not exist -#+end_src - -Then, given a deployment (eg. ~git~), run - -#+begin_src sh -nixops4 apply -#+end_src - -Alternatively, to run the ~default~ deployment, run - -#+begin_src sh -nixops4 apply -#+end_src - -* Deployments - -- default :: Contains everything -- ~git~ :: Machines hosting our Git infrastructure, eg. Forgejo and its actions - runners -- ~web~ :: Machines hosting our online content, eg. the website or the wiki -- ~other~ :: Machines without a specific purpose - -* Machines - -These machines are hosted on the Procolix Proxmox instance, to which -non-Procolix members of the project do not have access. They host our stable -infrastructure. - -| Machine | Proxmox | Description | Deployment | -|---------+-------------+------------------------+------------| -| vm02116 | Procolix | Forgejo | ~git~ | -| vm02179 | Procolix | /unused/ | ~other~ | -| vm02186 | Procolix | /unused/ | ~other~ | -| vm02187 | Procolix | Wiki | ~web~ | -| fedi300 | Fediversity | Forgejo actions runner | ~git~ |