Fediversity/deployment/README.md

# Provisioning VMs via Proxmox

## Quick links

Proxmox API doc
:   <https://pve.proxmox.com/pve-docs/api-viewer>

Fediversity Proxmox
:   <http://192.168.51.81:8006/>

## Basic terminology

Node
:   physical host

## Fediversity Proxmox

-   It is only accessible via Procolix\'s VPN:
    -   Get credentials for the VPN portal and Proxmox from
        [Kevin](https://git.fediversity.eu/kevin).

    -   Log in to the [VPN
        portal](https://vpn.fediversity.eu/vpn-user-portal/home).

        -   Create a **New Configuration**:
        -   Select **WireGuard (UDP)**
        -   Enter some name, e.g. `fediversity`
        -   Click Download

    -   Write the WireGuard configuration to a file
        `fediversity-vpn.config` next to your NixOS configuration

        -   Add that file's path to `.git/info/exclude` and make sure
            it doesn't otherwise leak (for example, use
            [Agenix](https://github.com/ryantm/agenix) to manage
            secrets)

    -   To your NixOS configuration, add

        ``` nix
        networking.wg-quick.interfaces.fediversity.configFile = toString ./fediversity-vpn.config;
        ```
-   Select "Promox VE authentication server".
-   Ignore the "You do not have a valid subscription" message.

## Automatically

This directory contains scripts that can automatically provision or
remove a Proxmox VM. For now, they are tied to one node in the
Fediversity Proxmox, but it would not be difficult to make them more
generic. Try:

```sh
bash proxmox/provision.sh --help
bash proxmox/remove.sh --help
```

## Preparing the machine configuration

-   It is nicer if the machine is a QEMU guest. On NixOS:

    ``` nix
    services.qemuGuest.enable = true
    ```

-   Choose name for your machine.

-   Choose static IPs for your machine. The IPv4 and IPv6 subnets
    available for Fediversity testing are:

    -   `95.215.187.0/24`. Gateway is `95.215.187.1`.
    -   `2a00:51c0:13:1305::/64`. Gateway is `2a00:51c0:13:1305::1`.

-   I have been using id `XXX` (starting from `001`), name `fediXXX`,
    `95.215.187.XXX` and `2a00:51c0:13:1305::XXX`.

-   Name servers should be `95.215.185.6` and `95.215.185.7`.

-   Check [Netbox](https://netbox.protagio.org) to see which addresses
    are free.

## Manually via the GUI

### Upload your ISO

-   Go to Fediversity proxmox.
-   In the left view, expand under the node that you want and click on
    "local".
-   Select "ISO Images", then click "Upload".
-   Note: You can also download from URL.
-   Note: You should click on "local" and not "local-zfs".

### Creating the VM

-   Click "Create VM" at the top right corner.

#### General

Node
:   which node will host the VM; has to be the same

VM ID
:   Has to be unique, probably best to use the `xxxx` in `vm0xxxx`
    (yet to be decided)

Name
:   Usually `vm` + 5 digits, e.g. `vm02199`

Resource pool
:   Fediversity

#### OS

Use CD/DVD disc image file (iso)

:

    Storage
    :   local, means storage of the node.

    ISO image
    :   select the image previously uploaded

No need to touch anything else

#### System

BIOS
:   OVMF (UEFI)

EFI Storage
:   `linstor_storage`; this is a storage shared by all of the Proxmox
    machines.

Pre-Enroll keys
:   MUST be unchecked

Qemu Agent
:   check

#### Disks

-   Tick "advanced" at the bottom.
-   Disk size (GiB) :: 40 (depending on requirements)
-   SSD emulation :: check (only visible if "Advanced" is checked)
-   Discard :: check, so that blocks of removed data are cleared

#### CPU

Sockets
:   1 (depending on requirements)

Cores
:   2 (depending on requirements)

Enable NUMA
:   check

#### Memory

Memory (MiB)
:   choose what you want

Ballooning Device
:   leave checked (only visible if "Advanced" is checked)

#### Network

Bridge
:   `vnet1306`. This is the provisioning bridge;
    we will change it later.

Firewall
:   uncheck, we will handle the firewall on the VM itself

#### Confirm

### Install and start the VM

-   Start the VM a first time.
    -   Select the VM in the left panel. You might have to expand the
        node on which it is hosted.
    -   Select "Console" and start the VM.
-   Install the VM as you would any other machine.
-   [*Shutdown the VM*]{.spurious-link target="Shutdown the VM"}.
-   After the VM has been installed:
    -   Select the VM again, then go to "Hardware".
    -   Double click on the CD/DVD Drive line. Select "Do not use any
        media" and press OK.
    -   Double click on Network Device, and change the bridge to
        `vnet1305`, the public bridge.
-   Start the VM again.

### Remove the VM

-   [*Shutdown the VM*]{.spurious-link target="Shutdown the VM"}.
-   On the top right corner, click "More", then "Remove".
-   Enter the ID of the machine.
-   Check "Purge from job configurations"
-   Check "Destroy unreferenced disks owned by guest"
-   Click "Remove".

### Move the VM to another node

-   Make sure there is no ISO plugged in.
-   Click on the VM. Click migrate. Choose target node. Go.
-   Since the storage is shared, it should go pretty fast (~1 minute).

### Shutdown the VM

-   Find the VM in the left panel.
-   At the top right corner appears a "Shutdown" button with a submenu.
-   Clicking "Shutdown" sends a signal to shutdown the machine. This
    might not work if the machine is not listening for that signal.
-   Brutal solution: in the submenu, select "Stop".
-   The checkbox "Overrule active shutdown tasks" means that the machine
    should be stopped even if a shutdown is currently ongoing. This is
    particularly important if you have tried to shut the machine down
    normally just before.