Add 'services/' from commit 'a13b1e9372412d03f7b629694ecef5242e6e568d'
git-subtree-dir: services git-subtree-mainline:e41e0daa82
git-subtree-split:a13b1e9372
This commit is contained in:
commit
c6cc92f5dc
1
services/.envrc
Normal file
1
services/.envrc
Normal file
|
@ -0,0 +1 @@
|
|||
use flake
|
8
services/.gitignore
vendored
Normal file
8
services/.gitignore
vendored
Normal file
|
@ -0,0 +1,8 @@
|
|||
nixos.qcow2
|
||||
result*
|
||||
.direnv
|
||||
.nixos-test-history
|
||||
*screenshot.png
|
||||
output
|
||||
todo
|
||||
|
127
services/README.md
Normal file
127
services/README.md
Normal file
|
@ -0,0 +1,127 @@
|
|||
# Fediverse VMs
|
||||
|
||||
This repo is, for now, an attempt to familiarize myself with NixOS options for Fediverse applications, and build up a configuration layer that will set most of the relevant options for you (in a semi-opinionated way) given some high-level configuration. The goal is something in the same vein as [nixos-mailserver](https://gitlab.com/simple-nixos-mailserver/nixos-mailserver) but for fediversity.
|
||||
|
||||
Eventually, this will be tailored to high-throughput multi-machine setups. For now, it's just a small set of configurations to run in VMs.
|
||||
|
||||
## Running the VMs
|
||||
|
||||
you can build a VM using
|
||||
|
||||
```bash
|
||||
nixos-rebuild build-vm --flake .#<vm_name>
|
||||
```
|
||||
|
||||
where `<vm_name>` is one of `mastodon`, `peertube`, `pixelfed`, or `all`
|
||||
|
||||
and then run it with
|
||||
```bash
|
||||
./result/bin/run-nixos-vm
|
||||
```
|
||||
|
||||
After the machine boots, you should be dropped into a root shell.
|
||||
|
||||
Note that state will be persisted in the `nixos.cqow2` file. Delete that and restart the VM to reset the state.
|
||||
|
||||
With the VM running, you can then access the apps on your local machine's web browser (using the magic of port forwarding) at the following addresses
|
||||
|
||||
NOTE: it sometimes takes a while for the services to start up, and in the meantime you will get 502 Bad Gateway.
|
||||
|
||||
- Mastodon: through the reverse proxy at <https://mastodon.localhost:8443> and directly at <http://mastodon.localhost:55001>
|
||||
- You can create accounts on the machine itself by running `mastodon-tootctl accounts create test --email test@test.com --confirmed --approve`
|
||||
- Account-related activities (logging in/out; preferences) can only be done on the insecure direct page <http://mastodon.localhost:55001>
|
||||
- After you've logged in, you can go back to the secure page and you will remain logged in
|
||||
- some operations may remove the port number from the URL. You'll have to add that back in manually
|
||||
|
||||
- PeerTube: <http://peertube.localhost:9000>
|
||||
- The root account can be accessed with username "root". The password can be obtained by running the following command on the VM:
|
||||
```bash
|
||||
journalctl -u peertube | perl -ne '/password: (.*)/ && print $1'
|
||||
```
|
||||
- Creating other accounts has to be enabled via the admin interface. `Administration > Configuration > Basic > Enable Signup` or just add an account directly from `Administration > Create user`. But functionality can also be tested from the root account.
|
||||
|
||||
- Pixelfed: <http://pixelfed.localhost:8000>
|
||||
- Account creation via the web interface won't work until we figure out email
|
||||
- For now, they can be created on the VM command line
|
||||
```bash
|
||||
pixelfed-manage user:create --name=test --username=test --email=test@test.com --password=testtest --confirm_email=1
|
||||
```
|
||||
# Building an installer image
|
||||
|
||||
Build an installer image for the desired configuration, e.g. for `peertube`:
|
||||
|
||||
```bash
|
||||
nix build .#installers.peertube
|
||||
```
|
||||
|
||||
Upload the image in `./result` to Proxmox when creating a VM.
|
||||
Booting the image will format the disk and install NixOS with the desired configuration.
|
||||
|
||||
# Deploying an updated machine configuration
|
||||
|
||||
> TODO: There is currently no way to specify an actual target machine by name.
|
||||
|
||||
Assuming you have SSH configuration with access to the remote `root` user stored for a machine called e.g. `peertube`, deploy the configuration by the same name:
|
||||
|
||||
```bash
|
||||
nix run .#deploy.peertube
|
||||
```
|
||||
|
||||
## debugging notes
|
||||
|
||||
- it is sometimes useful to `cat result/bin/run-nixos-vm` to see what's really going on (e.g. which ports are getting forwarded)
|
||||
- relevant systemd services:
|
||||
- mastodon-web.service
|
||||
- peertube.service
|
||||
- the `garage` CLI command gives information about garage storage, but cannot be used to actually inspect the contents. use `mc` (minio) for that
|
||||
- run `mc alias set garage http://s3.garage.localhost:3900 --api s3v4 --path off $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY`
|
||||
- in the chromium devtools, you can go to the networking tab and change things like response headers in a way that persists through reloads. this is much faster iteration time if that's what you need to epxeriment with.
|
||||
|
||||
## NixOS Tests
|
||||
|
||||
Tests live in the aptly named `tests/` directory, and can be accessed at the flake URI `.#checks.<system>.<test-name>` e.g. `nix build .#checks.x86_64-linux.mastodon-garage`.
|
||||
They can also be run interactively with
|
||||
```
|
||||
nix build .#checks.<system>.<test>.driverInteractive
|
||||
./result/bin/nixos-test-driver 2>output
|
||||
````
|
||||
you can `less output` and then `F` from a different terminal to follow along.
|
||||
|
||||
These tests are also equiped with the same port forwarding as the VMs, so when running interactively you should be able to access services through a browser running on your machine.
|
||||
|
||||
While running interactively, `rebuildableTests` allows you to modify the test nodes and then redeploy without restarting the test and waiting for the VMs to start up again. To do this you must start the jumphost by running `redeploy_jumphost.start()` inside the driver. Then from the command line
|
||||
|
||||
```
|
||||
nix build .#checks.<system>.<test>.driverInteractive
|
||||
./result/bin/rebuild
|
||||
```
|
||||
|
||||
# questions
|
||||
|
||||
- what is meant to be shared between instances?
|
||||
- this is relevant to the security model. If garage is being shared between instances, we have to be careful having configurations depend on each other.
|
||||
- they are to be shared, BUT the user will have no direct control over configuration.
|
||||
|
||||
# resources
|
||||
|
||||
- Tutorial for setting up better logging: https://krisztianfekete.org/self-hosting-mastodon-on-nixos-a-proof-of-concept/
|
||||
- Setting up mastodon development environment: https://docs.joinmastodon.org/dev/setup/
|
||||
|
||||
- Tutorial for PeerTube that doesn't use `createLocally`: https://nixos.wiki/wiki/PeerTube
|
||||
|
||||
- garage settings for specific apps: https://garagehq.deuxfleurs.fr/documentation/connect/apps/
|
||||
|
||||
- pixelfed has terrible / mostly non-existent documentation)
|
||||
|
||||
- for when we start worry about scaling up: https://docs.joinmastodon.org/admin/scaling/
|
||||
|
||||
# notes
|
||||
|
||||
When mastodon is running in production mode, we have a few problems:
|
||||
- you have to click "accept the security risk"
|
||||
- it takes a while for the webpage to come online. Until then you see "502 Bad Gateway"
|
||||
- email sent from the mastodon instance (e.g. for account confirmation) should be accessible at <https://mastodon.localhost:55001/letter_opener>, but it's not working.
|
||||
- mastodon is trying to fetch `missing.png` without ssl (`http://`). This isn't allowed, and i'm not sure why it's doing it.
|
||||
- mastodon is trying to fetch `custom.css` from https://mastodon.localhost (no port), which is not the configured `LOCAL_DOMAIN`, so it's unclear why.
|
||||
|
||||
NixOS tests do not take the configuration from `virtualisation.vmVariant`. This seems like an oversight since people don't tend to mix normal NixOS configurations with the ones they're using for tests. This should be pretty easy to rectify upstream.
|
13
services/deploy.nix
Normal file
13
services/deploy.nix
Normal file
|
@ -0,0 +1,13 @@
|
|||
{ writeShellApplication }:
|
||||
name: config:
|
||||
writeShellApplication {
|
||||
name = "deploy";
|
||||
text = ''
|
||||
result="$(nix build --print-out-paths ${./.}#nixosConfigurations#${name} --eval-store auto --store ssh-ng://${name})"
|
||||
# shellcheck disable=SC2087
|
||||
ssh ${name} << EOF
|
||||
nix-env -p /nix/var/nix/profiles/system --set "$result"
|
||||
"$result"/bin/switch-to-configuration switch
|
||||
EOF
|
||||
'';
|
||||
}
|
36
services/disk-layout.nix
Normal file
36
services/disk-layout.nix
Normal file
|
@ -0,0 +1,36 @@
|
|||
{ ... }:
|
||||
{
|
||||
disko.devices.disk.main = {
|
||||
device = "/dev/sda";
|
||||
type = "disk";
|
||||
content = {
|
||||
type = "gpt";
|
||||
partitions = {
|
||||
MBR = {
|
||||
priority = 0;
|
||||
size = "1M";
|
||||
type = "EF02";
|
||||
};
|
||||
ESP = {
|
||||
priority = 1;
|
||||
size = "500M";
|
||||
type = "EF00";
|
||||
content = {
|
||||
type = "filesystem";
|
||||
format = "vfat";
|
||||
mountpoint = "/boot";
|
||||
};
|
||||
};
|
||||
root = {
|
||||
priority = 2;
|
||||
size = "100%";
|
||||
content = {
|
||||
type = "filesystem";
|
||||
format = "ext4";
|
||||
mountpoint = "/";
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
}
|
118
services/fediversity/default.nix
Normal file
118
services/fediversity/default.nix
Normal file
|
@ -0,0 +1,118 @@
|
|||
{ lib, config, ... }:
|
||||
|
||||
let
|
||||
inherit (builtins) toString;
|
||||
inherit (lib) mkOption mkEnableOption mkForce;
|
||||
inherit (lib.types) types;
|
||||
|
||||
in {
|
||||
imports = [
|
||||
./garage.nix
|
||||
./mastodon.nix
|
||||
./pixelfed.nix
|
||||
./peertube.nix
|
||||
];
|
||||
|
||||
options = {
|
||||
fediversity = {
|
||||
enable = mkEnableOption "the collection of services bundled under Fediversity";
|
||||
|
||||
domain = mkOption {
|
||||
type = types.str;
|
||||
description = ''
|
||||
root domain for the Fediversity services
|
||||
|
||||
For instance, if this option is set to `foo.example.com`, then
|
||||
Pixelfed might be under `pixelfed.foo.example.com`.
|
||||
'';
|
||||
};
|
||||
|
||||
mastodon.enable = mkEnableOption "default Fediversity Mastodon configuration";
|
||||
pixelfed.enable = mkEnableOption "default Fediversity Pixelfed configuration";
|
||||
peertube.enable = mkEnableOption "default Fediversity PeerTube configuration";
|
||||
|
||||
internal = mkOption {
|
||||
description = "options that are only meant to be used internally; change at your own risk";
|
||||
default = {};
|
||||
type = types.submodule {
|
||||
options = {
|
||||
garage = {
|
||||
api = {
|
||||
domain = mkOption {
|
||||
type = types.str;
|
||||
default = "s3.garage.${config.fediversity.domain}";
|
||||
};
|
||||
port = mkOption {
|
||||
type = types.int;
|
||||
default = 3900;
|
||||
};
|
||||
url = mkOption {
|
||||
type = types.str;
|
||||
default = "http://${config.fediversity.internal.garage.api.domain}:${toString config.fediversity.internal.garage.api.port}";
|
||||
};
|
||||
};
|
||||
|
||||
rpc = {
|
||||
port = mkOption {
|
||||
type = types.int;
|
||||
default = 3901;
|
||||
};
|
||||
};
|
||||
|
||||
web = {
|
||||
rootDomain = mkOption {
|
||||
type = types.str;
|
||||
default = "web.garage.${config.fediversity.domain}";
|
||||
};
|
||||
internalPort = mkOption {
|
||||
type = types.int;
|
||||
default = 3902;
|
||||
};
|
||||
domainForBucket = mkOption {
|
||||
type = types.functionTo types.str;
|
||||
default = bucket: "${bucket}.${config.fediversity.internal.garage.web.rootDomain}";
|
||||
};
|
||||
urlForBucket = mkOption {
|
||||
type = types.functionTo types.str;
|
||||
default = bucket: "http://${config.fediversity.internal.garage.web.domainForBucket bucket}";
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
## REVIEW: Do we want to recreate options under
|
||||
## `fediversity.internal` or would we rather use the options from
|
||||
## the respective services? See Taeer's comment:
|
||||
## https://git.fediversity.eu/taeer/simple-nixos-fediverse/pulls/22#issuecomment-124
|
||||
pixelfed.domain = mkOption {
|
||||
type = types.str;
|
||||
default = "pixelfed.${config.fediversity.domain}";
|
||||
};
|
||||
mastodon.domain = mkOption {
|
||||
type = types.str;
|
||||
default = "mastdodon.${config.fediversity.domain}";
|
||||
};
|
||||
peertube.domain = mkOption {
|
||||
type = types.str;
|
||||
default = "peertube.${config.fediversity.domain}";
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
config = {
|
||||
## FIXME: This should clearly go somewhere else; and we should have a
|
||||
## `staging` vs. `production` setting somewhere.
|
||||
security.acme = {
|
||||
acceptTerms = true;
|
||||
defaults.email = "nicolas.jeannerod+fediversity@moduscreate.com";
|
||||
# defaults.server = "https://acme-staging-v02.api.letsencrypt.org/directory";
|
||||
};
|
||||
|
||||
## NOTE: For a one-machine deployment, this removes the need to provide an
|
||||
## `s3.garage.<domain>` domain. However, this will quickly stop working once
|
||||
## we go to multi-machines deployment.
|
||||
fediversity.internal.garage.api.domain = mkForce "s3.garage.localhost";
|
||||
};
|
||||
}
|
213
services/fediversity/garage.nix
Normal file
213
services/fediversity/garage.nix
Normal file
|
@ -0,0 +1,213 @@
|
|||
let
|
||||
# generate one using openssl (somehow)
|
||||
# XXX: when importing, garage tells you importing is only meant for keys previously generated by garage. is it okay to generate them using openssl? it seems to work fine
|
||||
snakeoil_key = {
|
||||
id = "GK22a15201acacbd51cd43e327";
|
||||
secret = "82b2b4cbef27bf8917b350d5b10a87c92fa9c8b13a415aeeea49726cf335d74e";
|
||||
};
|
||||
in
|
||||
|
||||
# TODO: expand to a multi-machine setup
|
||||
{ config, lib, pkgs, ... }:
|
||||
|
||||
let
|
||||
inherit (builtins) toString;
|
||||
inherit (lib) types mkOption mkEnableOption optionalString concatStringsSep;
|
||||
inherit (lib.strings) escapeShellArg;
|
||||
cfg = config.services.garage;
|
||||
fedicfg = config.fediversity.internal.garage;
|
||||
concatMapAttrs = scriptFn: attrset: concatStringsSep "\n" (lib.mapAttrsToList scriptFn attrset);
|
||||
ensureBucketScriptFn = bucket: { website, aliases, corsRules }:
|
||||
let
|
||||
bucketArg = escapeShellArg bucket;
|
||||
corsRulesJSON = escapeShellArg (builtins.toJSON {
|
||||
CORSRules = [{
|
||||
AllowedHeaders = corsRules.allowedHeaders;
|
||||
AllowedMethods = corsRules.allowedMethods;
|
||||
AllowedOrigins = corsRules.allowedOrigins;
|
||||
}];
|
||||
});
|
||||
in ''
|
||||
# garage bucket info tells us if the bucket already exists
|
||||
garage bucket info ${bucketArg} || garage bucket create ${bucketArg}
|
||||
|
||||
# TODO: should this --deny the website if `website` is false?
|
||||
${optionalString website ''
|
||||
garage bucket website --allow ${bucketArg}
|
||||
''}
|
||||
|
||||
${concatStringsSep "\n" (map (alias: ''
|
||||
garage bucket alias ${bucketArg} ${escapeShellArg alias}
|
||||
'') aliases)}
|
||||
|
||||
${optionalString corsRules.enable ''
|
||||
garage bucket allow --read --write --owner ${bucketArg} --key tmp
|
||||
# TODO: endpoin-url should not be hard-coded
|
||||
aws --region ${cfg.settings.s3_api.s3_region} --endpoint-url ${fedicfg.api.url} s3api put-bucket-cors --bucket ${bucketArg} --cors-configuration ${corsRulesJSON}
|
||||
garage bucket deny --read --write --owner ${bucketArg} --key tmp
|
||||
''}
|
||||
'';
|
||||
ensureBucketsScript = concatMapAttrs ensureBucketScriptFn cfg.ensureBuckets;
|
||||
ensureAccessScriptFn = key: bucket: { read, write, owner }: ''
|
||||
garage bucket allow ${optionalString read "--read"} ${optionalString write "--write"} ${optionalString owner "--owner"} \
|
||||
${escapeShellArg bucket} --key ${escapeShellArg key}
|
||||
'';
|
||||
ensureKeyScriptFn = key: {id, secret, ensureAccess}: ''
|
||||
## FIXME: Check whether the key exist and skip this step if that is the case. Get rid of this `|| :`
|
||||
garage key import --yes -n ${escapeShellArg key} ${escapeShellArg id} ${escapeShellArg secret} || :
|
||||
${concatMapAttrs (ensureAccessScriptFn key) ensureAccess}
|
||||
'';
|
||||
ensureKeysScript = concatMapAttrs ensureKeyScriptFn cfg.ensureKeys;
|
||||
in
|
||||
|
||||
{
|
||||
# add in options to ensure creation of buckets and keys
|
||||
options = {
|
||||
services.garage = {
|
||||
ensureBuckets = mkOption {
|
||||
type = types.attrsOf (types.submodule {
|
||||
options = {
|
||||
website = mkOption {
|
||||
type = types.bool;
|
||||
default = false;
|
||||
};
|
||||
# I think setting corsRules should allow another website to show images from your bucket
|
||||
corsRules = {
|
||||
enable = mkEnableOption "CORS Rules";
|
||||
allowedHeaders = mkOption {
|
||||
type = types.listOf types.str;
|
||||
default = [];
|
||||
};
|
||||
allowedMethods = mkOption {
|
||||
type = types.listOf types.str;
|
||||
default = [];
|
||||
};
|
||||
allowedOrigins = mkOption {
|
||||
type = types.listOf types.str;
|
||||
default = [];
|
||||
};
|
||||
};
|
||||
aliases = mkOption {
|
||||
type = types.listOf types.str;
|
||||
default = [];
|
||||
};
|
||||
};
|
||||
});
|
||||
default = {};
|
||||
};
|
||||
ensureKeys = mkOption {
|
||||
type = types.attrsOf (types.submodule {
|
||||
# TODO: these should be managed as secrets, not in the nix store
|
||||
options = {
|
||||
id = mkOption {
|
||||
type = types.str;
|
||||
};
|
||||
secret = mkOption {
|
||||
type = types.str;
|
||||
};
|
||||
# TODO: assert at least one of these is true
|
||||
# NOTE: this currently needs to be done at the top level module
|
||||
ensureAccess = mkOption {
|
||||
type = types.attrsOf (types.submodule {
|
||||
options = {
|
||||
read = mkOption {
|
||||
type = types.bool;
|
||||
default = false;
|
||||
};
|
||||
write = mkOption {
|
||||
type = types.bool;
|
||||
default = false;
|
||||
};
|
||||
owner = mkOption {
|
||||
type = types.bool;
|
||||
default = false;
|
||||
};
|
||||
};
|
||||
});
|
||||
default = [];
|
||||
};
|
||||
};
|
||||
});
|
||||
default = {};
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
config = lib.mkIf config.fediversity.enable {
|
||||
environment.systemPackages = [ pkgs.minio-client pkgs.awscli ];
|
||||
|
||||
networking.firewall.allowedTCPPorts = [
|
||||
fedicfg.rpc.port
|
||||
];
|
||||
services.garage = {
|
||||
enable = true;
|
||||
package = pkgs.garage_0_9;
|
||||
settings = {
|
||||
replication_mode = "none";
|
||||
# TODO: use a secret file
|
||||
rpc_secret = "d576c4478cc7d0d94cfc127138cbb82018b0155c037d1c827dfb6c36be5f6625";
|
||||
# TODO: why does this have to be set? is there not a sensible default?
|
||||
rpc_bind_addr = "[::]:${toString fedicfg.rpc.port}";
|
||||
rpc_public_addr = "[::1]:${toString fedicfg.rpc.port}";
|
||||
s3_api.api_bind_addr = "[::]:${toString fedicfg.api.port}";
|
||||
s3_web.bind_addr = "[::]:${toString fedicfg.web.internalPort}";
|
||||
s3_web.root_domain = ".${fedicfg.web.rootDomain}";
|
||||
index = "index.html";
|
||||
|
||||
s3_api.s3_region = "garage";
|
||||
s3_api.root_domain = ".${fedicfg.api.domain}";
|
||||
};
|
||||
};
|
||||
|
||||
services.nginx.virtualHosts.${fedicfg.web.rootDomain} = {
|
||||
forceSSL = true;
|
||||
enableACME = true;
|
||||
serverAliases = lib.mapAttrsToList (bucket: _: fedicfg.web.domainForBucket bucket) cfg.ensureBuckets; ## TODO: use wildcard certificates?
|
||||
locations."/" = {
|
||||
proxyPass = "http://localhost:3902";
|
||||
extraConfig = ''
|
||||
proxy_set_header Host $host;
|
||||
'';
|
||||
};
|
||||
};
|
||||
|
||||
systemd.services.ensure-garage = {
|
||||
after = [ "garage.service" ];
|
||||
wantedBy = [ "garage.service" ];
|
||||
serviceConfig = {
|
||||
Type = "oneshot";
|
||||
};
|
||||
path = [ cfg.package pkgs.perl pkgs.awscli ];
|
||||
script = ''
|
||||
set -xeuo pipefail
|
||||
|
||||
# Give Garage time to start up by waiting until somethings speaks HTTP
|
||||
# behind Garage's API URL.
|
||||
until ${pkgs.curl}/bin/curl -sio /dev/null ${fedicfg.api.url}; do sleep 1; done
|
||||
|
||||
# XXX: this is very sensitive to being a single instance
|
||||
# (doing the bare minimum to get garage up and running)
|
||||
# also, it's crazy that we have to parse command output like this
|
||||
# TODO: talk to garage maintainer about making this nicer to work with in Nix
|
||||
# before I do that though, I should figure out how setting it up across multiple machines will work
|
||||
GARAGE_ID=$(garage node id 2>/dev/null | perl -ne '/(.*)@.*/ && print $1')
|
||||
garage layout assign -z g1 -c 1G $GARAGE_ID
|
||||
LAYOUT_VER=$(garage layout show | perl -ne '/Current cluster layout version: (\d*)/ && print $1')
|
||||
garage layout apply --version $((LAYOUT_VER + 1))
|
||||
|
||||
# XXX: this is a hack because we want to write to the buckets here but we're not guaranteed any access keys
|
||||
# TODO: generate this key here rather than using a well-known key
|
||||
# TODO: if the key already exists, we get an error; hacked with this `|| :` which needs to be removed
|
||||
garage key import --yes -n tmp ${snakeoil_key.id} ${snakeoil_key.secret} || :
|
||||
export AWS_ACCESS_KEY_ID=${snakeoil_key.id};
|
||||
export AWS_SECRET_ACCESS_KEY=${snakeoil_key.secret};
|
||||
|
||||
${ensureBucketsScript}
|
||||
${ensureKeysScript}
|
||||
|
||||
# garage doesn't like re-adding keys that once existed, so we can't delete / recreate it every time
|
||||
# garage key delete ${snakeoil_key.id} --yes
|
||||
'';
|
||||
};
|
||||
};
|
||||
}
|
85
services/fediversity/mastodon.nix
Normal file
85
services/fediversity/mastodon.nix
Normal file
|
@ -0,0 +1,85 @@
|
|||
let
|
||||
snakeoil_key = {
|
||||
id = "GK3515373e4c851ebaad366558";
|
||||
secret = "7d37d093435a41f2aab8f13c19ba067d9776c90215f56614adad6ece597dbb34";
|
||||
};
|
||||
in
|
||||
|
||||
{ config, lib, pkgs, ... }:
|
||||
|
||||
lib.mkIf (config.fediversity.enable && config.fediversity.mastodon.enable) {
|
||||
#### garage setup
|
||||
services.garage = {
|
||||
ensureBuckets = {
|
||||
mastodon = {
|
||||
website = true;
|
||||
corsRules = {
|
||||
enable = true;
|
||||
allowedHeaders = [ "*" ];
|
||||
allowedMethods = [ "GET" ];
|
||||
allowedOrigins = [ "*" ];
|
||||
};
|
||||
};
|
||||
};
|
||||
ensureKeys = {
|
||||
mastodon = {
|
||||
inherit (snakeoil_key) id secret;
|
||||
ensureAccess = {
|
||||
mastodon = {
|
||||
read = true;
|
||||
write = true;
|
||||
owner = true;
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
services.mastodon = {
|
||||
extraConfig = rec {
|
||||
S3_ENABLED = "true";
|
||||
# TODO: this shouldn't be hard-coded, it should come from the garage configuration
|
||||
S3_ENDPOINT = config.fediversity.internal.garage.api.url;
|
||||
S3_REGION = "garage";
|
||||
S3_BUCKET = "mastodon";
|
||||
# use <S3_BUCKET>.<S3_ENDPOINT>
|
||||
S3_OVERRIDE_PATH_STLE = "true";
|
||||
AWS_ACCESS_KEY_ID = snakeoil_key.id;
|
||||
AWS_SECRET_ACCESS_KEY = snakeoil_key.secret;
|
||||
S3_PROTOCOL = "http";
|
||||
S3_HOSTNAME = config.fediversity.internal.garage.web.rootDomain;
|
||||
# by default it tries to use "<S3_HOSTNAME>/<S3_BUCKET>"
|
||||
S3_ALIAS_HOST = "${S3_BUCKET}.${S3_HOSTNAME}";
|
||||
# SEE: the last section in https://docs.joinmastodon.org/admin/optional/object-storage/
|
||||
# TODO: can we set up ACLs with garage?
|
||||
S3_PERMISSION = "";
|
||||
};
|
||||
};
|
||||
|
||||
#### mastodon setup
|
||||
|
||||
# open up access to the mastodon web interface
|
||||
networking.firewall.allowedTCPPorts = [ 443 ];
|
||||
|
||||
services.mastodon = {
|
||||
enable = true;
|
||||
|
||||
localDomain = config.fediversity.internal.mastodon.domain;
|
||||
configureNginx = true;
|
||||
|
||||
# TODO: configure a mailserver so this works
|
||||
smtp = {
|
||||
fromAddress = "noreply@${config.fediversity.internal.mastodon.domain}";
|
||||
createLocally = false;
|
||||
};
|
||||
|
||||
# TODO: this is hardware-dependent. let's figure it out when we have hardware
|
||||
# streamingProcesses = 1;
|
||||
};
|
||||
|
||||
security.acme = {
|
||||
acceptTerms = true;
|
||||
preliminarySelfsigned = true;
|
||||
# TODO: configure a mailserver so we can set up acme
|
||||
# defaults.email = "test@example.com";
|
||||
};
|
||||
}
|
97
services/fediversity/peertube.nix
Normal file
97
services/fediversity/peertube.nix
Normal file
|
@ -0,0 +1,97 @@
|
|||
let
|
||||
snakeoil_key = {
|
||||
id = "GK1f9feea9960f6f95ff404c9b";
|
||||
secret = "7295c4201966a02c2c3d25b5cea4a5ff782966a2415e3a196f91924631191395";
|
||||
};
|
||||
in
|
||||
|
||||
{ config, lib, pkgs, ... }:
|
||||
|
||||
lib.mkIf (config.fediversity.enable && config.fediversity.peertube.enable) {
|
||||
networking.firewall.allowedTCPPorts = [ 80 9000 ];
|
||||
|
||||
services.garage = {
|
||||
ensureBuckets = {
|
||||
peertube-videos = {
|
||||
website = true;
|
||||
# TODO: these are too broad, after getting everything works narrow it down to the domain we actually want
|
||||
corsRules = {
|
||||
enable = true;
|
||||
allowedHeaders = [ "*" ];
|
||||
allowedMethods = [ "GET" ];
|
||||
allowedOrigins = [ "*" ];
|
||||
};
|
||||
};
|
||||
# TODO: these are too broad, after getting everything works narrow it down to the domain we actually want
|
||||
peertube-playlists = {
|
||||
website = true;
|
||||
corsRules = {
|
||||
enable = true;
|
||||
allowedHeaders = [ "*" ];
|
||||
allowedMethods = [ "GET" ];
|
||||
allowedOrigins = [ "*" ];
|
||||
};
|
||||
};
|
||||
};
|
||||
ensureKeys = {
|
||||
peertube = {
|
||||
inherit (snakeoil_key) id secret;
|
||||
ensureAccess = {
|
||||
peertube-videos = {
|
||||
read = true;
|
||||
write = true;
|
||||
owner = true;
|
||||
};
|
||||
peertube-playlists = {
|
||||
read = true;
|
||||
write = true;
|
||||
owner = true;
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
services.peertube = {
|
||||
enable = true;
|
||||
localDomain = config.fediversity.internal.peertube.domain;
|
||||
|
||||
# TODO: in most of nixpkgs, these are true by default. upstream that unless there's a good reason not to.
|
||||
redis.createLocally = true;
|
||||
database.createLocally = true;
|
||||
configureNginx = true;
|
||||
|
||||
settings = {
|
||||
object_storage = {
|
||||
enabled = true;
|
||||
endpoint = config.fediversity.internal.garage.api.url;
|
||||
region = "garage";
|
||||
|
||||
# not supported by garage
|
||||
# SEE: https://garagehq.deuxfleurs.fr/documentation/connect/apps/#peertube
|
||||
proxy.proxyify_private_files = false;
|
||||
|
||||
web_videos = rec {
|
||||
bucket_name = "peertube-videos";
|
||||
prefix = "";
|
||||
base_url = config.fediversity.internal.garage.web.urlForBucket bucket_name;
|
||||
};
|
||||
videos = rec {
|
||||
bucket_name = "peertube-videos";
|
||||
prefix = "";
|
||||
base_url = config.fediversity.internal.garage.web.urlForBucket bucket_name;
|
||||
};
|
||||
streaming_playlists = rec {
|
||||
bucket_name = "peertube-playlists";
|
||||
prefix = "";
|
||||
base_url = config.fediversity.internal.garage.web.urlForBucket bucket_name;
|
||||
};
|
||||
};
|
||||
};
|
||||
serviceEnvironmentFile = "/etc/peertube-env";
|
||||
};
|
||||
environment.etc.peertube-env.text = ''
|
||||
AWS_ACCESS_KEY_ID=${snakeoil_key.id}
|
||||
AWS_SECRET_ACCESS_KEY=${snakeoil_key.secret}
|
||||
'';
|
||||
}
|
18
services/fediversity/pixelfed-group-permissions.patch
Normal file
18
services/fediversity/pixelfed-group-permissions.patch
Normal file
|
@ -0,0 +1,18 @@
|
|||
diff --git a/config/filesystems.php b/config/filesystems.php
|
||||
index 00254e93..fc1a58f3 100644
|
||||
--- a/config/filesystems.php
|
||||
+++ b/config/filesystems.php
|
||||
@@ -49,11 +49,11 @@ return [
|
||||
'permissions' => [
|
||||
'file' => [
|
||||
'public' => 0644,
|
||||
- 'private' => 0600,
|
||||
+ 'private' => 0640,
|
||||
],
|
||||
'dir' => [
|
||||
'public' => 0755,
|
||||
- 'private' => 0700,
|
||||
+ 'private' => 0750,
|
||||
],
|
||||
],
|
||||
],
|
84
services/fediversity/pixelfed.nix
Normal file
84
services/fediversity/pixelfed.nix
Normal file
|
@ -0,0 +1,84 @@
|
|||
let
|
||||
snakeoil_key = {
|
||||
id = "GKb5615457d44214411e673b7b";
|
||||
secret = "5be6799a88ca9b9d813d1a806b64f15efa49482dbe15339ddfaf7f19cf434987";
|
||||
};
|
||||
in
|
||||
|
||||
{ config, lib, pkgs, ... }:
|
||||
|
||||
lib.mkIf (config.fediversity.enable && config.fediversity.pixelfed.enable) {
|
||||
services.garage = {
|
||||
ensureBuckets = {
|
||||
pixelfed = {
|
||||
website = true;
|
||||
# TODO: these are too broad, after getting everything works narrow it down to the domain we actually want
|
||||
corsRules = {
|
||||
enable = true;
|
||||
allowedHeaders = [ "*" ];
|
||||
allowedMethods = [ "GET" ];
|
||||
allowedOrigins = [ "*" ];
|
||||
};
|
||||
};
|
||||
};
|
||||
ensureKeys = {
|
||||
pixelfed = {
|
||||
inherit (snakeoil_key) id secret;
|
||||
ensureAccess = {
|
||||
pixelfed = {
|
||||
read = true;
|
||||
write = true;
|
||||
owner = true;
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
services.pixelfed = {
|
||||
enable = true;
|
||||
domain = config.fediversity.internal.pixelfed.domain;
|
||||
|
||||
# TODO: secrets management!!!
|
||||
secretFile = pkgs.writeText "secrets.env" ''
|
||||
APP_KEY=adKK9EcY8Hcj3PLU7rzG9rJ6KKTOtYfA
|
||||
'';
|
||||
|
||||
## Taeer feels like this way of configuring Nginx is odd; there should
|
||||
## instead be a `services.pixefed.nginx.enable` option and the actual Nginx
|
||||
## configuration should be in `services.nginx`. See eg. `pretix`.
|
||||
##
|
||||
## TODO: If that indeed makes sense, upstream.
|
||||
nginx = {
|
||||
forceSSL = true;
|
||||
enableACME = true;
|
||||
# locations."/public/".proxyPass = "${config.fediversity.internal.garage.web.urlForBucket "pixelfed"}/public/";
|
||||
};
|
||||
};
|
||||
|
||||
services.pixelfed.settings = {
|
||||
## NOTE: This depends on the targets, eg. universities might want control
|
||||
## over who has an account. We probably want a universal
|
||||
## `fediversity.openRegistration` option.
|
||||
OPEN_REGISTRATION = true;
|
||||
|
||||
# DANGEROUSLY_SET_FILESYSTEM_DRIVER = "s3";
|
||||
FILESYSTEM_CLOUD = "s3";
|
||||
PF_ENABLE_CLOUD = true;
|
||||
AWS_ACCESS_KEY_ID = snakeoil_key.id;
|
||||
AWS_SECRET_ACCESS_KEY = snakeoil_key.secret;
|
||||
AWS_DEFAULT_REGION = "garage";
|
||||
AWS_URL = config.fediversity.internal.garage.web.urlForBucket "pixelfed";
|
||||
AWS_BUCKET = "pixelfed";
|
||||
AWS_ENDPOINT = config.fediversity.internal.garage.api.url;
|
||||
AWS_USE_PATH_STYLE_ENDPOINT = false;
|
||||
};
|
||||
|
||||
## Only ever run `pixelfed-data-setup` after `ensure-garage` has done its job.
|
||||
## Otherwise, everything crashed dramatically.
|
||||
systemd.services.pixelfed-data-setup = {
|
||||
after = [ "ensure-garage.service" ];
|
||||
};
|
||||
|
||||
networking.firewall.allowedTCPPorts = [ 80 443 ];
|
||||
}
|
96
services/flake.lock
Normal file
96
services/flake.lock
Normal file
|
@ -0,0 +1,96 @@
|
|||
{
|
||||
"nodes": {
|
||||
"disko": {
|
||||
"inputs": {
|
||||
"nixpkgs": "nixpkgs"
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1727347829,
|
||||
"narHash": "sha256-y7cW6TjJKy+tu7efxeWI6lyg4VVx/9whx+OmrhmRShU=",
|
||||
"owner": "nix-community",
|
||||
"repo": "disko",
|
||||
"rev": "1879e48907c14a70302ff5d0539c3b9b6f97feaa",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "nix-community",
|
||||
"repo": "disko",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nixpkgs": {
|
||||
"locked": {
|
||||
"lastModified": 1725194671,
|
||||
"narHash": "sha256-tLGCFEFTB5TaOKkpfw3iYT9dnk4awTP/q4w+ROpMfuw=",
|
||||
"owner": "NixOS",
|
||||
"repo": "nixpkgs",
|
||||
"rev": "b833ff01a0d694b910daca6e2ff4a3f26dee478c",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "NixOS",
|
||||
"ref": "nixpkgs-unstable",
|
||||
"repo": "nixpkgs",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nixpkgs-latest": {
|
||||
"locked": {
|
||||
"lastModified": 1727220152,
|
||||
"narHash": "sha256-6ezRTVBZT25lQkvaPrfJSxYLwqcbNWm6feD/vG1FO0o=",
|
||||
"owner": "nixos",
|
||||
"repo": "nixpkgs",
|
||||
"rev": "24959f933187217890b206788a85bfa73ba75949",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "nixos",
|
||||
"repo": "nixpkgs",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nixpkgs_2": {
|
||||
"locked": {
|
||||
"lastModified": 1723726852,
|
||||
"narHash": "sha256-lRzlx4fPRtzA+dgz9Rh4WK5yAW3TsAXx335DQqxY2XY=",
|
||||
"owner": "radvendii",
|
||||
"repo": "nixpkgs",
|
||||
"rev": "9286249a1673cf5b14a4793e22dd44b70cb69a0d",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "radvendii",
|
||||
"ref": "nixos_rebuild_tests",
|
||||
"repo": "nixpkgs",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"pixelfed": {
|
||||
"flake": false,
|
||||
"locked": {
|
||||
"lastModified": 1719823820,
|
||||
"narHash": "sha256-CKjqnxp7p2z/13zfp4HQ1OAmaoUtqBKS6HFm6TV8Jwg=",
|
||||
"owner": "pixelfed",
|
||||
"repo": "pixelfed",
|
||||
"rev": "4c245cf429330d01fcb8ebeb9aa8c84a9574a645",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "pixelfed",
|
||||
"ref": "v0.12.3",
|
||||
"repo": "pixelfed",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"root": {
|
||||
"inputs": {
|
||||
"disko": "disko",
|
||||
"nixpkgs": "nixpkgs_2",
|
||||
"nixpkgs-latest": "nixpkgs-latest",
|
||||
"pixelfed": "pixelfed"
|
||||
}
|
||||
}
|
||||
},
|
||||
"root": "root",
|
||||
"version": 7
|
||||
}
|
121
services/flake.nix
Normal file
121
services/flake.nix
Normal file
|
@ -0,0 +1,121 @@
|
|||
{
|
||||
inputs = {
|
||||
nixpkgs.url = "github:radvendii/nixpkgs/nixos_rebuild_tests";
|
||||
nixpkgs-latest.url = "github:nixos/nixpkgs";
|
||||
pixelfed = {
|
||||
url = "github:pixelfed/pixelfed?ref=v0.12.3";
|
||||
flake = false;
|
||||
};
|
||||
disko.url = "github:nix-community/disko";
|
||||
};
|
||||
|
||||
outputs = { self, nixpkgs, nixpkgs-latest, pixelfed, disko }:
|
||||
let
|
||||
system = "x86_64-linux";
|
||||
lib = nixpkgs.lib;
|
||||
pkgs = nixpkgs.legacyPackages.${system};
|
||||
pkgsLatest = nixpkgs-latest.legacyPackages.${system};
|
||||
bleedingFediverseOverlay = (self: super: {
|
||||
pixelfed = pkgsLatest.pixelfed.overrideAttrs (old: {
|
||||
src = pixelfed;
|
||||
patches = (old.patches or [ ]) ++ [ ./fediversity/pixelfed-group-permissions.patch ];
|
||||
});
|
||||
## TODO: give mastodon, peertube the same treatment
|
||||
});
|
||||
in {
|
||||
nixosModules = {
|
||||
## Bleeding-edge fediverse packages
|
||||
bleedingFediverse = {
|
||||
nixpkgs.overlays = [ bleedingFediverseOverlay ];
|
||||
};
|
||||
## Fediversity modules
|
||||
fediversity = import ./fediversity;
|
||||
|
||||
## VM-specific modules
|
||||
interactive-vm = import ./vm/interactive-vm.nix;
|
||||
garage-vm = import ./vm/garage-vm.nix;
|
||||
mastodon-vm = import ./vm/mastodon-vm.nix;
|
||||
peertube-vm = import ./vm/peertube-vm.nix;
|
||||
pixelfed-vm = import ./vm/pixelfed-vm.nix;
|
||||
|
||||
disk-layout = import ./disk-layout.nix;
|
||||
};
|
||||
|
||||
nixosConfigurations = {
|
||||
mastodon = nixpkgs.lib.nixosSystem {
|
||||
inherit system;
|
||||
modules = with self.nixosModules; [
|
||||
disko.nixosModules.default
|
||||
disk-layout
|
||||
bleedingFediverse
|
||||
fediversity
|
||||
interactive-vm
|
||||
garage-vm
|
||||
mastodon-vm
|
||||
];
|
||||
};
|
||||
|
||||
peertube = nixpkgs.lib.nixosSystem {
|
||||
inherit system;
|
||||
modules = with self.nixosModules; [
|
||||
disko.nixosModules.default
|
||||
disk-layout
|
||||
bleedingFediverse
|
||||
fediversity
|
||||
interactive-vm
|
||||
garage-vm
|
||||
peertube-vm
|
||||
];
|
||||
};
|
||||
|
||||
pixelfed = nixpkgs.lib.nixosSystem {
|
||||
inherit system;
|
||||
modules = with self.nixosModules; [
|
||||
disko.nixosModules.default
|
||||
disk-layout
|
||||
bleedingFediverse
|
||||
fediversity
|
||||
interactive-vm
|
||||
garage-vm
|
||||
pixelfed-vm
|
||||
];
|
||||
};
|
||||
|
||||
all = nixpkgs.lib.nixosSystem {
|
||||
inherit system;
|
||||
modules = with self.nixosModules; [
|
||||
disko.nixosModules.default
|
||||
disk-layout
|
||||
bleedingFediverse
|
||||
fediversity
|
||||
interactive-vm
|
||||
garage-vm
|
||||
peertube-vm
|
||||
pixelfed-vm
|
||||
mastodon-vm
|
||||
];
|
||||
};
|
||||
};
|
||||
|
||||
## Fully-feature ISO installer
|
||||
mkInstaller = import ./installer.nix;
|
||||
installers = lib.mapAttrs (_: config: self.mkInstaller nixpkgs config) self.nixosConfigurations;
|
||||
|
||||
deploy =
|
||||
let
|
||||
deployCommand = (pkgs.callPackage ./deploy.nix { });
|
||||
in
|
||||
lib.mapAttrs (name: config: deployCommand name config) self.nixosConfigurations;
|
||||
|
||||
checks.${system} = {
|
||||
mastodon-garage = import ./tests/mastodon-garage.nix { inherit pkgs self; };
|
||||
pixelfed-garage = import ./tests/pixelfed-garage.nix { inherit pkgs self; };
|
||||
};
|
||||
|
||||
devShells.${system}.default = pkgs.mkShell {
|
||||
inputs = with pkgs; [
|
||||
nil
|
||||
];
|
||||
};
|
||||
};
|
||||
}
|
39
services/installer.nix
Normal file
39
services/installer.nix
Normal file
|
@ -0,0 +1,39 @@
|
|||
/**
|
||||
Convert a NixOS configuration to one for a minimal installer ISO
|
||||
|
||||
WARNING: Running this installer will format the target disk!
|
||||
*/
|
||||
nixpkgs: machine:
|
||||
let
|
||||
installer = { config, pkgs, lib, ... }:
|
||||
let
|
||||
bootstrap = pkgs.writeShellApplication {
|
||||
name = "bootstrap";
|
||||
runtimeInputs = with pkgs; [ nixos-install-tools ];
|
||||
text = ''
|
||||
${machine.config.system.build.diskoScript}
|
||||
nixos-install --no-root-password --no-channel-copy --system ${machine.config.system.build.toplevel}
|
||||
'';
|
||||
};
|
||||
in
|
||||
{
|
||||
imports = [
|
||||
"${nixpkgs}/nixos/modules/installer/cd-dvd/installation-cd-minimal.nix"
|
||||
];
|
||||
nixpkgs.hostPlatform = "x86_64-linux";
|
||||
services.getty.autologinUser = lib.mkForce "root";
|
||||
programs.bash.loginShellInit = ''
|
||||
${nixpkgs.lib.getExe bootstrap}
|
||||
'';
|
||||
|
||||
isoImage = {
|
||||
compressImage = false;
|
||||
squashfsCompression = "lz4";
|
||||
isoName = lib.mkForce "installer.iso";
|
||||
## ^^ FIXME: Use a more interesting name or keep the default name and
|
||||
## use `isoImage.isoName` in the tests.
|
||||
};
|
||||
};
|
||||
in
|
||||
(nixpkgs.lib.nixosSystem { modules = [installer];}).config.system.build.isoImage
|
||||
|
BIN
services/tests/fediversity.png
Normal file
BIN
services/tests/fediversity.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 5.8 KiB |
BIN
services/tests/green.png
Normal file
BIN
services/tests/green.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 692 B |
144
services/tests/mastodon-garage.nix
Normal file
144
services/tests/mastodon-garage.nix
Normal file
|
@ -0,0 +1,144 @@
|
|||
{ pkgs, self }:
|
||||
let
|
||||
lib = pkgs.lib;
|
||||
rebuildableTest = import ./rebuildableTest.nix pkgs;
|
||||
seleniumScript = pkgs.writers.writePython3Bin "selenium-script"
|
||||
{
|
||||
libraries = with pkgs.python3Packages; [ selenium ];
|
||||
} ''
|
||||
from selenium import webdriver
|
||||
from selenium.webdriver.common.by import By
|
||||
from selenium.webdriver.firefox.options import Options
|
||||
from selenium.webdriver.support.ui import WebDriverWait
|
||||
|
||||
print(1)
|
||||
|
||||
options = Options()
|
||||
options.add_argument("--headless")
|
||||
# devtools don't show up in headless screenshots
|
||||
# options.add_argument("-devtools")
|
||||
service = webdriver.FirefoxService(executable_path="${lib.getExe pkgs.geckodriver}") # noqa: E501
|
||||
|
||||
driver = webdriver.Firefox(options=options, service=service)
|
||||
driver.get("http://mastodon.localhost:55001/public/local")
|
||||
|
||||
# wait until the statuses load
|
||||
WebDriverWait(driver, 90).until(
|
||||
lambda x: x.find_element(By.CLASS_NAME, "status"))
|
||||
|
||||
driver.save_screenshot("/mastodon-screenshot.png")
|
||||
|
||||
driver.close()
|
||||
'';
|
||||
in
|
||||
pkgs.nixosTest {
|
||||
name = "test-mastodon-garage";
|
||||
|
||||
nodes = {
|
||||
server = { config, ... }: {
|
||||
virtualisation.memorySize = lib.mkVMOverride 4096;
|
||||
imports = with self.nixosModules; [
|
||||
bleedingFediverse
|
||||
fediversity
|
||||
garage-vm
|
||||
mastodon-vm
|
||||
];
|
||||
# TODO: pair down
|
||||
environment.systemPackages = with pkgs; [
|
||||
python3
|
||||
firefox-unwrapped
|
||||
geckodriver
|
||||
toot
|
||||
xh
|
||||
seleniumScript
|
||||
helix
|
||||
imagemagick
|
||||
];
|
||||
environment.variables = {
|
||||
POST_MEDIA = ./green.png;
|
||||
AWS_ACCESS_KEY_ID = config.services.garage.ensureKeys.mastodon.id;
|
||||
AWS_SECRET_ACCESS_KEY = config.services.garage.ensureKeys.mastodon.secret;
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
testScript = { nodes, ... }: ''
|
||||
import re
|
||||
import time
|
||||
|
||||
server.start()
|
||||
|
||||
with subtest("Mastodon starts"):
|
||||
server.wait_for_unit("mastodon-web.service")
|
||||
|
||||
# make sure mastodon is fully up and running before we interact with it
|
||||
# TODO: is there a way to test for this?
|
||||
time.sleep(180)
|
||||
|
||||
with subtest("Account creation"):
|
||||
account_creation_output = server.succeed("mastodon-tootctl accounts create test --email test@test.com --confirmed --approve")
|
||||
password_match = re.match('.*New password: ([^\n]*).*', account_creation_output, re.S)
|
||||
if password_match is None:
|
||||
raise Exception(f"account creation did not generate a password.\n{account_creation_output}")
|
||||
password = password_match.group(1)
|
||||
|
||||
with subtest("TTY Login"):
|
||||
server.wait_until_tty_matches("1", "login: ")
|
||||
server.send_chars("root\n");
|
||||
|
||||
with subtest("Log in with toot"):
|
||||
# toot doesn't provide a way to just specify our login details as arguments, so we have to pretend we're typing them in at the prompt
|
||||
server.send_chars("toot login_cli --instance http://mastodon.localhost:55001 --email test@test.com\n")
|
||||
server.wait_until_tty_matches("1", "Password: ")
|
||||
server.send_chars(password + "\n")
|
||||
server.wait_until_tty_matches("1", "Successfully logged in.")
|
||||
|
||||
with subtest("post text"):
|
||||
server.succeed("echo 'hello mastodon' | toot post")
|
||||
|
||||
with subtest("post image"):
|
||||
server.succeed("toot post --media $POST_MEDIA")
|
||||
|
||||
with subtest("access garage"):
|
||||
server.succeed("mc alias set garage ${nodes.server.fediversity.internal.garage.api.url} --api s3v4 --path off $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY")
|
||||
server.succeed("mc ls garage/mastodon")
|
||||
|
||||
with subtest("access image in garage"):
|
||||
image = server.succeed("mc find garage --regex original")
|
||||
image = image.rstrip()
|
||||
if image == "":
|
||||
raise Exception("image posted to mastodon did not get stored in garage")
|
||||
server.succeed(f"mc cat {image} >/garage-image.webp")
|
||||
garage_image_hash = server.succeed("identify -quiet -format '%#' /garage-image.webp")
|
||||
image_hash = server.succeed("identify -quiet -format '%#' $POST_MEDIA")
|
||||
if garage_image_hash != image_hash:
|
||||
raise Exception("image stored in garage did not match image uploaded")
|
||||
|
||||
with subtest("Content security policy allows garage images"):
|
||||
headers = server.succeed("xh -h http://mastodon.localhost:55001/public/local")
|
||||
csp_match = None
|
||||
# I can't figure out re.MULTILINE
|
||||
for header in headers.split("\n"):
|
||||
csp_match = re.match('^Content-Security-Policy: (.*)$', header)
|
||||
if csp_match is not None:
|
||||
break
|
||||
if csp_match is None:
|
||||
raise Exception("mastodon did not send a content security policy header")
|
||||
csp = csp_match.group(1)
|
||||
# the img-src content security policy should include the garage server
|
||||
## TODO: use `nodes.server.fediversity.internal.garage.api.url` same as above, but beware of escaping the regex.
|
||||
garage_csp = re.match(".*; img-src[^;]*web\.garage\.localhost:3902.*", csp)
|
||||
if garage_csp is None:
|
||||
raise Exception("Mastodon's content security policy does not include garage server. image will not be displayed properly on mastodon.")
|
||||
|
||||
# this could in theory give a false positive if mastodon changes it's colorscheme to include pure green.
|
||||
with subtest("image displays"):
|
||||
server.succeed("selenium-script")
|
||||
server.copy_from_vm("/mastodon-screenshot.png", "")
|
||||
displayed_colors = server.succeed("convert /mastodon-screenshot.png -define histogram:unique-colors=true -format %c histogram:info:")
|
||||
# check that the green image displayed somewhere
|
||||
green_check = re.match(".*#00FF00.*", displayed_colors, re.S)
|
||||
if green_check is None:
|
||||
raise Exception("cannot detect the uploaded image on mastodon page.")
|
||||
'';
|
||||
}
|
215
services/tests/pixelfed-garage.nix
Normal file
215
services/tests/pixelfed-garage.nix
Normal file
|
@ -0,0 +1,215 @@
|
|||
{ pkgs, self }:
|
||||
let
|
||||
lib = pkgs.lib;
|
||||
rebuildableTest = import ./rebuildableTest.nix pkgs;
|
||||
|
||||
email = "test@test.com";
|
||||
password = "testtest";
|
||||
|
||||
# FIXME: Replace all the By.XPATH by By.CSS_SELECTOR.
|
||||
|
||||
seleniumImports = ''
|
||||
import sys
|
||||
from selenium import webdriver
|
||||
from selenium.webdriver.common.by import By
|
||||
from selenium.webdriver.chrome.options import Options
|
||||
'';
|
||||
|
||||
seleniumSetup = ''
|
||||
print("Create and configure driver...", file=sys.stderr)
|
||||
options = Options()
|
||||
# options.add_argument("--headless=new")
|
||||
service = webdriver.ChromeService(executable_path="${lib.getExe pkgs.chromedriver}") # noqa: E501
|
||||
driver = webdriver.Chrome(options=options, service=service)
|
||||
driver.implicitly_wait(30)
|
||||
driver.set_window_size(1280, 960)
|
||||
'';
|
||||
|
||||
seleniumPixelfedLogin = ''
|
||||
print("Open login page...", file=sys.stderr)
|
||||
driver.get("http://pixelfed.localhost/login")
|
||||
print("Enter email...", file=sys.stderr)
|
||||
driver.find_element(By.ID, "email").send_keys("${email}")
|
||||
print("Enter password...", file=sys.stderr)
|
||||
driver.find_element(By.ID, "password").send_keys("${password}")
|
||||
# FIXME: This is disgusting. Find instead the input type submit in the form
|
||||
# with action ending in "/login".
|
||||
print("Click “Login” button...", file=sys.stderr)
|
||||
driver.find_element(By.XPATH, "//button[normalize-space()='Login']").click()
|
||||
'';
|
||||
|
||||
## NOTE: `path` must be a valid python string, either a variable or _quoted_.
|
||||
seleniumTakeScreenshot = path: ''
|
||||
print("Take screenshot...", file=sys.stderr)
|
||||
if not driver.save_screenshot(${path}):
|
||||
raise Exception("selenium could not save screenshot")
|
||||
'';
|
||||
|
||||
seleniumQuit = ''
|
||||
print("Quitting...", file=sys.stderr)
|
||||
driver.quit()
|
||||
'';
|
||||
|
||||
seleniumScriptPostPicture = pkgs.writers.writePython3Bin "selenium-script-post-picture"
|
||||
{
|
||||
libraries = with pkgs.python3Packages; [ selenium ];
|
||||
} ''
|
||||
import os
|
||||
import time
|
||||
${seleniumImports}
|
||||
from selenium.webdriver.support.wait import WebDriverWait
|
||||
|
||||
${seleniumSetup}
|
||||
${seleniumPixelfedLogin}
|
||||
time.sleep(3)
|
||||
|
||||
media_path = os.environ['POST_MEDIA']
|
||||
|
||||
# Find the new post form, fill it in with our pictureand a caption.
|
||||
print("Click on “Create New Post”...", file=sys.stderr)
|
||||
driver.find_element(By.LINK_TEXT, "Create New Post").click()
|
||||
print("Add file to input element...", file=sys.stderr)
|
||||
driver.find_element(By.XPATH, "//input[@type='file']").send_keys(media_path)
|
||||
print("Add a caption", file=sys.stderr)
|
||||
driver.find_element(By.CSS_SELECTOR, ".media-body textarea").send_keys(
|
||||
"Fediversity test of image upload to pixelfed with garage storage."
|
||||
)
|
||||
time.sleep(3)
|
||||
print("Click on “Post” button...", file=sys.stderr)
|
||||
driver.find_element(By.LINK_TEXT, "Post").click()
|
||||
|
||||
# Wait until the post loads, and in particular its picture, then take a
|
||||
# screenshot of the whole page.
|
||||
print("Wait for post and image to be loaded...", file=sys.stderr)
|
||||
img = driver.find_element(
|
||||
By.XPATH,
|
||||
"//div[@class='timeline-status-component-content']//img"
|
||||
)
|
||||
WebDriverWait(driver, timeout=10).until(
|
||||
lambda d: d.execute_script("return arguments[0].complete", img)
|
||||
)
|
||||
time.sleep(3)
|
||||
|
||||
${seleniumTakeScreenshot "\"/home/selenium/screenshot.png\""}
|
||||
${seleniumQuit}'';
|
||||
|
||||
seleniumScriptGetSrc = pkgs.writers.writePython3Bin "selenium-script-get-src"
|
||||
{
|
||||
libraries = with pkgs.python3Packages; [ selenium ];
|
||||
} ''
|
||||
${seleniumImports}
|
||||
${seleniumSetup}
|
||||
${seleniumPixelfedLogin}
|
||||
|
||||
img = driver.find_element(
|
||||
By.XPATH,
|
||||
"//div[@class='timeline-status-component-content']//img"
|
||||
)
|
||||
# REVIEW: Need to wait for it to be loaded?
|
||||
print(img.get_attribute('src'))
|
||||
|
||||
${seleniumQuit}'';
|
||||
|
||||
in
|
||||
pkgs.nixosTest {
|
||||
name = "test-pixelfed-garage";
|
||||
|
||||
nodes = {
|
||||
server = { config, ... }: {
|
||||
|
||||
services = {
|
||||
xserver = {
|
||||
enable = true;
|
||||
displayManager.lightdm.enable = true;
|
||||
desktopManager.lxqt.enable = true;
|
||||
};
|
||||
|
||||
displayManager.autoLogin = {
|
||||
enable = true;
|
||||
user = "selenium";
|
||||
};
|
||||
};
|
||||
virtualisation.resolution = { x = 1680; y = 1050; };
|
||||
|
||||
|
||||
virtualisation = {
|
||||
memorySize = lib.mkVMOverride 8192;
|
||||
cores = 8;
|
||||
};
|
||||
imports = with self.nixosModules; [
|
||||
bleedingFediverse
|
||||
fediversity
|
||||
garage-vm
|
||||
pixelfed-vm
|
||||
];
|
||||
# TODO: pair down
|
||||
environment.systemPackages = with pkgs; [
|
||||