Fediversity/simple-nixos-fediverse
Archived
7
1
Fork 2
Code Issues 23 Pull requests 1 Actions Projects Wiki Activity

nixos-test-pixelfed-wip #22

Merged
taeer merged 11 commits from nixos-test-pixelfed-wip into main 2024-09-20 17:47:21 +02:00
Conversation 0 Commits 11 Files changed 13 +231 -94
13 changed files with 231 additions and 94 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

103
fediversity/default.nix Normal file
View file

@ -0,0 +1,103 @@
{ lib, config, ... }:
let
inherit (builtins) toString;
inherit (lib) mkOption mkEnableOption;
inherit (lib.types) types;
in {
imports = [
./garage.nix
./mastodon.nix
./pixelfed.nix
./peertube.nix
];
options = {
fediversity = {
enable = mkEnableOption "the collection of services bundled under Fediversity";
domain = mkOption {
type = types.str;
Niols marked this conversation as resolved Outdated
taeer commented 2024-09-17 22:44:05 +02:00
Outdated
Review
Copy link

This can just be

fediversity = mkEnableOption "the collection of services bundled under fediversity";
This can just be ``` fediversity = mkEnableOption "the collection of services bundled under fediversity"; ```
Niols commented 2024-09-20 16:34:32 +02:00
Outdated
Review
Copy link

Thanks! Done in 2ff8975b6b.

Thanks! Done in 2ff8975b6b2f19d5ffbd7574bd76494bf532d048.
description = ''
root domain for the Fediversity services
Niols marked this conversation as resolved Outdated
taeer commented 2024-09-17 22:49:35 +02:00
Outdated
Review
Copy link

You almost never need types.anything. And certainly in this case the set of sub-options should be defined. This can be done with simple attrsets

garage = {
  api = {
    url = mkOption {
      ...
    };
  };
};

Also

We don't need to have a very clear sense of which options belong where yet, but eventually I think the top-level fediversity options should be the ones that should be set publicly (via NixPanel), and probably the ports being used in garage shouldn't be among them.

I'm not sure what the right namespace to use for those options is... maybe fediversity-private or fediversity.private...

You almost never need `types.anything`. And certainly in this case the set of sub-options should be defined. This can be done with simple attrsets ``` garage = { api = { url = mkOption { ... }; }; }; ``` --- Also We don't need to have a very clear sense of which options belong where **yet**, but eventually I think the top-level `fediversity` options should be the ones that should be set publicly (via NixPanel), and probably the ports being used in garage shouldn't be among them. I'm not sure what the right namespace to use for those options is... maybe `fediversity-private` or `fediversity.private`...
Niols commented 2024-09-20 17:16:23 +02:00
Outdated
Review
Copy link

Done in 73939b9d87.

Done in 73939b9d8752ed4193ebae1b865c306d8eae4971.
For instance, if this option is set to `foo.example.com`, then
Pixelfed might be under `pixelfed.foo.example.com`.
'';
};
Niols marked this conversation as resolved Outdated
taeer commented 2024-09-17 22:53:40 +02:00
Outdated
Review
Copy link

should be types.str
types.string was being used for too many thing so it got deprecated and now there are several different string types. types.str is the one where you don't want to merge definitions if it's defined in more than one place

should be `types.str` `types.string` was being used for too many thing so it got deprecated and now there are several different string types. `types.str` is the one where you don't want to merge definitions if it's defined in more than one place
Niols commented 2024-09-20 16:36:10 +02:00
Outdated
Review
Copy link

Done in d97772ccc4.

Done in d97772ccc4008417aec2117d62031e9123d8deba.
mastodon.enable = mkEnableOption "default Fediversity Mastodon configuration";
pixelfed.enable = mkEnableOption "default Fediversity Pixelfed configuration";
peertube.enable = mkEnableOption "default Fediversity PeerTube configuration";
Niols marked this conversation as resolved Outdated
taeer commented 2024-09-17 22:44:39 +02:00
Outdated
Review
Copy link

Similarly, mkEnableOption here

Similarly, `mkEnableOption` here
internal = mkOption {
description = "options that are only meant to be used internally; change at your own risk";
default = {};
type = types.submodule {
Niols marked this conversation as resolved Outdated
taeer commented 2024-09-17 23:05:23 +02:00
Outdated
Review
Copy link

This should probably be under a call to lib.mkDefault. Or simply put behind the default argument of mkOption. It seems to be the custom (where possible---sometimes it's not) to put it under the default argument.

This should probably be under a call to `lib.mkDefault`. Or simply put behind the `default` argument of `mkOption`. It seems to be the custom (where possible---sometimes it's not) to put it under the `default` argument.
Niols commented 2024-09-20 17:16:33 +02:00
Outdated
Review
Copy link

Done in 73939b9d87.

Done in 73939b9d8752ed4193ebae1b865c306d8eae4971.
options = {
garage = {
api = {
domain = mkOption {
type = types.str;
default = "s3.garage.${config.fediversity.domain}";
};
port = mkOption {
type = types.int;
default = 3900;
};
url = mkOption {
type = types.str;
default = "http://${config.fediversity.internal.garage.api.domain}:${toString config.fediversity.internal.garage.api.port}";
};
};
rpc = {
port = mkOption {
type = types.int;
default = 3901;
};
taeer commented 2024-09-17 23:00:52 +02:00
Outdated
Review
Copy link

there's a style decision here between

  1. defining our own top-level e.g. fediversity.pixelfed.domain, and
  2. referring to e.g. services.pixelfed.domain everywhere.

(1) has the advantage of grouping everything in one place and under one naming scheme, as opposed to needing to know that e.g. mastodon's is under services.mastodon.localDomain instead.
It has the disadvantage of having basically a duplicate option, that really should always be exactly identical to another option. Now if someone sets services.pixelfed.domain somewhere, it won't get propagated to all the other places that use it, because they refer to fediversity.pixelfed.domain. It's also hard to know where to stop with option (1), since you could recategorize all the options under some different scheme at the top level, which might be somewhat more consistent.

I tend to go with option (2), but in a tightly controlled module like we're planning, I don't think it's objectively the right choice, just a style difference as I said.

In the case of the garage options above, we sort of have the same question. But in that case there's information we can't easily get out of the options that are available in services.garage (e.g. just the port itself), so I think it does make sense to define our own options.

Though we should probably discuss with the maintainer of the nixpkgs module if it makes sense to upstream those changes.

there's a style decision here between 1. defining our own top-level e.g. `fediversity.pixelfed.domain`, and 2. referring to e.g. `services.pixelfed.domain` everywhere. (1) has the advantage of grouping everything in one place and under one naming scheme, as opposed to needing to know that e.g. mastodon's is under `services.mastodon.localDomain` instead. It has the disadvantage of having basically a duplicate option, that really should always be exactly identical to another option. Now if someone sets `services.pixelfed.domain` somewhere, it won't get propagated to all the other places that use it, because they refer to `fediversity.pixelfed.domain`. It's also hard to know where to stop with option (1), since you could recategorize all the options under some different scheme at the top level, which might be somewhat more consistent. I tend to go with option (2), but in a tightly controlled module like we're planning, I don't think it's objectively the right choice, just a style difference as I said. --- In the case of the `garage` options above, we sort of have the same question. But in that case there's information we can't easily get out of the options that are available in `services.garage` (e.g. just the port itself), so I think it does make sense to define our own options. Though we should probably discuss with the maintainer of the nixpkgs module if it makes sense to upstream those changes.
taeer commented 2024-09-17 23:01:22 +02:00
Outdated
Review
Copy link

Though if you are going to define fediversity.pixelfed.domain, it does have to be declared as an option somewhere 😆

Though if you are going to define `fediversity.pixelfed.domain`, it does have to be declared as an option somewhere 😆
};
web = {
rootDomain = mkOption {
type = types.str;
default = "web.garage.${config.fediversity.domain}";
};
port = mkOption {
type = types.int;
default = 3902;
};
rootDomainAndPort = mkOption {
type = types.str;
default = "${config.fediversity.internal.garage.web.rootDomain}:${toString config.fediversity.internal.garage.web.port}";
};
urlFor = mkOption {
type = types.functionTo types.str;
default = bucket: "http://${bucket}.${config.fediversity.internal.garage.web.rootDomainAndPort}";
};
};
};
## REVIEW: Do we want to recreate options under
## `fediversity.internal` or would we rather use the options from
## the respective services? See Taeer's comment:
## https://git.fediversity.eu/taeer/simple-nixos-fediverse/pulls/22#issuecomment-124
pixelfed.domain = mkOption {
type = types.str;
default = "pixelfed.${config.fediversity.domain}";
};
mastodon.domain = mkOption {
type = types.str;
default = "mastdodon.${config.fediversity.domain}";
};
peertube.domain = mkOption {
type = types.str;
default = "peertube.${config.fediversity.domain}";
};
};
};
};
};
};
}

42
garage.nix → fediversity/garage.nix
View file

@ -6,9 +6,12 @@ let
secret = "82b2b4cbef27bf8917b350d5b10a87c92fa9c8b13a415aeeea49726cf335d74e"; secret = "82b2b4cbef27bf8917b350d5b10a87c92fa9c8b13a415aeeea49726cf335d74e";
}; };
in in
# TODO: expand to a multi-machine setup # TODO: expand to a multi-machine setup
{ config, lib, pkgs, ... }: { config, lib, pkgs, ... }:
let let
inherit (builtins) toString;
inherit (lib) types mkOption mkEnableOption optionalString concatStringsSep; inherit (lib) types mkOption mkEnableOption optionalString concatStringsSep;
inherit (lib.strings) escapeShellArg; inherit (lib.strings) escapeShellArg;
cfg = config.services.garage; cfg = config.services.garage;
@ -39,7 +42,7 @@ let
${optionalString corsRules.enable '' ${optionalString corsRules.enable ''
garage bucket allow --read --write --owner ${bucketArg} --key tmp garage bucket allow --read --write --owner ${bucketArg} --key tmp
# TODO: endpoin-url should not be hard-coded # TODO: endpoin-url should not be hard-coded
aws --region ${cfg.settings.s3_api.s3_region} --endpoint-url http://s3.garage.localhost:3900 s3api put-bucket-cors --bucket ${bucketArg} --cors-configuration ${corsRulesJSON} aws --region ${cfg.settings.s3_api.s3_region} --endpoint-url ${config.fediversity.internal.garage.api.url} s3api put-bucket-cors --bucket ${bucketArg} --cors-configuration ${corsRulesJSON}
garage bucket deny --read --write --owner ${bucketArg} --key tmp garage bucket deny --read --write --owner ${bucketArg} --key tmp
''} ''}
''; '';
@ -53,7 +56,9 @@ let
${concatMapAttrs (ensureAccessScriptFn key) ensureAccess} ${concatMapAttrs (ensureAccessScriptFn key) ensureAccess}
''; '';
ensureKeysScript = concatMapAttrs ensureKeyScriptFn cfg.ensureKeys; ensureKeysScript = concatMapAttrs ensureKeyScriptFn cfg.ensureKeys;
in { in
{
# add in options to ensure creation of buckets and keys # add in options to ensure creation of buckets and keys
options = { options = {
services.garage = { services.garage = {
@ -126,24 +131,27 @@ in {
}; };
}; };
config = { config = lib.mkIf config.fediversity.enable {
virtualisation.diskSize = 2048; virtualisation.diskSize = 2048;
virtualisation.forwardPorts = [ virtualisation.forwardPorts = [
{ {
from = "host"; from = "host";
host.port = 3901; host.port = config.fediversity.internal.garage.rpc.port;
guest.port = 3901; guest.port = config.fediversity.internal.garage.rpc.port;
} }
{ {
from = "host"; from = "host";
host.port = 3902; host.port = config.fediversity.internal.garage.web.port;
guest.port = 3902; guest.port = config.fediversity.internal.garage.web.port;
} }
]; ];
environment.systemPackages = [ pkgs.minio-client pkgs.awscli ]; environment.systemPackages = [ pkgs.minio-client pkgs.awscli ];
networking.firewall.allowedTCPPorts = [ 3901 3902 ]; networking.firewall.allowedTCPPorts = [
config.fediversity.internal.garage.rpc.port
config.fediversity.internal.garage.web.port
];
services.garage = { services.garage = {
enable = true; enable = true;
package = pkgs.garage_0_9; package = pkgs.garage_0_9;
@ -152,15 +160,15 @@ in {
# TODO: use a secret file # TODO: use a secret file
rpc_secret = "d576c4478cc7d0d94cfc127138cbb82018b0155c037d1c827dfb6c36be5f6625"; rpc_secret = "d576c4478cc7d0d94cfc127138cbb82018b0155c037d1c827dfb6c36be5f6625";
# TODO: why does this have to be set? is there not a sensible default? # TODO: why does this have to be set? is there not a sensible default?
rpc_bind_addr = "[::]:3901"; rpc_bind_addr = "[::]:${toString config.fediversity.internal.garage.rpc.port}";
rpc_public_addr = "[::1]:3901"; rpc_public_addr = "[::1]:${toString config.fediversity.internal.garage.rpc.port}";
s3_api.api_bind_addr = "[::]:3900"; s3_api.api_bind_addr = "[::]:${toString config.fediversity.internal.garage.api.port}";
s3_web.bind_addr = "[::]:3902"; s3_web.bind_addr = "[::]:${toString config.fediversity.internal.garage.web.port}";
s3_web.root_domain = ".web.garage.localhost"; s3_web.root_domain = ".${config.fediversity.internal.garage.web.rootDomain}";
index = "index.html"; index = "index.html";
s3_api.s3_region = "garage"; s3_api.s3_region = "garage";
s3_api.root_domain = ".s3.garage.localhost"; s3_api.root_domain = ".${config.fediversity.internal.garage.api.domain}";
}; };
}; };
systemd.services.ensure-garage = { systemd.services.ensure-garage = {
@ -173,9 +181,9 @@ in {
script = '' script = ''
set -xeuo pipefail set -xeuo pipefail
# Give garage time to start up by waiting until somethings speaks HTTP # Give Garage time to start up by waiting until somethings speaks HTTP
# behind localhost:3900. # behind Garage's API URL.
until curl -sio /dev/null http://localhost:3900/; do sleep 1; done until ${pkgs.curl}/bin/curl -sio /dev/null ${config.fediversity.internal.garage.api.url}; do sleep 1; done
# XXX: this is very sensitive to being a single instance # XXX: this is very sensitive to being a single instance
# (doing the bare minimum to get garage up and running) # (doing the bare minimum to get garage up and running)

18
mastodon.nix → fediversity/mastodon.nix
View file

@ -4,7 +4,10 @@ let
secret = "7d37d093435a41f2aab8f13c19ba067d9776c90215f56614adad6ece597dbb34"; secret = "7d37d093435a41f2aab8f13c19ba067d9776c90215f56614adad6ece597dbb34";
}; };
in in
{ config, lib, pkgs, ... }: {
{ config, lib, pkgs, ... }:
lib.mkIf (config.fediversity.enable && config.fediversity.mastodon.enable) {
#### garage setup #### garage setup
services.garage = { services.garage = {
ensureBuckets = { ensureBuckets = {
@ -35,7 +38,7 @@ in
extraConfig = rec { extraConfig = rec {
S3_ENABLED = "true"; S3_ENABLED = "true";
# TODO: this shouldn't be hard-coded, it should come from the garage configuration # TODO: this shouldn't be hard-coded, it should come from the garage configuration
S3_ENDPOINT = "http://s3.garage.localhost:3900"; S3_ENDPOINT = config.fediversity.internal.garage.api.url;
S3_REGION = "garage"; S3_REGION = "garage";
S3_BUCKET = "mastodon"; S3_BUCKET = "mastodon";
# use <S3_BUCKET>.<S3_ENDPOINT> # use <S3_BUCKET>.<S3_ENDPOINT>
@ -43,7 +46,7 @@ in
AWS_ACCESS_KEY_ID = snakeoil_key.id; AWS_ACCESS_KEY_ID = snakeoil_key.id;
AWS_SECRET_ACCESS_KEY = snakeoil_key.secret; AWS_SECRET_ACCESS_KEY = snakeoil_key.secret;
S3_PROTOCOL = "http"; S3_PROTOCOL = "http";
S3_HOSTNAME = "web.garage.localhost:3902"; S3_HOSTNAME = config.fediversity.internal.garage.web.rootDomainAndPort;
# by default it tries to use "<S3_HOSTNAME>/<S3_BUCKET>" # by default it tries to use "<S3_HOSTNAME>/<S3_BUCKET>"
S3_ALIAS_HOST = "${S3_BUCKET}.${S3_HOSTNAME}"; S3_ALIAS_HOST = "${S3_BUCKET}.${S3_HOSTNAME}";
# SEE: the last section in https://docs.joinmastodon.org/admin/optional/object-storage/ # SEE: the last section in https://docs.joinmastodon.org/admin/optional/object-storage/
@ -60,12 +63,14 @@ in
services.mastodon = { services.mastodon = {
enable = true; enable = true;
# TODO: set up a domain name, and a DNS service so that this can run not in a vm localDomain = config.fediversity.internal.mastodon.domain;
# localDomain = "domain.social";
configureNginx = true; configureNginx = true;
# TODO: configure a mailserver so this works # TODO: configure a mailserver so this works
# smtp.fromAddress = "mastodon@domain.social"; smtp = {
fromAddress = "noreply@${config.fediversity.internal.mastodon.domain}";
createLocally = false;
};
# TODO: this is hardware-dependent. let's figure it out when we have hardware # TODO: this is hardware-dependent. let's figure it out when we have hardware
# streamingProcesses = 1; # streamingProcesses = 1;
@ -78,4 +83,3 @@ in
# defaults.email = "test@example.com"; # defaults.email = "test@example.com";
}; };
} }

27
peertube.nix → fediversity/peertube.nix
View file

@ -4,7 +4,10 @@ let
secret = "7295c4201966a02c2c3d25b5cea4a5ff782966a2415e3a196f91924631191395"; secret = "7295c4201966a02c2c3d25b5cea4a5ff782966a2415e3a196f91924631191395";
}; };
in in
{ config, lib, pkgs, ... }: {
{ config, lib, pkgs, ... }:
lib.mkIf (config.fediversity.enable && config.fediversity.peertube.enable) {
networking.firewall.allowedTCPPorts = [ 80 9000 ]; networking.firewall.allowedTCPPorts = [ 80 9000 ];
services.garage = { services.garage = {
@ -50,30 +53,38 @@ in
}; };
services.peertube = { services.peertube = {
enable = true;
localDomain = config.fediversity.internal.peertube.domain;
# TODO: in most of nixpkgs, these are true by default. upstream that unless there's a good reason not to.
redis.createLocally = true;
database.createLocally = true;
configureNginx = true;
settings = { settings = {
object_storage = { object_storage = {
enabled = true; enabled = true;
endpoint = "http://s3.garage.localhost:3900"; endpoint = config.fediversity.internal.garage.api.url;
region = "garage"; region = "garage";
# not supported by garage # not supported by garage
# SEE: https://garagehq.deuxfleurs.fr/documentation/connect/apps/#peertube # SEE: https://garagehq.deuxfleurs.fr/documentation/connect/apps/#peertube
proxy.proxyify_private_files = false; proxy.proxyify_private_files = false;
web_videos = { web_videos = rec {
bucket_name = "peertube-videos"; bucket_name = "peertube-videos";
prefix = ""; prefix = "";
base_url = "http://peertube-videos.web.garage.localhost:3902"; base_url = config.fediversity.internal.garage.web.urlFor bucket_name;
}; };
videos = { videos = rec {
bucket_name = "peertube-videos"; bucket_name = "peertube-videos";
prefix = ""; prefix = "";
base_url = "http://peertube-videos.web.garage.localhost:3902"; base_url = config.fediversity.internal.garage.web.urlFor bucket_name;
}; };
streaming_playlists = { streaming_playlists = rec {
bucket_name = "peertube-playlists"; bucket_name = "peertube-playlists";
prefix = ""; prefix = "";
base_url = "http://peertube-playlists.web.garage.localhost:3902"; base_url = config.fediversity.internal.garage.web.urlFor bucket_name;
}; };
}; };
}; };

0
pixelfed-group-permissions.patch → fediversity/pixelfed-group-permissions.patch
View file

15
pixelfed.nix → fediversity/pixelfed.nix
View file

@ -4,7 +4,10 @@ let
secret = "5be6799a88ca9b9d813d1a806b64f15efa49482dbe15339ddfaf7f19cf434987"; secret = "5be6799a88ca9b9d813d1a806b64f15efa49482dbe15339ddfaf7f19cf434987";
}; };
in in
{ config, lib, pkgs, ... }: {
{ config, lib, pkgs, ... }:
lib.mkIf (config.fediversity.enable && config.fediversity.pixelfed.enable) {
services.garage = { services.garage = {
ensureBuckets = { ensureBuckets = {
pixelfed = { pixelfed = {
@ -32,9 +35,11 @@ in
}; };
}; };
services.pixelfed.enable = true; services.pixelfed = {
enable = true;
domain = config.fediversity.internal.pixelfed.domain;
};
# TODO: factor these out so we're only defining e.g. s3.garage.localhost and port 3900 in one place
services.pixelfed.settings = { services.pixelfed.settings = {
# DANGEROUSLY_SET_FILESYSTEM_DRIVER = "s3"; # DANGEROUSLY_SET_FILESYSTEM_DRIVER = "s3";
FILESYSTEM_CLOUD = "s3"; FILESYSTEM_CLOUD = "s3";
@ -42,9 +47,9 @@ in
AWS_ACCESS_KEY_ID = snakeoil_key.id; AWS_ACCESS_KEY_ID = snakeoil_key.id;
AWS_SECRET_ACCESS_KEY = snakeoil_key.secret; AWS_SECRET_ACCESS_KEY = snakeoil_key.secret;
AWS_DEFAULT_REGION = "garage"; AWS_DEFAULT_REGION = "garage";
AWS_URL = "http://pixelfed.web.garage.localhost:3902/"; AWS_URL = config.fediversity.internal.garage.web.urlFor "pixelfed";
AWS_BUCKET = "pixelfed"; AWS_BUCKET = "pixelfed";
AWS_ENDPOINT = "http://s3.garage.localhost:3900"; AWS_ENDPOINT = config.fediversity.internal.garage.api.url;
AWS_USE_PATH_STYLE_ENDPOINT = false; AWS_USE_PATH_STYLE_ENDPOINT = false;
}; };

30
flake.nix
View file

@ -12,40 +12,40 @@
in { in {
nixosModules = { nixosModules = {
interactive-vm = import ./interactive-vm.nix; ## Fediversity modules
mastodon = import ./mastodon.nix; fediversity = import ./fediversity;
mastodon-vm = import ./mastodon-vm.nix;
peertube = import ./peertube.nix; ## VM-specific modules
peertube-vm = import ./peertube-vm.nix; interactive-vm = import ./vm/interactive-vm.nix;
pixelfed = import ./pixelfed.nix; mastodon-vm = import ./vm/mastodon-vm.nix;
pixelfed-vm = import ./pixelfed-vm.nix; peertube-vm = import ./vm/peertube-vm.nix;
garage = import ./garage.nix; pixelfed-vm = import ./vm/pixelfed-vm.nix;
}; };
nixosConfigurations = { nixosConfigurations = {
mastodon = nixpkgs.lib.nixosSystem { mastodon = nixpkgs.lib.nixosSystem {
inherit system; inherit system;
modules = with self.nixosModules; [ interactive-vm mastodon mastodon-vm garage ]; modules = with self.nixosModules; [ fediversity interactive-vm mastodon-vm ];
}; };
peertube = nixpkgs.lib.nixosSystem { peertube = nixpkgs.lib.nixosSystem {
inherit system; inherit system;
modules = with self.nixosModules; [ interactive-vm peertube peertube-vm garage ]; modules = with self.nixosModules; [ fediversity interactive-vm peertube-vm ];
}; };
pixelfed = nixpkgs.lib.nixosSystem { pixelfed = nixpkgs.lib.nixosSystem {
inherit system; inherit system;
modules = with self.nixosModules; [ interactive-vm pixelfed pixelfed-vm garage ]; modules = with self.nixosModules; [ fediversity interactive-vm pixelfed-vm ];
}; };
all = nixpkgs.lib.nixosSystem { all = nixpkgs.lib.nixosSystem {
inherit system; inherit system;
modules = with self.nixosModules; [ modules = with self.nixosModules; [
fediversity
interactive-vm interactive-vm
peertube peertube-vm peertube-vm
pixelfed pixelfed-vm pixelfed-vm
mastodon mastodon-vm mastodon-vm
garage
]; ];
}; };
}; };

7
tests/mastodon-garage.nix
View file

@ -37,7 +37,7 @@ pkgs.nixosTest {
nodes = { nodes = {
server = { config, ... }: { server = { config, ... }: {
virtualisation.memorySize = lib.mkVMOverride 4096; virtualisation.memorySize = lib.mkVMOverride 4096;
imports = with self.nixosModules; [ garage mastodon mastodon-vm ]; imports = with self.nixosModules; [ mastodon-vm ];
# TODO: pair down # TODO: pair down
environment.systemPackages = with pkgs; [ environment.systemPackages = with pkgs; [
python3 python3
@ -57,7 +57,7 @@ pkgs.nixosTest {
}; };
}; };
testScript = '' testScript = { nodes, ... }: ''
import re import re
import time import time
@ -95,7 +95,7 @@ pkgs.nixosTest {
server.succeed("toot post --media $POST_MEDIA") server.succeed("toot post --media $POST_MEDIA")
with subtest("access garage"): with subtest("access garage"):
server.succeed("mc alias set garage http://s3.garage.localhost:3900 --api s3v4 --path off $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY") server.succeed("mc alias set garage ${nodes.server.fediversity.internal.garage.api.url} --api s3v4 --path off $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY")
server.succeed("mc ls garage/mastodon") server.succeed("mc ls garage/mastodon")
taeer commented 2024-09-17 23:15:39 +02:00
Review
Copy link

Yes! In typical nix dynamically-typed fashion, testScript can also be a function! And we can have

testScript = { nodes, ... }: ''
...
   ${nodes.server.fediversity.garage.api.url}
...
'';
Yes! In typical nix dynamically-typed fashion, `testScript` can also be a function! And we can have ``` testScript = { nodes, ... }: '' ... ${nodes.server.fediversity.garage.api.url} ... ''; ```
with subtest("access image in garage"): with subtest("access image in garage"):
@ -121,6 +121,7 @@ pkgs.nixosTest {
raise Exception("mastodon did not send a content security policy header") raise Exception("mastodon did not send a content security policy header")
csp = csp_match.group(1) csp = csp_match.group(1)
# the img-src content security policy should include the garage server # the img-src content security policy should include the garage server
## TODO: use `nodes.server.fediversity.internal.garage.api.url` same as above, but beware of escaping the regex.
garage_csp = re.match(".*; img-src[^;]*web\.garage\.localhost:3902.*", csp) garage_csp = re.match(".*; img-src[^;]*web\.garage\.localhost:3902.*", csp)
taeer commented 2024-09-17 23:16:01 +02:00
Outdated
Review
Copy link

Yes, though we need to be careful about escaping for the regex

Yes, though we need to be careful about escaping for the regex
if garage_csp is None: if garage_csp is None:
raise Exception("Mastodon's content security policy does not include garage server. image will not be displayed properly on mastodon.") raise Exception("Mastodon's content security policy does not include garage server. image will not be displayed properly on mastodon.")

27
tests/pixelfed-garage.nix
View file

@ -29,15 +29,16 @@ let
print("Open login page...", file=sys.stderr) print("Open login page...", file=sys.stderr)
driver.get("http://pixelfed.localhost/login") driver.get("http://pixelfed.localhost/login")
print("Enter email...", file=sys.stderr) print("Enter email...", file=sys.stderr)
driver.find_element(By.ID, "email").send_keys(${email}) driver.find_element(By.ID, "email").send_keys("${email}")
print("Enter password...", file=sys.stderr) print("Enter password...", file=sys.stderr)
driver.find_element(By.ID, "password").send_keys(${password}) driver.find_element(By.ID, "password").send_keys("${password}")
# FIXME: This is disgusting. Find instead the input type submit in the form # FIXME: This is disgusting. Find instead the input type submit in the form
# with action ending in "/login". # with action ending in "/login".
print("Click “Login” button...", file=sys.stderr) print("Click “Login” button...", file=sys.stderr)
driver.find_element(By.XPATH, "//button[normalize-space()='Login']").click() driver.find_element(By.XPATH, "//button[normalize-space()='Login']").click()
''; '';
## NOTE: `path` must be a valid python string, either a variable or _quoted_.
seleniumTakeScreenshot = path: '' seleniumTakeScreenshot = path: ''
print("Take screenshot...", file=sys.stderr) print("Take screenshot...", file=sys.stderr)
if not driver.save_screenshot(${path}): if not driver.save_screenshot(${path}):
@ -135,11 +136,7 @@ pkgs.nixosTest {
memorySize = lib.mkVMOverride 8192; memorySize = lib.mkVMOverride 8192;
cores = 8; cores = 8;
}; };
imports = with self.nixosModules; [ imports = with self.nixosModules; [ pixelfed-vm ];
garage
pixelfed
pixelfed-vm
];
# TODO: pair down # TODO: pair down
environment.systemPackages = with pkgs; [ environment.systemPackages = with pkgs; [
python3 python3
@ -163,7 +160,7 @@ pkgs.nixosTest {
}; };
}; };
testScript = '' testScript = { nodes, ... }: ''
import re import re
server.start() server.start()
@ -172,7 +169,7 @@ pkgs.nixosTest {
server.wait_for_unit("phpfpm-pixelfed.service") server.wait_for_unit("phpfpm-pixelfed.service")
with subtest("Account creation"): with subtest("Account creation"):
server.succeed(f"pixelfed-manage user:create --name=test --username=test --email=${email} --password=${password} --confirm_email=1") server.succeed("pixelfed-manage user:create --name=test --username=test --email=${email} --password=${password} --confirm_email=1")
# NOTE: This could in theory give a false positive if pixelfed changes it's # NOTE: This could in theory give a false positive if pixelfed changes it's
# colorscheme to include pure green. (see same problem in pixelfed-garage.nix). # colorscheme to include pure green. (see same problem in pixelfed-garage.nix).
@ -180,7 +177,7 @@ pkgs.nixosTest {
# there, then post a green image and check that the green pixel IS there. # there, then post a green image and check that the green pixel IS there.
with subtest("Image displays"): with subtest("Image displays"):
server.succeed(f"su - selenium -c 'selenium-script-post-picture ${email} ${password}'") server.succeed("su - selenium -c 'selenium-script-post-picture ${email} ${password}'")
server.copy_from_vm("/home/selenium/screenshot.png", "") server.copy_from_vm("/home/selenium/screenshot.png", "")
displayed_colors = server.succeed("magick /home/selenium/screenshot.png -define histogram:unique-colors=true -format %c histogram:info:") displayed_colors = server.succeed("magick /home/selenium/screenshot.png -define histogram:unique-colors=true -format %c histogram:info:")
# check that the green image displayed somewhere # check that the green image displayed somewhere
@ -189,14 +186,14 @@ pkgs.nixosTest {
raise Exception("cannot detect the uploaded image on pixelfed page.") raise Exception("cannot detect the uploaded image on pixelfed page.")
with subtest("access garage"): with subtest("access garage"):
server.succeed("mc alias set garage http://s3.garage.localhost:3900 --api s3v4 --path off $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY") server.succeed("mc alias set garage ${nodes.server.fediversity.internal.garage.api.url} --api s3v4 --path off $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY")
server.succeed("mc ls garage/pixelfed") server.succeed("mc ls garage/pixelfed")
with subtest("access image in garage"): with subtest("access image in garage"):
image = server.succeed("mc find garage --regex '\.png' --ignore '*_thumb.png'") image = server.succeed("mc find garage --regex '\\.png' --ignore '*_thumb.png'")
image = image.rstrip() image = image.rstrip()
if image == "": if image == "":
raise Exception("image posted to mastodon did not get stored in garage") raise Exception("image posted to Pixelfed did not get stored in garage")
server.succeed(f"mc cat {image} >/garage-image.png") server.succeed(f"mc cat {image} >/garage-image.png")
garage_image_hash = server.succeed("identify -quiet -format '%#' /garage-image.png") garage_image_hash = server.succeed("identify -quiet -format '%#' /garage-image.png")
image_hash = server.succeed("identify -quiet -format '%#' $POST_MEDIA") image_hash = server.succeed("identify -quiet -format '%#' $POST_MEDIA")
@ -204,8 +201,8 @@ pkgs.nixosTest {
raise Exception("image stored in garage did not match image uploaded") raise Exception("image stored in garage did not match image uploaded")
with subtest("Check that image comes from garage"): with subtest("Check that image comes from garage"):
src = server.succeed(f"su - selenium -c 'selenium-script-get-src ${email} ${password}'") src = server.succeed("su - selenium -c 'selenium-script-get-src ${email} ${password}'")
if not src.startswith("http://pixelfed.web.garage.localhost:3902/"): if not src.startswith("${nodes.server.fediversity.internal.garage.web.urlFor "pixelfed"}"):
raise Exception("image does not come from garage") raise Exception("image does not come from garage")
''; '';
} }

0
interactive-vm.nix → vm/interactive-vm.nix
View file

21
mastodon-vm.nix → vm/mastodon-vm.nix
View file

@ -1,18 +1,19 @@
{ modulesPath, lib, config, ... }: { { modulesPath, lib, config, ... }: {
imports = [ (modulesPath + "/virtualisation/qemu-vm.nix") ]; imports = [
../fediversity
(modulesPath + "/virtualisation/qemu-vm.nix")
];
config = lib.mkMerge [ config = lib.mkMerge [
{ {
fediversity = {
enable = true;
domain = "localhost";
mastodon.enable = true;
};
services.mastodon = { services.mastodon = {
# redirects to localhost, but allows it to have a proper domain name
localDomain = "mastodon.localhost";
smtp = {
fromAddress = "mastodon@mastodon.localhost";
createLocally = false;
};
extraConfig = { extraConfig = {
EMAIL_DOMAIN_ALLOWLIST = "example.com"; EMAIL_DOMAIN_ALLOWLIST = "example.com";
}; };
@ -56,7 +57,7 @@
BIND = "0.0.0.0"; BIND = "0.0.0.0";
# for letter_opener (still doesn't work though) # for letter_opener (still doesn't work though)
REMOTE_DEV = "true"; REMOTE_DEV = "true";
LOCAL_DOMAIN = "mastodon.localhost:8443"; LOCAL_DOMAIN = "${config.fediversity.internal.mastodon.domain}:8443";
}; };
}; };

15
peertube-vm.nix → vm/peertube-vm.nix
View file

@ -1,9 +1,11 @@
{ pkgs, modulesPath, ... }: { { pkgs, modulesPath, ... }: {
imports = [ (modulesPath + "/virtualisation/qemu-vm.nix") ];
imports = [
../fediversity
(modulesPath + "/virtualisation/qemu-vm.nix")
];
services.peertube = { services.peertube = {
enable = true;
# redirects to localhost, but allows it to have a proper domain name
localDomain = "peertube.localhost";
enableWebHttps = false; enableWebHttps = false;
settings = { settings = {
listen.hostname = "0.0.0.0"; listen.hostname = "0.0.0.0";
@ -13,11 +15,6 @@
secrets.secretsFile = pkgs.writeText "secret" '' secrets.secretsFile = pkgs.writeText "secret" ''
574e093907d1157ac0f8e760a6deb1035402003af5763135bae9cbd6abe32b24 574e093907d1157ac0f8e760a6deb1035402003af5763135bae9cbd6abe32b24
''; '';
# TODO: in most of nixpkgs, these are true by default. upstream that unless there's a good reason not to.
redis.createLocally = true;
database.createLocally = true;
configureNginx = true;
}; };
virtualisation.forwardPorts = [ virtualisation.forwardPorts = [

16
pixelfed-vm.nix → vm/pixelfed-vm.nix
View file

@ -1,8 +1,18 @@
{ pkgs, modulesPath, ... }: { { pkgs, modulesPath, ... }: {
imports = [ (modulesPath + "/virtualisation/qemu-vm.nix") ];
imports = [
../fediversity
(modulesPath + "/virtualisation/qemu-vm.nix")
];
fediversity = {
enable = true;
domain = "localhost";
pixelfed.enable = true;
};
networking.firewall.allowedTCPPorts = [ 80 ]; networking.firewall.allowedTCPPorts = [ 80 ];
services.pixelfed = { services.pixelfed = {
domain = "pixelfed.localhost";
# TODO: secrets management! # TODO: secrets management!
secretFile = pkgs.writeText "secrets.env" '' secretFile = pkgs.writeText "secrets.env" ''
APP_KEY=adKK9EcY8Hcj3PLU7rzG9rJ6KKTOtYfA APP_KEY=adKK9EcY8Hcj3PLU7rzG9rJ6KKTOtYfA
@ -14,7 +24,7 @@
# I feel like this should have an `enable` option and be configured via `services.nginx` rather than mirroring those options in services.pixelfed.nginx # I feel like this should have an `enable` option and be configured via `services.nginx` rather than mirroring those options in services.pixelfed.nginx
# TODO: If that indeed makes sense, upstream it. # TODO: If that indeed makes sense, upstream it.
nginx = { nginx = {
# locations."/public/".proxyPass = "http://pixelfed.web.garage.localhost:3902/public/"; # locations."/public/".proxyPass = "${config.fediversity.internal.garage.web.urlFor "pixelfed"}/public/";
}; };
}; };
virtualisation.memorySize = 2048; virtualisation.memorySize = 2048;