nixos-test-pixelfed-wip #22

Merged
taeer merged 11 commits from nixos-test-pixelfed-wip into main 2024-09-20 17:47:21 +02:00
9 changed files with 98 additions and 57 deletions
Showing only changes of commit 73939b9d87 - Show all commits

View file

@ -17,42 +17,83 @@ in {
fediversity = { fediversity = {
enable = mkEnableOption "the collection of services bundled under Fediversity"; enable = mkEnableOption "the collection of services bundled under Fediversity";
garage = mkOption {
type = types.anything;
};
domain = mkOption { domain = mkOption {
type = types.str; type = types.str;
Niols marked this conversation as resolved Outdated
Outdated
Review

This can just be

fediversity = mkEnableOption "the collection of services bundled under fediversity";
This can just be ``` fediversity = mkEnableOption "the collection of services bundled under fediversity"; ```
Outdated
Review

Thanks! Done in 2ff8975b6b.

Thanks! Done in 2ff8975b6b2f19d5ffbd7574bd76494bf532d048.
description = ''
root domain for the Fediversity services
Niols marked this conversation as resolved Outdated
Outdated
Review

You almost never need types.anything. And certainly in this case the set of sub-options should be defined. This can be done with simple attrsets

garage = {
  api = {
    url = mkOption {
      ...
    };
  };
};

Also

We don't need to have a very clear sense of which options belong where yet, but eventually I think the top-level fediversity options should be the ones that should be set publicly (via NixPanel), and probably the ports being used in garage shouldn't be among them.

I'm not sure what the right namespace to use for those options is... maybe fediversity-private or fediversity.private...

You almost never need `types.anything`. And certainly in this case the set of sub-options should be defined. This can be done with simple attrsets ``` garage = { api = { url = mkOption { ... }; }; }; ``` --- Also We don't need to have a very clear sense of which options belong where **yet**, but eventually I think the top-level `fediversity` options should be the ones that should be set publicly (via NixPanel), and probably the ports being used in garage shouldn't be among them. I'm not sure what the right namespace to use for those options is... maybe `fediversity-private` or `fediversity.private`...
Outdated
Review

Done in 73939b9d87.

Done in 73939b9d8752ed4193ebae1b865c306d8eae4971.
For instance, if this option is set to `foo.example.com`, then
Pixelfed might be under `pixelfed.foo.example.com`.
'';
}; };
Niols marked this conversation as resolved Outdated
Outdated
Review

should be types.str
types.string was being used for too many thing so it got deprecated and now there are several different string types. types.str is the one where you don't want to merge definitions if it's defined in more than one place

should be `types.str` `types.string` was being used for too many thing so it got deprecated and now there are several different string types. `types.str` is the one where you don't want to merge definitions if it's defined in more than one place
Outdated
Review

Done in d97772ccc4.

Done in d97772ccc4008417aec2117d62031e9123d8deba.
mastodon.enable = mkEnableOption "default Fediversity Mastodon configuration"; mastodon.enable = mkEnableOption "default Fediversity Mastodon configuration";
pixelfed.enable = mkEnableOption "default Fediversity Pixelfed configuration"; pixelfed.enable = mkEnableOption "default Fediversity Pixelfed configuration";
peertube.enable = mkEnableOption "default Fediversity PeerTube configuration"; peertube.enable = mkEnableOption "default Fediversity PeerTube configuration";
};
};
Niols marked this conversation as resolved Outdated
Outdated
Review

Similarly, mkEnableOption here

Similarly, `mkEnableOption` here
config.fediversity = { internal = mkOption {
garage = { description = "options that are only meant to be used internally; change at your own risk";
api = rec { default = {};
domain = "s3.garage.${config.fediversity.domain}"; type = types.submodule {
Niols marked this conversation as resolved Outdated
Outdated
Review

This should probably be under a call to lib.mkDefault. Or simply put behind the default argument of mkOption. It seems to be the custom (where possible---sometimes it's not) to put it under the default argument.

This should probably be under a call to `lib.mkDefault`. Or simply put behind the `default` argument of `mkOption`. It seems to be the custom (where possible---sometimes it's not) to put it under the `default` argument.
Outdated
Review

Done in 73939b9d87.

Done in 73939b9d8752ed4193ebae1b865c306d8eae4971.
port = 3900; options = {
url = "http://${domain}:${toString port}"; garage = {
}; api = {
domain = mkOption {
type = types.str;
default = "s3.garage.${config.fediversity.domain}";
};
port = mkOption {
type = types.int;
default = 3900;
};
url = mkOption {
type = types.str;
default = "http://${config.fediversity.internal.garage.api.domain}:${toString config.fediversity.internal.garage.api.port}";
};
};
rpc = rec { rpc = {
port = 3901; port = mkOption {
}; type = types.int;
default = 3901;
};
Outdated
Review

there's a style decision here between

  1. defining our own top-level e.g. fediversity.pixelfed.domain, and
  2. referring to e.g. services.pixelfed.domain everywhere.

(1) has the advantage of grouping everything in one place and under one naming scheme, as opposed to needing to know that e.g. mastodon's is under services.mastodon.localDomain instead.
It has the disadvantage of having basically a duplicate option, that really should always be exactly identical to another option. Now if someone sets services.pixelfed.domain somewhere, it won't get propagated to all the other places that use it, because they refer to fediversity.pixelfed.domain. It's also hard to know where to stop with option (1), since you could recategorize all the options under some different scheme at the top level, which might be somewhat more consistent.

I tend to go with option (2), but in a tightly controlled module like we're planning, I don't think it's objectively the right choice, just a style difference as I said.


In the case of the garage options above, we sort of have the same question. But in that case there's information we can't easily get out of the options that are available in services.garage (e.g. just the port itself), so I think it does make sense to define our own options.

Though we should probably discuss with the maintainer of the nixpkgs module if it makes sense to upstream those changes.

there's a style decision here between 1. defining our own top-level e.g. `fediversity.pixelfed.domain`, and 2. referring to e.g. `services.pixelfed.domain` everywhere. (1) has the advantage of grouping everything in one place and under one naming scheme, as opposed to needing to know that e.g. mastodon's is under `services.mastodon.localDomain` instead. It has the disadvantage of having basically a duplicate option, that really should always be exactly identical to another option. Now if someone sets `services.pixelfed.domain` somewhere, it won't get propagated to all the other places that use it, because they refer to `fediversity.pixelfed.domain`. It's also hard to know where to stop with option (1), since you could recategorize all the options under some different scheme at the top level, which might be somewhat more consistent. I tend to go with option (2), but in a tightly controlled module like we're planning, I don't think it's objectively the right choice, just a style difference as I said. --- In the case of the `garage` options above, we sort of have the same question. But in that case there's information we can't easily get out of the options that are available in `services.garage` (e.g. just the port itself), so I think it does make sense to define our own options. Though we should probably discuss with the maintainer of the nixpkgs module if it makes sense to upstream those changes.
Outdated
Review

Though if you are going to define fediversity.pixelfed.domain, it does have to be declared as an option somewhere 😆

Though if you are going to define `fediversity.pixelfed.domain`, it does have to be declared as an option somewhere 😆
};
web = rec { web = {
rootDomain = "web.garage.${config.fediversity.domain}"; rootDomain = mkOption {
port = 3902; type = types.str;
rootDomainAndPort = "${rootDomain}:${toString port}"; default = "web.garage.${config.fediversity.domain}";
urlFor = bucket: "http://${bucket}.${rootDomainAndPort}"; };
port = mkOption {
type = types.int;
default = 3902;
};
rootDomainAndPort = mkOption {
type = types.str;
default = "${config.fediversity.internal.garage.web.rootDomain}:${toString config.fediversity.internal.garage.web.port}";
};
urlFor = mkOption {
type = types.functionTo types.str;
default = bucket: "http://${bucket}.${config.fediversity.internal.garage.web.rootDomainAndPort}";
};
};
};
pixelfed.domain = mkOption {
type = types.str;
default = "pixelfed.${config.fediversity.domain}";
};
mastodon.domain = mkOption {
type = types.str;
default = "mastdodon.${config.fediversity.domain}";
};
peertube.domain = mkOption {
type = types.str;
default = "peertube.${config.fediversity.domain}";
};
};
};
}; };
}; };
pixelfed.domain = "pixelfed.${config.fediversity.domain}";
mastodon.domain = "mastdodon.${config.fediversity.domain}";
peertube.domain = "peertube.${config.fediversity.domain}";
}; };
} }

View file

@ -42,7 +42,7 @@ let
${optionalString corsRules.enable '' ${optionalString corsRules.enable ''
garage bucket allow --read --write --owner ${bucketArg} --key tmp garage bucket allow --read --write --owner ${bucketArg} --key tmp
# TODO: endpoin-url should not be hard-coded # TODO: endpoin-url should not be hard-coded
aws --region ${cfg.settings.s3_api.s3_region} --endpoint-url ${config.fediversity.garage.api.url} s3api put-bucket-cors --bucket ${bucketArg} --cors-configuration ${corsRulesJSON} aws --region ${cfg.settings.s3_api.s3_region} --endpoint-url ${config.fediversity.internal.garage.api.url} s3api put-bucket-cors --bucket ${bucketArg} --cors-configuration ${corsRulesJSON}
garage bucket deny --read --write --owner ${bucketArg} --key tmp garage bucket deny --read --write --owner ${bucketArg} --key tmp
''} ''}
''; '';
@ -136,21 +136,21 @@ in
virtualisation.forwardPorts = [ virtualisation.forwardPorts = [
{ {
from = "host"; from = "host";
host.port = config.fediversity.garage.rpc.port; host.port = config.fediversity.internal.garage.rpc.port;
guest.port = config.fediversity.garage.rpc.port; guest.port = config.fediversity.internal.garage.rpc.port;
} }
{ {
from = "host"; from = "host";
host.port = config.fediversity.garage.web.port; host.port = config.fediversity.internal.garage.web.port;
guest.port = config.fediversity.garage.web.port; guest.port = config.fediversity.internal.garage.web.port;
} }
]; ];
environment.systemPackages = [ pkgs.minio-client pkgs.awscli ]; environment.systemPackages = [ pkgs.minio-client pkgs.awscli ];
networking.firewall.allowedTCPPorts = [ networking.firewall.allowedTCPPorts = [
config.fediversity.garage.rpc.port config.fediversity.internal.garage.rpc.port
config.fediversity.garage.web.port config.fediversity.internal.garage.web.port
]; ];
services.garage = { services.garage = {
enable = true; enable = true;
@ -160,15 +160,15 @@ in
# TODO: use a secret file # TODO: use a secret file
rpc_secret = "d576c4478cc7d0d94cfc127138cbb82018b0155c037d1c827dfb6c36be5f6625"; rpc_secret = "d576c4478cc7d0d94cfc127138cbb82018b0155c037d1c827dfb6c36be5f6625";
# TODO: why does this have to be set? is there not a sensible default? # TODO: why does this have to be set? is there not a sensible default?
rpc_bind_addr = "[::]:${toString config.fediversity.garage.rpc.port}"; rpc_bind_addr = "[::]:${toString config.fediversity.internal.garage.rpc.port}";
rpc_public_addr = "[::1]:${toString config.fediversity.garage.rpc.port}"; rpc_public_addr = "[::1]:${toString config.fediversity.internal.garage.rpc.port}";
s3_api.api_bind_addr = "[::]:${toString config.fediversity.garage.api.port}"; s3_api.api_bind_addr = "[::]:${toString config.fediversity.internal.garage.api.port}";
s3_web.bind_addr = "[::]:${toString config.fediversity.garage.web.port}"; s3_web.bind_addr = "[::]:${toString config.fediversity.internal.garage.web.port}";
s3_web.root_domain = ".${config.fediversity.garage.web.rootDomain}"; s3_web.root_domain = ".${config.fediversity.internal.garage.web.rootDomain}";
index = "index.html"; index = "index.html";
s3_api.s3_region = "garage"; s3_api.s3_region = "garage";
s3_api.root_domain = ".${config.fediversity.garage.api.domain}"; s3_api.root_domain = ".${config.fediversity.internal.garage.api.domain}";
}; };
}; };
systemd.services.ensure-garage = { systemd.services.ensure-garage = {
@ -183,7 +183,7 @@ in
# Give Garage time to start up by waiting until somethings speaks HTTP # Give Garage time to start up by waiting until somethings speaks HTTP
# behind Garage's API URL. # behind Garage's API URL.
until ${pkgs.curl}/bin/curl -sio /dev/null ${config.fediversity.garage.api.url}; do sleep 1; done until ${pkgs.curl}/bin/curl -sio /dev/null ${config.fediversity.internal.garage.api.url}; do sleep 1; done
# XXX: this is very sensitive to being a single instance # XXX: this is very sensitive to being a single instance
# (doing the bare minimum to get garage up and running) # (doing the bare minimum to get garage up and running)

View file

@ -38,7 +38,7 @@ lib.mkIf (config.fediversity.enable && config.fediversity.mastodon.enable) {
extraConfig = rec { extraConfig = rec {
S3_ENABLED = "true"; S3_ENABLED = "true";
# TODO: this shouldn't be hard-coded, it should come from the garage configuration # TODO: this shouldn't be hard-coded, it should come from the garage configuration
S3_ENDPOINT = config.fediversity.garage.api.url; S3_ENDPOINT = config.fediversity.internal.garage.api.url;
S3_REGION = "garage"; S3_REGION = "garage";
S3_BUCKET = "mastodon"; S3_BUCKET = "mastodon";
# use <S3_BUCKET>.<S3_ENDPOINT> # use <S3_BUCKET>.<S3_ENDPOINT>
@ -46,7 +46,7 @@ lib.mkIf (config.fediversity.enable && config.fediversity.mastodon.enable) {
AWS_ACCESS_KEY_ID = snakeoil_key.id; AWS_ACCESS_KEY_ID = snakeoil_key.id;
AWS_SECRET_ACCESS_KEY = snakeoil_key.secret; AWS_SECRET_ACCESS_KEY = snakeoil_key.secret;
S3_PROTOCOL = "http"; S3_PROTOCOL = "http";
S3_HOSTNAME = config.fediversity.garage.web.rootDomainAndPort; S3_HOSTNAME = config.fediversity.internal.garage.web.rootDomainAndPort;
# by default it tries to use "<S3_HOSTNAME>/<S3_BUCKET>" # by default it tries to use "<S3_HOSTNAME>/<S3_BUCKET>"
S3_ALIAS_HOST = "${S3_BUCKET}.${S3_HOSTNAME}"; S3_ALIAS_HOST = "${S3_BUCKET}.${S3_HOSTNAME}";
# SEE: the last section in https://docs.joinmastodon.org/admin/optional/object-storage/ # SEE: the last section in https://docs.joinmastodon.org/admin/optional/object-storage/
@ -63,12 +63,12 @@ lib.mkIf (config.fediversity.enable && config.fediversity.mastodon.enable) {
services.mastodon = { services.mastodon = {
enable = true; enable = true;
localDomain = config.fediversity.mastodon.domain; localDomain = config.fediversity.internal.mastodon.domain;
configureNginx = true; configureNginx = true;
# TODO: configure a mailserver so this works # TODO: configure a mailserver so this works
smtp = { smtp = {
fromAddress = "noreply@${config.fediversity.mastodon.domain}"; fromAddress = "noreply@${config.fediversity.internal.mastodon.domain}";
createLocally = false; createLocally = false;
}; };

View file

@ -54,7 +54,7 @@ lib.mkIf (config.fediversity.enable && config.fediversity.peertube.enable) {
services.peertube = { services.peertube = {
enable = true; enable = true;
localDomain = config.fediversity.peertube.domain; localDomain = config.fediversity.internal.peertube.domain;
# TODO: in most of nixpkgs, these are true by default. upstream that unless there's a good reason not to. # TODO: in most of nixpkgs, these are true by default. upstream that unless there's a good reason not to.
redis.createLocally = true; redis.createLocally = true;
@ -64,7 +64,7 @@ lib.mkIf (config.fediversity.enable && config.fediversity.peertube.enable) {
settings = { settings = {
object_storage = { object_storage = {
enabled = true; enabled = true;
endpoint = config.fediversity.garage.api.url; endpoint = config.fediversity.internal.garage.api.url;
region = "garage"; region = "garage";
# not supported by garage # not supported by garage
@ -74,17 +74,17 @@ lib.mkIf (config.fediversity.enable && config.fediversity.peertube.enable) {
web_videos = rec { web_videos = rec {
bucket_name = "peertube-videos"; bucket_name = "peertube-videos";
prefix = ""; prefix = "";
base_url = config.fediversity.garage.web.urlFor bucket_name; base_url = config.fediversity.internal.garage.web.urlFor bucket_name;
}; };
videos = rec { videos = rec {
bucket_name = "peertube-videos"; bucket_name = "peertube-videos";
prefix = ""; prefix = "";
base_url = config.fediversity.garage.web.urlFor bucket_name; base_url = config.fediversity.internal.garage.web.urlFor bucket_name;
}; };
streaming_playlists = rec { streaming_playlists = rec {
bucket_name = "peertube-playlists"; bucket_name = "peertube-playlists";
prefix = ""; prefix = "";
base_url = config.fediversity.garage.web.urlFor bucket_name; base_url = config.fediversity.internal.garage.web.urlFor bucket_name;
}; };
}; };
}; };

View file

@ -37,7 +37,7 @@ lib.mkIf (config.fediversity.enable && config.fediversity.pixelfed.enable) {
services.pixelfed = { services.pixelfed = {
enable = true; enable = true;
domain = config.fediversity.pixelfed.domain; domain = config.fediversity.internal.pixelfed.domain;
}; };
services.pixelfed.settings = { services.pixelfed.settings = {
@ -47,9 +47,9 @@ lib.mkIf (config.fediversity.enable && config.fediversity.pixelfed.enable) {
AWS_ACCESS_KEY_ID = snakeoil_key.id; AWS_ACCESS_KEY_ID = snakeoil_key.id;
AWS_SECRET_ACCESS_KEY = snakeoil_key.secret; AWS_SECRET_ACCESS_KEY = snakeoil_key.secret;
AWS_DEFAULT_REGION = "garage"; AWS_DEFAULT_REGION = "garage";
AWS_URL = config.fediversity.garage.web.urlFor "pixelfed"; AWS_URL = config.fediversity.internal.garage.web.urlFor "pixelfed";
AWS_BUCKET = "pixelfed"; AWS_BUCKET = "pixelfed";
AWS_ENDPOINT = config.fediversity.garage.api.url; AWS_ENDPOINT = config.fediversity.internal.garage.api.url;
AWS_USE_PATH_STYLE_ENDPOINT = false; AWS_USE_PATH_STYLE_ENDPOINT = false;
}; };

View file

@ -95,7 +95,7 @@ pkgs.nixosTest {
server.succeed("toot post --media $POST_MEDIA") server.succeed("toot post --media $POST_MEDIA")
with subtest("access garage"): with subtest("access garage"):
## REVIEW: could we grab `config.fediversity.garage.api.url` here in some way? ## REVIEW: could we grab `config.fediversity.internal.garage.api.url` here in some way?
server.succeed("mc alias set garage http://s3.garage.localhost:3900 --api s3v4 --path off $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY") server.succeed("mc alias set garage http://s3.garage.localhost:3900 --api s3v4 --path off $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY")
Review

Yes! In typical nix dynamically-typed fashion, testScript can also be a function! And we can have

testScript = { nodes, ... }: ''
...
   ${nodes.server.fediversity.garage.api.url}
...
'';
Yes! In typical nix dynamically-typed fashion, `testScript` can also be a function! And we can have ``` testScript = { nodes, ... }: '' ... ${nodes.server.fediversity.garage.api.url} ... ''; ```
server.succeed("mc ls garage/mastodon") server.succeed("mc ls garage/mastodon")
@ -122,7 +122,7 @@ pkgs.nixosTest {
raise Exception("mastodon did not send a content security policy header") raise Exception("mastodon did not send a content security policy header")
csp = csp_match.group(1) csp = csp_match.group(1)
# the img-src content security policy should include the garage server # the img-src content security policy should include the garage server
## REVIEW: could we grab `config.fediversity.garage.web.url` here in some way? ## REVIEW: could we grab `config.fediversity.internal.garage.web.url` here in some way?
Outdated
Review

Yes, though we need to be careful about escaping for the regex

Yes, though we need to be careful about escaping for the regex
garage_csp = re.match(".*; img-src[^;]*web\.garage\.localhost:3902.*", csp) garage_csp = re.match(".*; img-src[^;]*web\.garage\.localhost:3902.*", csp)
if garage_csp is None: if garage_csp is None:
raise Exception("Mastodon's content security policy does not include garage server. image will not be displayed properly on mastodon.") raise Exception("Mastodon's content security policy does not include garage server. image will not be displayed properly on mastodon.")

View file

@ -186,7 +186,7 @@ pkgs.nixosTest {
raise Exception("cannot detect the uploaded image on pixelfed page.") raise Exception("cannot detect the uploaded image on pixelfed page.")
with subtest("access garage"): with subtest("access garage"):
## REVIEW: could we grab `config.fediversity.garage.api.url` here in some way? ## REVIEW: could we grab `config.fediversity.internal.garage.api.url` here in some way?
server.succeed("mc alias set garage http://s3.garage.localhost:3900 --api s3v4 --path off $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY") server.succeed("mc alias set garage http://s3.garage.localhost:3900 --api s3v4 --path off $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY")
server.succeed("mc ls garage/pixelfed") server.succeed("mc ls garage/pixelfed")
@ -203,7 +203,7 @@ pkgs.nixosTest {
with subtest("Check that image comes from garage"): with subtest("Check that image comes from garage"):
src = server.succeed("su - selenium -c 'selenium-script-get-src ${email} ${password}'") src = server.succeed("su - selenium -c 'selenium-script-get-src ${email} ${password}'")
## REVIEW: could we grab `config.fediversity.garage.web.url` here in some way? ## REVIEW: could we grab `config.fediversity.internal.garage.web.url` here in some way?
if not src.startswith("http://pixelfed.web.garage.localhost:3902/"): if not src.startswith("http://pixelfed.web.garage.localhost:3902/"):
raise Exception("image does not come from garage") raise Exception("image does not come from garage")
''; '';

View file

@ -57,7 +57,7 @@
BIND = "0.0.0.0"; BIND = "0.0.0.0";
# for letter_opener (still doesn't work though) # for letter_opener (still doesn't work though)
REMOTE_DEV = "true"; REMOTE_DEV = "true";
LOCAL_DOMAIN = "${config.fediversity.mastodon.domain}:8443"; LOCAL_DOMAIN = "${config.fediversity.internal.mastodon.domain}:8443";
}; };
}; };

View file

@ -24,7 +24,7 @@
# I feel like this should have an `enable` option and be configured via `services.nginx` rather than mirroring those options in services.pixelfed.nginx # I feel like this should have an `enable` option and be configured via `services.nginx` rather than mirroring those options in services.pixelfed.nginx
# TODO: If that indeed makes sense, upstream it. # TODO: If that indeed makes sense, upstream it.
nginx = { nginx = {
# locations."/public/".proxyPass = "${config.fediversity.garage.web.urlFor "pixelfed"}/public/"; # locations."/public/".proxyPass = "${config.fediversity.internal.garage.web.urlFor "pixelfed"}/public/";
}; };
}; };
virtualisation.memorySize = 2048; virtualisation.memorySize = 2048;