Configuration schema available dynamically #1288

Open
opened 2026-08-05 16:21:59 +02:00 by toonn · 8 comments
Member

As a Hosting Provider
I want to fetch the configuration schema(s) for Applications dynamically
so that deployment of my Operator-facing UI is independent from Fediversity releases.

Hosting Providers who want to deploy Fediversity need an interface for Operators to configure/deploy Applications. To that end Fediversity includes a simple panel, as a starting point for Hosting Providers' own implementations.

One small step further is to take a white-label approach to the panel, intended for Hosting Providers to easily add their branding without having to implement a custom front-end.

Some Hosting Providers will already have a front-end and will want to add support for Application configuration/deployment there, rather than adopt a new panel.

Mobile app-based front-ends are impractical to "redeploy" alongside Fediversity, decoupling them via an API is basically a necessity.

**As** a Hosting Provider **I want** to fetch the configuration schema(s) for Applications dynamically **so that** deployment of my Operator-facing UI is independent from Fediversity releases. Hosting Providers who want to deploy Fediversity need an interface for Operators to configure/deploy Applications. To that end Fediversity includes a simple panel, as a starting point for Hosting Providers' own implementations. One small step further is to take a white-label approach to the panel, intended for Hosting Providers to easily add their branding without having to implement a custom front-end. Some Hosting Providers will already have a front-end and will want to add support for Application configuration/deployment there, rather than adopt a new panel. Mobile app-based front-ends are impractical to "redeploy" alongside Fediversity, decoupling them via an API is basically a necessity.
Owner

so that deployment of my Operator-facing UI is independent from Fediversity releases.

that goal i think is served by #1282.

note that that is orthogonal from the stated desire here of switching from static inclusion to run-time, as demonstrated by today's static extension mechanism of #1189 that is the sole reason how api/panel applications are deployed to abundos.eu today despite being out of scope from core/.

now, that isn't to say the stated desire of run-time inclusion here might not be useful, but rather than that being over some need for it to accomplish #1282, as per #1209 (comment) i wonder if reloads over static inclusion might invalidate sessions, which this may address (not sure if the only way, but it sounds valid).

> so that deployment of my Operator-facing UI is independent from Fediversity releases. that goal i think is served by #1282. note that that is orthogonal from the stated desire here of switching from static inclusion to run-time, as demonstrated by today's static extension mechanism of #1189 that is the sole reason how api/panel applications are deployed to abundos.eu today despite being out of scope from `core/`. now, that isn't to say the stated desire of run-time inclusion here might not be useful, but rather than that being over some need for it to accomplish #1282, as per https://git.fediversity.eu/fediversity/fediversity/issues/1209#issuecomment-18294 i wonder if reloads over static inclusion might invalidate sessions, which this may address (not sure if the only way, but it sounds valid).
Author
Member

#1282 puts Hosting Providers in control of what Applications show up in their panel.

This issue is about decoupling panels from Fediversity deployments. The newly deployed back-end would serve different schemas over the API, the panel simply adopts these next time it needs to load them. #1189 does not achieve this decoupling, nor does it try to.

#1282 puts Hosting Providers in control of what Applications show up in their panel. This issue is about decoupling panels from Fediversity deployments. The newly deployed back-end would serve different schemas over the API, the panel simply adopts these next time it needs to load them. #1189 does not achieve this decoupling, nor does it try to.
Owner

to maybe try and clarify (?), while #304 (implemented as https://swagger.pages.git.fediversity.eu/#/default/schema_get_for) could be phrased as this 'serv[ing] different schemas over the API', one reading of this ticket would be about facilitating swapping out the recommended (so non-foreign) deployment logic / schema, while that existing endpoint is intended to facilitate foreign (many) vs a single default 'recommended' fediversity pin (i.e. flake that rendered core's attributes, which aside from core itself api/panel have today as well).

perhaps not like #1282 (if not #1189 just as well), my concern with logic taken outside of (any maintained, rendered) VCS is, quoting from #1282 (comment):

raises questions on to what extent such applications could be migrated to different hosting providers (#100), and what keeping them updated after a hypothetical migration might look like

you noted tensions between supporting this vs 'real-world use-cases' at fediversity/module-schema#12 (comment), and i'd love to not be bound by this feature (as scrolling down #100 should reveal how many of my concerns that'd help dissipate) - tho given NLNet pitched the project specifically on that feature, i'd consider backing out on it above my pay-grade - so then we'd likely need to prepare well-founded considerations on this to set expectations with @michielleenaars. tbf, to my knowledge our mission is less to ace every theoretical feature in the sense we are essentially exploring this space, and "this involves trade-offs, which we'd try and navigate as per real-world priorities" certainly seems one valid outcome there.

for what it's worth, i maybe have some hope still there may be ways to reconcile things, so would like to maybe still better understand the underlying desire(s) here.

fediversity/module-schema#12 mentioned not wanting to await our security updates, which #1189 should solve.
like, i presume they would probably want their stuff in VCS anyway? because then (+ published too, which may be more controversial, tho then again our EUPL is copyleft) the concern quoted above should be done.

another point by @toonn on this from fediversity/module-schema#12:

Hosting Providers should be able to set Nixpkgs pins per Operator or Application but start with a Nixpkgs pin hardcoded in Fediversity Core.

so:

  • our services have used distinct pins (with e.g. forgejo having been ahead for a bit)
  • #1189 should empower hosting providers to that end as well
  • pins per operator (tho core+api lack notion of users like operators) seem to flow from the fact we don't force any versions on existing deployments, tho these pins so far target nixpkgs only by proxy (which i think fediversity/module-schema#12's point 1 aims to resolve)

to be fair, that may not yet make for the full freedom you're describing - but i think it illustrates there should be quite a bit we can do still without raising questions on #100 we may not know the answers to yet.

to maybe try and clarify (?), while #304 (implemented as https://swagger.pages.git.fediversity.eu/#/default/schema_get_for) could be phrased as this 'serv[ing] different schemas over the API', one reading of this ticket would be about facilitating swapping out the _recommended_ (so non-foreign) deployment logic / schema, while that existing endpoint is intended to facilitate foreign (many) vs a single default 'recommended' fediversity pin (i.e. flake that `render`ed core's attributes, which aside from core itself api/panel have today as well). perhaps not like #1282 (if not #1189 just as well), my concern with logic taken outside of (any maintained, `render`ed) VCS is, quoting from https://git.fediversity.eu/fediversity/fediversity/issues/1282#issuecomment-18401: > raises questions on to what extent such applications could be migrated to different hosting providers (#100), and what keeping them updated after a hypothetical migration might look like you noted tensions between supporting this vs 'real-world use-cases' at https://git.fediversity.eu/fediversity/module-schema/issues/12#issuecomment-17574, and i'd love to not be bound by this feature (as scrolling down #100 should reveal how many of my concerns that'd help dissipate) - tho given NLNet pitched the project specifically on that feature, i'd consider backing out on it above my pay-grade - so then we'd likely need to prepare well-founded considerations on this to set expectations with @michielleenaars. tbf, to my knowledge our mission is less to ace every theoretical feature in the sense we _are_ essentially exploring this space, and "this involves trade-offs, which we'd try and navigate as per real-world priorities" certainly seems one valid outcome there. for what it's worth, i maybe have some hope still there may be ways to reconcile things, so would like to maybe still better understand the underlying desire(s) here. fediversity/module-schema#12 mentioned not wanting to await our security updates, which #1189 should solve. like, i presume they would probably want their stuff in VCS anyway? because then (+ published too, which may be more controversial, tho then again our EUPL is copyleft) the concern quoted above should be done. another point by @toonn on this from fediversity/module-schema#12: > Hosting Providers should be able to set Nixpkgs pins per Operator or Application but start with a Nixpkgs pin hardcoded in Fediversity Core. so: - our services have used distinct pins (with e.g. forgejo having been ahead for a bit) - #1189 should empower hosting providers to that end as well - pins per operator (tho core+api lack notion of users like operators) seem to flow from the fact we don't force any versions on existing deployments, tho these pins so far target nixpkgs only by proxy (which i think fediversity/module-schema#12's point 1 aims to resolve) to be fair, that may not yet make for the full freedom you're describing - but i think it illustrates there should be quite a bit we can do still without raising questions on #100 we may not know the answers to yet.
Owner

heck, now that i think of it, i think my concern doesn't need #100: if an operator hasn't updated yet, their pin would already be 'foreign' anyway - how would one know what updating might mean there (i.e. how to change underlying core/ code), if not by a VCS-backed pin, given this ticket would ostensibly steer away from that?

heck, now that i think of it, i think my concern doesn't need #100: if an operator hasn't updated yet, their pin would already be 'foreign' anyway - how would one know what updating might mean there (i.e. how to change underlying `core/` code), if not by a VCS-backed pin, given this ticket would ostensibly steer away from that?
Owner

if not by a VCS-backed pin

to rephrase that: if we have a deployment (or theoretically a deployed application, for that matter), we want to know what updating it means. we now do this with a fetchTree-compatible reference to a given revision of a given branch on a git repository (now fediversity).
if code were no longer in any type of version control, i think what updating means there becomes unclear, if it's no longer "keep following the branch" - including after migrations elsewhere.
in other words, who maintains used code (or configuration, if you'd like to call it that), including after a theoretical migration (#100)?


on distinct pins (let's for now say of rendered VCS-tracked flakes like fediversity, as per #1189) among different applications, it just occurred to me that applications currently control the NixOS code that gets emitted for them on behalf of a module, eliminating distinct nixpkgs pins there (currently, an application is just evaluated as part of some pkgs.nixos NixOS derivation construction).
as such, it may be possible to independently test them with different nixpkgs pins, rather than just some services.<service>.package as i'd implied in 'forgejo having been ahead for a bit'. this should help improve the odds we can plug in applications of distinct render-emitted pins (with freed nixpkgs revision is a step further, still).
that isn't so much to say that today no NixOS code gets emitted from outside of the application definition:

  • hosts-common.nix still contains some NixOS configuration today, tho to be fair i would consider this a hack that is probably to be replaced with (contracts-backed) data model resources, to keep the application code in control - tho then at least interface (if not further underlying contracts) on both sides of the equation (requests in the application modules, plugged in resources from a place like hosts-common.nix) should then at least be compatible (and the application side being behind there seems maybe something we could handle by #634, tho with contracts if not resources as well we might wanna think that out beyond the trivial case of like mkRenamedOptionModule).
  • not technically distinct from the above, but core/setups/nixos/, while in practice mediated by data model resources, now emit NixOS code deduplicated between applications
> if not by a VCS-backed pin to rephrase that: if we have a deployment (or theoretically a deployed application, for that matter), we want to know what updating it means. we now do this with a `fetchTree`-compatible reference to a given revision of a given branch on a git repository (now fediversity). if code were no longer in any type of version control, i think what updating means there becomes unclear, if it's no longer "keep following the branch" - including after migrations elsewhere. in other words, who maintains used code (or configuration, if you'd like to call it that), including after a theoretical migration (#100)? --- on distinct pins (let's for now say of `render`ed VCS-tracked flakes like fediversity, as per #1189) among different applications, it just occurred to me that applications currently control the NixOS code that gets emitted for them on behalf of a module, eliminating distinct nixpkgs pins there (currently, an application is just evaluated as part of some `pkgs.nixos` NixOS derivation construction). as such, it may be possible to independently test them with different nixpkgs pins, rather than just some `services.<service>.package` as i'd implied in 'forgejo having been ahead for a bit'. this should help improve the odds we can plug in applications of distinct `render`-emitted pins (with freed nixpkgs revision is a step further, still). that isn't so much to say that today no NixOS code gets emitted from outside of the application definition: - `hosts-common.nix` still contains some NixOS configuration today, tho to be fair i would consider this a hack that is probably to be replaced with (contracts-backed) data model resources, to keep the application code in control - tho then at least interface (if not further underlying contracts) on both sides of the equation (requests in the application modules, plugged in resources from a place like `hosts-common.nix`) should then at least be compatible (and the application side being behind there seems maybe something we could handle by #634, tho with contracts if not resources as well we might wanna think that out beyond the trivial case of like `mkRenamedOptionModule`). - not technically distinct from the above, but `core/setups/nixos/`, while in practice mediated by data model resources, now emit NixOS code deduplicated between applications
Owner

probably applications could probably be separate repos, with stated resource versions at the boundary.
that way, they could have their own pins to follow, even if they could still to whatever extent interface with resources that the hosting provider's environment would support across different applications (maybe stock, potentially enhanced).

probably applications could probably be separate repos, with stated resource versions at the boundary. that way, they could have their own pins to follow, even if they could still to whatever extent interface with resources that the hosting provider's environment would support across different applications (maybe stock, potentially enhanced).
Owner

heck resources could be separate repos too - application repos could maybe have resource repos as their deps, such that an environment could override those

heck resources could be separate repos too - application repos could maybe have resource repos as their deps, such that an environment could override those
Author
Member

Answered most of the questions in #1282 and one in #1189.

@kiara wrote in #1288 (comment):

to maybe try and clarify (?), while #304 (implemented as https://swagger.pages.git.fediversity.eu/#/default/schema_get_for) could be phrased as this 'serv[ing] different schemas over the API', one reading of this ticket would be about facilitating swapping out the recommended (so non-foreign) deployment logic / schema, while that existing endpoint is intended to facilitate foreign (many) vs a single default 'recommended' fediversity pin (i.e. flake that rendered core's attributes, which aside from core itself api/panel have today as well).

This is not suggesting swapping out deployment logic, which would be changing
Fediversity or Nixpkgs pins. This is about serving configuration schemas so
panel implementations can render an appropriate form with all the Operator
facing options that the Fediversity deployment at a Hosting Provider expects.

These Operator facing options are influenced by the Nixpkgs revision of the
underlying NixOS module for the Application, by the Fediversity revision
because that module is wrapped and should be influenced by configuration from
the Hosting Provider indicating which options are considered Operator facing.

Answered most of the questions in #1282 and one in #1189. @kiara wrote in https://git.fediversity.eu/fediversity/fediversity/issues/1288#issuecomment-18517: > to maybe try and clarify (?), while #304 (implemented as https://swagger.pages.git.fediversity.eu/#/default/schema_get_for) could be phrased as this 'serv[ing] different schemas over the API', one reading of this ticket would be about facilitating swapping out the _recommended_ (so non-foreign) deployment logic / schema, while that existing endpoint is intended to facilitate foreign (many) vs a single default 'recommended' fediversity pin (i.e. flake that `render`ed core's attributes, which aside from core itself api/panel have today as well). This is not suggesting swapping out deployment logic, which would be changing Fediversity or Nixpkgs pins. This is about serving configuration schemas so panel implementations can render an appropriate form with all the Operator facing options that the Fediversity deployment at a Hosting Provider expects. These Operator facing options are influenced by the Nixpkgs revision of the underlying NixOS module for the Application, by the Fediversity revision because that module is wrapped and should be influenced by configuration from the Hosting Provider indicating which options are considered Operator facing.
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
fediversity/fediversity#1288
No description provided.