Hosting Providers control their offering #1282

Open
opened 2026-08-04 21:06:19 +02:00 by toonn · 4 comments
Member

As a Hosting Provider,
I want to be able to control what Applications I offer,
so that I can guarantee the quality of my service and attract Operators by offering more/different Applications.

A Hosting Provider might not want to offer a Matrix Application, because their hardware is not performant enough with Synapse. They might want to offer Matrix Applications but using different server implementations, configured with different NixOS modules. Or they might want to offer a Mattermost Application, something not currently planned as a supported Application for Fediversity, to attract Operators.

Since Fediversity is an open source project, all of this is possible through forking the project. That comes with a cost though. Any patches may need to be altered whenever Fediversity releases a new version.

We should offer configuration options that allow Hosting Providers to provide all the information Fediversity needs in order to deploy an Application.

This addresses point 2 of the discussion here.

**As** a Hosting Provider, **I want** to be able to control what Applications I offer, **so that** I can guarantee the quality of my service and attract Operators by offering more/different Applications. A Hosting Provider might not want to offer a Matrix Application, because their hardware is not performant enough with Synapse. They might want to offer Matrix Applications but using different server implementations, configured with different NixOS modules. Or they might want to offer a Mattermost Application, something not currently planned as a supported Application for Fediversity, to attract Operators. Since Fediversity is an open source project, all of this is possible through forking the project. That comes with a cost though. Any patches may need to be altered whenever Fediversity releases a new version. We should offer configuration options that allow Hosting Providers to provide all the information Fediversity needs in order to deploy an Application. This addresses point 2 of the [discussion here](https://git.fediversity.eu/fediversity/module-schema/issues/12).
Owner

i linked this as #1189's sub-ticket focused on applications.

Since Fediversity is an open source project, all of this is possible through forking the project. That comes with a cost though. Any patches may need to be altered whenever Fediversity releases a new version.

so, there would seem to be different ways to go around this.

  • a configuration-based way (e.g. HOSTING_ENV_CONFIG)
    • raises questions on to what extent such applications could be migrated to different hosting providers (#100), and what keeping them updated after a hypothetical migration might look like
  • as a consuming code-base that re-exports core/'s attributes, as introduced for the environment entity in #1202 (as part of #1189 had yet to be extended to idiomatically support the application entity)
i linked this as #1189's sub-ticket focused on applications. > Since Fediversity is an open source project, all of this is possible through forking the project. That comes with a cost though. Any patches may need to be altered whenever Fediversity releases a new version. so, there would seem to be different ways to go around this. - a configuration-based way (e.g. `HOSTING_ENV_CONFIG`) - raises questions on to what extent such applications could be migrated to different hosting providers (#100), and what keeping them updated after a hypothetical migration might look like - as a consuming code-base that re-exports `core/`'s attributes, as introduced for the environment entity in #1202 (as part of #1189 had yet to be extended to idiomatically support the `application` entity)
Owner

i updated #1189 w.r.t. applications: so far added like how api/panel applications are injected despite lacking from core, tho this mechanism seems hacky (custom helpers) rather than idiomatic data model extension still

i updated #1189 w.r.t. applications: so far added like how api/panel applications are injected despite lacking from core, tho this mechanism seems hacky (custom helpers) rather than idiomatic data model extension still
Author
Member

I'm answering parts of the comments on #1288 here, because I think it's closer
to the topic of this issue.

@kiara wrote in #1288 (comment):

raises questions on to what extent such applications could be migrated to different hosting providers (#100), and what keeping them updated after a hypothetical migration might look like

So, migrations are a complicated beast, giving Hosting Providers the power to
implement their own Applications complicates that. But, consider the
alternative, we don't make it possible to add/remove Applications easily, the
Hosting Providers business interests don't change, they want to (not) offer
some Application, so they fork Fediversity and remove or implement the
Application. Now we're in a similar situation regarding migrations, except
they're not even running the same revision of Fediversity. That sounds strictly
more difficult to deal with to me.

you noted tensions between supporting this vs 'real-world use-cases' at fediversity/module-schema#12 (comment), and i'd love to not be bound by this feature (as scrolling down #100 should reveal how many of my concerns that'd help dissipate) - tho given NLNet pitched the project specifically on that feature, i'd consider backing out on it above my pay-grade - so then we'd likely need to prepare well-founded considerations on this to set expectations with @michielleenaars. tbf, to my knowledge our mission is less to ace every theoretical feature in the sense we are essentially exploring this space, and "this involves trade-offs, which we'd try and navigate as per real-world priorities" certainly seems one valid outcome there.

for what it's worth, i maybe have some hope still there may be ways to reconcile things, so would like to maybe still better understand the underlying desire(s) here.

The code that is necessary to facilitate migrations would be part of the
contract of what implementing an Application means. Granted, incentives for
Hosting Providers to implement migrations properly may not be high but if they
fork Fediversity there's also no reason to implement them right.

At the same time I think we need to consider partial/per-Application migrations
anyway. I might be happy hosting my e-mail and a Matrix server at a small host
that I know personally but they just don't seem to have the capacity to host my
Peertube instance smoothly. I'd want to migrate just the Peertube instance to a
bigger Hosting Provider.

Being able to migrate each Application individually also allows doing a gradual
migration, to build trust in the new Hosting Provider. Or a migration to a
Hosting Provider that can handle everything except for Peertube, which you'd
maintain at your current Hosting Provider. A full migration would presumably be
easy to implement on top of per-Application migrations.

The following questions stray a bit from this issue, I think they're really
addressing point 1 of fediversity/module-schema#12 but I don't have a better
place for them.

@kiara wrote in #1288 (comment):

heck, now that i think of it, i think my concern doesn't need #100: if an operator hasn't updated yet, their pin would already be 'foreign' anyway - how would one know what updating might mean there (i.e. how to change underlying core/ code), if not by a VCS-backed pin, given this ticket would ostensibly steer away from that?

How does the revision being in VCS help with determining whether something is
an update? Maybe you're assuming Fediversity's VCS history has both revisions
involved and we can pick the more recent one as the one to update to? What
about a fork with a divergent history of Nixpkgs revisions?

@kiara wrote in #1288 (comment):

if not by a VCS-backed pin

to rephrase that: if we have a deployment (or theoretically a deployed application, for that matter), we want to know what updating it means. we now do this with a fetchTree-compatible reference to a given revision of a given branch on a git repository (now fediversity). if code were no longer in any type of version control, i think what updating means there becomes unclear, if it's no longer "keep following the branch" - including after migrations elsewhere. in other words, who maintains used code (or configuration, if you'd like to call it that), including after a theoretical migration (#100)?

I'm not sure I follow. If Hosting Providers determine what Nixpkgs revision to
use then they decide when to use another one, which would presumably be an
update.

I'm answering parts of the comments on #1288 here, because I think it's closer to the topic of this issue. @kiara wrote in https://git.fediversity.eu/fediversity/fediversity/issues/1288#issuecomment-18517: > > raises questions on to what extent such applications could be migrated to different hosting providers (#100), and what keeping them updated after a hypothetical migration might look like So, migrations are a complicated beast, giving Hosting Providers the power to implement their own Applications complicates that. But, consider the alternative, we don't make it possible to add/remove Applications easily, the Hosting Providers business interests don't change, they want to (not) offer some Application, so they fork Fediversity and remove or implement the Application. Now we're in a similar situation regarding migrations, except they're not even running the same revision of Fediversity. That sounds strictly more difficult to deal with to me. > you noted tensions between supporting this vs 'real-world use-cases' at fediversity/module-schema#12 (comment), and i'd love to not be bound by this feature (as scrolling down #100 should reveal how many of my concerns that'd help dissipate) - tho given NLNet pitched the project specifically on that feature, i'd consider backing out on it above my pay-grade - so then we'd likely need to prepare well-founded considerations on this to set expectations with @michielleenaars. tbf, to my knowledge our mission is less to ace every theoretical feature in the sense we _are_ essentially exploring this space, and "this involves trade-offs, which we'd try and navigate as per real-world priorities" certainly seems one valid outcome there. > > for what it's worth, i maybe have some hope still there may be ways to reconcile things, so would like to maybe still better understand the underlying desire(s) here. The code that is necessary to facilitate migrations would be part of the contract of what implementing an Application means. Granted, incentives for Hosting Providers to implement migrations properly may not be high but if they fork Fediversity there's also no reason to implement them right. At the same time I think we need to consider partial/per-Application migrations anyway. I might be happy hosting my e-mail and a Matrix server at a small host that I know personally but they just don't seem to have the capacity to host my Peertube instance smoothly. I'd want to migrate just the Peertube instance to a bigger Hosting Provider. Being able to migrate each Application individually also allows doing a gradual migration, to build trust in the new Hosting Provider. Or a migration to a Hosting Provider that can handle everything except for Peertube, which you'd maintain at your current Hosting Provider. A full migration would presumably be easy to implement on top of per-Application migrations. The following questions stray a bit from this issue, I think they're really addressing point 1 of fediversity/module-schema#12 but I don't have a better place for them. @kiara wrote in https://git.fediversity.eu/fediversity/fediversity/issues/1288#issuecomment-18523: > heck, now that i think of it, i think my concern doesn't need #100: if an operator hasn't updated yet, their pin would already be 'foreign' anyway - how would one know what updating might mean there (i.e. how to change underlying `core/` code), if not by a VCS-backed pin, given this ticket would ostensibly steer away from that? How does the revision being in VCS help with determining whether something is an update? Maybe you're assuming Fediversity's VCS history has both revisions involved and we can pick the more recent one as the one to update to? What about a fork with a divergent history of Nixpkgs revisions? @kiara wrote in https://git.fediversity.eu/fediversity/fediversity/issues/1288#issuecomment-18525: > > if not by a VCS-backed pin > > to rephrase that: if we have a deployment (or theoretically a deployed application, for that matter), we want to know what updating it means. we now do this with a `fetchTree`-compatible reference to a given revision of a given branch on a git repository (now fediversity). if code were no longer in any type of version control, i think what updating means there becomes unclear, if it's no longer "keep following the branch" - including after migrations elsewhere. in other words, who maintains used code (or configuration, if you'd like to call it that), including after a theoretical migration (#100)? I'm not sure I follow. If Hosting Providers determine what Nixpkgs revision to use then they decide when to use another one, which would presumably be an update.
Owner

so they fork Fediversity [...]. That sounds strictly more difficult to deal with to me.

they don't have to fork for this: as per core.lib.render (implemented part of #1189) one may make a downstream distribution, as we do today (core->api->panel) - no forking involved, necessarily.
so while some code may need to stay aligned at some point on update, this doesn't just imply hundreds of merge conflicts, per se - we'd expose clear boundaries.

migrate each Application individually

this may merit its own ticket

A full migration would presumably be easy to implement on top of per-Application migrations.

i would technically agree here, tho only because partial ones are probably are probably more involved than the full thing

How does the revision being in VCS help with determining whether something is an update?
Maybe you're assuming Fediversity's VCS history has both revisions involved and we can pick the more recent one as the one to update to?

what did you mean by "whether something is an update"?
further, what did you mean by 'both versions'? this to me implies one would know future versions to update to (upfront?).
so our current updating model (module_source, shown at the bottom of panel/'s operator form) uses fetchtree objects, like how flakes track dependencies, using objects like:

{
  type = "github";
  owner = "NixOS";
  repo = "nixpkgs";
  ref = "master";
  rev = "ae2e6b3958682513d28f7d633734571fb18285dd";
}

(except in our case pointing to our fediversity repo today.)
the point there is, as long as you know the location (rev) and a direction (ref), you don't just know where you are today, you've also specified a direction the deployment can update in - just like how flakes/npins updates for a dep could get you to a newer revision on your dependency (whether a new release, tag, or a commit on some branch).

this then also specifies the current responsibility model over a deployment: unless change somehow (e.g. fork, downstream distribution - tho friendly mechanisms to change the module source are todo), updates would follow what is published at the specified repo branch.

What about a fork with a divergent history of Nixpkgs revisions?

as long as the target revision there is at least as recent (if not resource versions too), then that should work, i think: any intermediate versions could just get skipped, if i understand the scenario you're describing correctly.

If Hosting Providers determine what Nixpkgs revision to use then they decide when to use another one, which would presumably be an update.

right so i think what i'm describing is a submitted deployment providing all the info declaratively describing where it is and can go, while you're describing (nixpkgs) versions as maybe somewhat more of a separate property of deployments, rather than the whole thing being kind of self-contained

> so they fork Fediversity [...]. That sounds strictly more difficult to deal with to me. they don't have to fork for this: as per `core.lib.render` (implemented part of #1189) one may make a downstream distribution, as we do today (core->api->panel) - no forking involved, necessarily. so while some code may need to stay aligned at some point on update, this doesn't just imply hundreds of merge conflicts, per se - we'd expose clear boundaries. > migrate each Application individually this may merit its own ticket > A full migration would presumably be easy to implement on top of per-Application migrations. i would technically agree here, tho only because partial ones are probably are probably more involved than the full thing > How does the revision being in VCS help with determining whether something is an update? > Maybe you're assuming Fediversity's VCS history has both revisions involved and we can pick the more recent one as the one to update to? what did you mean by "whether something is an update"? further, what did you mean by 'both versions'? this to me implies one would know future versions to update to (upfront?). so our current updating model (`module_source`, shown at the bottom of `panel/`'s operator form) uses [fetchtree](https://noogle.dev/f/builtins/fetchTree/) objects, like how flakes track dependencies, using objects like: ```nix { type = "github"; owner = "NixOS"; repo = "nixpkgs"; ref = "master"; rev = "ae2e6b3958682513d28f7d633734571fb18285dd"; } ``` (except in our case pointing to our fediversity repo today.) the point there is, as long as you know the location (`rev`) and a direction (`ref`), you don't just know where you are today, you've also specified a direction the deployment can update in - just like how flakes/npins updates for a dep could get you to a newer revision on your dependency (whether a new release, tag, or a commit on some branch). this then also specifies the current responsibility model over a deployment: unless change somehow (e.g. fork, downstream distribution - tho friendly mechanisms to change the module source are todo), updates would follow what is published at the specified repo branch. > What about a fork with a divergent history of Nixpkgs revisions? as long as the target revision there is at least as recent (if not resource versions too), then that should work, i think: any intermediate versions could just get skipped, if i understand the scenario you're describing correctly. > If Hosting Providers determine what Nixpkgs revision to use then they decide when to use another one, which would presumably be an update. right so i think what i'm describing is a submitted deployment providing all the info declaratively describing where it is and can go, while you're describing (nixpkgs) versions as maybe somewhat more of a separate property of deployments, rather than the whole thing being kind of self-contained
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
fediversity/fediversity#1282
No description provided.