WIP: configuration/README.md: First draft #846
No reviewers
Labels
No labels
ambition
application-offering
ambition
configure-applications
ambition
front-end
ambition/install-applications
ambition
security
ambition
switch-host
ambition
update-applications
ambition
user-management
blocked
component
api-service
component
fediversity-panel
component
nixops4
documentation
points
0
points
0.5
points
1
points
13
points
2
points
21
points
3
points
34
points
5
points
55
points
8
points
infinite
productisation
project-management
question
role
application-developer
role
application-operator
role
hosting-provider
role
maintainer
security
technical debt
testing
type
bug
type
deliverable
type
key-result
type
objective
type
task
type
unclear
type
user-story
user experience
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
fediversity/fediversity!846
Loading…
Reference in a new issue
No description provided.
Delete branch "toonn/fediversity:configuration"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
An attempt to collect my ideas around the topic of configuration.
Primarily to surface misconceptions I might have.
Reviews of both the contents and the form are welcome, where does a document like this actually belong?
maybe https://git.fediversity.eu/fediversity/meta/src/branch/main/architecture-docs for documents not particularly tightly coupled with code (i.e. not expected to evolve with code changes). :)
@ -0,0 +33,4 @@setting it is possible for a new version of Mastodon to drop support. Mastodoncould handle this multiple ways:- Ignore the presence of the environment variable. The configuration wouldan environment variable i would imagine as being at a technical level not directly exposed to operators. rather, an application like mastodon would expose configuration options described in a JSON schema, whose possible settings might potentially correspond to certain environment variables.
if the setting were to be removed from the schema, it could no longer be set in the operator configuration (or, in case we wish to preserve info for the purpose of lossless backward migration, at the very least no longer be passed on to the application's conf file generated from such operator configuration).
as such, i would not foresee this scenario as happening, in the sense it would have been prevented before it would hit this level.
this would further prevent having to reason about the semantics of the configuration settings involved, which i would imagine a machine could not reasonably be expected to reason about like that without like involving stochastic processes
I'm intentionally describing the problem as-is, without considering any planned/potential solutions, as to not fall into the trap of painting ourselves into a corner. Current/potential ideas for solutions definitely deserve either a section in this document or a separate document though.
In a way having schemas that describe allowed configuration options doesn't really change the problem but moves it from upstream into Fediversity. The schema no longer allows an option the Operator has in their current configuration. So Fediversity now has to decide what to do during a deployment. Silently drop the setting as it has been removed from the schema would be the scenario of the first bullet. Refusing to deploy and requiring interaction from the Operator would be like the second bullet point.
There is a difference in that Fediversity can be somewhat of an expert system, falling back to deploying a known-safe configuration instead, imagine something like deploying a Mastodon instance read-only so Users can at least still access existing content in the time until the Operator has dealt with the configuration change.
@ -0,0 +64,4 @@Migrations could carry metadata about the versions of Applications currently inuse. The target Provider could then deploy the version in question andimmediately initiate update protocols if the source version is old (Would thisbe a potential attach vector on Providers? Get a vulnerable deployment at some^ attack (sp.)
@ -0,0 +66,4 @@immediately initiate update protocols if the source version is old (Would thisbe a potential attach vector on Providers? Get a vulnerable deployment at somesmall Provider then request a Migration with the intent of exploiting knownvulnerabilities when deployed at a target Provider), or temporarily run newerthanks, i hadn't thought about this so far -- pinging #291 to cross-link.
so, i would say it would at least be on hosting providers to secure their environments like proxmox to guarantee containment.
even contained tho, you're right hosting providers would have an incentive to prevent drawbacks of vulnerabilities like spam sent from their address or resource usage spikes from crypto mining.
as such, (target) hosting providers may indeed push deployments toward upgrading.
now, given that unified versioning may have drawbacks such as not managing to await operator consent on updates, ideally, i imagine we might be able to utilize CVE metadata on severity levels in the long run to assess risks of maintaining an un-upgraded application, so as to then cross-references such risks with service-level agreements with customers (the operators) to weigh how long to await consent.
now, we may want to settle for a simpler model initially for an 'MVP', so we may want to kind of figure out what such simple scenarios (e.g. initially ignore operator preferences) and paths to nuanced handling might look like and how to order such steps within our broader roadmap.
@ -0,0 +73,4 @@implicitly do through choice of Provider, maybe explicit (limited) control isdesireable. Might serve as a way for Providers to get incremental roll outs.More adventurous Operators are allowed to update beyond the current standarddeployment versions with the understanding that is a risk they choose to take.for what it's worth, we are of course, kinda limited to nix in the sense any application version must have been reproduced to be usable for our purposes. you're right tho theoretically a hosting provider could 'recommend' nixos stable yet allow the choice to follow like unstable - for in as far as such nixpkgs versions wouldn't already end up tightly coupled to our own code (note that, while ideally any of our functionality were available in nixpkgs (and we can upstream things to bring us closer to that goal!), and therefore tested there already, our current codebase has not at all been trivial to swap out nixpkgs versions for, so at the very least this scenario should also involve fediversity versions/channels).
Do you mean that you consider this a strict subproblem of the next section that talks about different Fediversity versions?
I feel like that limits the understanding of the problem by the current implementation/plans of Fediversity. This problem is a consequence of giving control of Application versions to Hosts (#159), whether that can only happen through differences in Fediversity version or not is something I see as orthogonal.
@ -0,0 +93,4 @@## TestsWhat and how can we test?thanks, these scenarios look like good follow-up test cases beyond the base test case in #100 (/cc @Toast).
Something that could be interesting is to test a scenario where the new provider has differing versions for both our applications and fediversity itself, although I'm not sure if we want to support that for anything that's not an upgrade
we may need to specify what we mean by fediversity version there (codebase run by new host when migrating deployments from an old host? the version of one of our components as we switch away from our mono-repo structure?), but yeah
Yes, "Fediversity version" is a bit of a nebulous concept for now, currently I mean the entirety of the framework developed by the Fediversity Team, everything that sits between Nixpkgs and a Hosting Provider.
@ -0,0 +83,4 @@front-ends (a mobile app, for instance) to connect to any Provider's API.How do we decouple the back-end? The revision of Nixpkgs in use should be underthe control of the Provider, not Fediversity. Module options will need to bethis may be optimistic still as our tests can and will fail once you change the nixpkgs pin by any significant amount (e.g. to an update of any of our applications).
that said, a hosting provider could of course recommend a pin (branch+commit) of fediversity, thereby implicitly specifying a nixpkgs version.
to be fair, me saying 'fediversity' there may be a bit of a misnomer in the sense we probably just need to find a sensible way to factor the repo into separate layers that are not tightly coupled, such that different versions of one layer may be able to (to a reasonable extent) work with different versions of another layer.
What I mean here is actually the Nixpkgs that Application packages and modules come from, not the Nixpkgs that Fediversity uses internally. I don't think it's reasonable to put control of the latter in the hands of Hosting Providers. (As an explicit toggle somewhere, Fediversity being an open source project means that they have full control over everything in principle.)
I do think it's important to decouple the Nixpkgs that Applications come from. Deploying a new revision of Nixpkgs should not be delayed by the Fediversity Team pushing a new version of Fediversity.
Another motivation is that a Nixpkgs revision tends to have only one version of Applications packaged (there are exceptions, e.g., compiler versions, but even if they exist they are somewhat cumbersome to use) and a Hosting Provider may not be able/willing to update all Applications at once or they may want to patch an Application ASAP. Being able to overlay Nixpkgs would technically enable this but may not be enough as they cannot really deal with module changes.
@ -0,0 +82,4 @@provide a web-based front-end, Operators should be able to use third-partyfront-ends (a mobile app, for instance) to connect to any Provider's API.How do we decouple the back-end? The revision of Nixpkgs in use should be underi added some sloppy ideas at #100, maybe high-level still but there may be something in thereedit: removed in favor of v
i now updated #100 to mention hosting provider configuration (newly filed as #848) and resource versioning (#634), that should help account for decoupling there
Added a reference to #100.
thanks, i've added a few comments. now, part of the topics i've mentioned should be touched upon in the issue tracker already, tho i hadn't quite bothered to link any reference here just yet.
so we may have more info out here, but i hope this feedback serves as a quick starting point at least.
@ -0,0 +13,4 @@## [Hosting Providers can update configuration][159]Hosting Providers (Providers for short) provide part of the Application(i may have previously used short-hand 'host' - the term 'provider' has felt a bit overloaded bearing meaning both in our data model as well as in nix contracts)
@kiara wrote in #846 (comment):
This sounds right. It'll make a bit of a mess of the review process but better migrate it earlier rather than later. Should I migrate the unresolved discussions from this PR?
Closing in favor of fediversity/meta#38.
Pull request closed