fedi-goals/architecture.md

Actors

• Developers

  The group working on this repository. We are creating the deployment workflows and service configurations.

• Hosting provider

  They provide and maintain the physical infrastructure, and run the software in this repository, through which operators interact with their deployments. Hosting providers are technical administrators for these deployments, ensuring availability and appropriate performance.

  We target small- to medium-scale hosting providers with 20+ physical machines.

• Operator

  They select the applications they want to run. They don't need to own hardware or deal with operations. Operators administer their applications in a non-technical fashion, e.g. as moderators. They pay the hosting provider for registering a domain name, maintaining physical resources, and monitoring deployments.

• User

  They are individuals using applications run by the operators, and e.g. post content.

Glossary

• Fediverse

  A collection of social networking applications that can communicate with each other using a common protocol.

• Application

  User-facing software (e.g. from Fediverse) run by the hosting provider for an operator.

• Configuration

  A collection of settings for a machine running NixOS.

  Example: Configurations are deployed to VMs.

• Provision

  Make a resource, such as a virtual machine, available for use.

• Deploy

  Put software, such as applications, onto computers. The software includes technical configuration that links software components. In our context, this a Configuration deployed to a runtime environment. Most user-facing configuration remains untouched by the deployment process.

• Migrate

  Move service configurations and deployment (including user data) from one hosting provider to another.

• Resource

  A resource for NixOps4 is any external entity that can be declared with NixOps4 expressions and manipulated with NixOps4, such as a virtual machine, an active NixOS configuration, a DNS entry, or customer database.

• Resource provider

  A resource provider for NixOps4 is an executable that communicates between a resource and NixOps4 using a standardised protocol, allowing CRUD operations on the resources to be performed by NixOps4. Refer to the NixOps4 manual for details.

  Example: We need a resource provider for obtaining deployment secrets from a database.

• Runtime backend

  A type of digital environment one can run operating systems such as NixOS on, e.g. bare-metal, a hypervisor, or a container runtime.

• Runtime environment

  The thing a deployment runs on, an interface against which the deployment is working. See runtime backend.

• Runtime config

  Configuration logic specific to a runtime backend, e.g. how to deploy, how to access object storage.

Architecture

At the core of Fediversity lies a NixOS configuration template containing selected applications. We use this to deploy to selected run-time environments, so far targeting hypervisor ProxmoX. We further provide a reference front-end to configure our template. To ensure reproducibility, we also offer Nix packaging for our software.

To reach our goals, we aim to implement the following interactions between actors (depicted with rounded corners) and system components (see the glossary, depicted with rectangles).

Entity relationships

Relationships among the entities used to model migrations are as follows, using the crow's foot notation to denote cardinality:

Data model

Host architecture

Whereas the core abstraction in Fediversity is a NixOS configuration template, a more full-fledged example architecture of the web host use-case we aim to support as part of our exploitation would be as follows, where VMs in question run Fediversity to offer our selected applications:

Break-down of project milestones

Whereas details of the implementation may need to be decided as the technical challenges involved become clear, we can already give a higher-level planning of relevant milestones and some of their salient features:

1. Implement a way to run online services in a way that emphasises user autonomy and portability
   • 'one-click' deployment of Fediversity applications
   • 'one-click' portability between hosting providers
2. Disseminate our results by engaging the open-source community to further expand on work in this direction
   • automated dev-ops workflows:
     • separate test environments for staging vs. production
     • continuous deployment
     • CI rejects failing deployments
     • Nix package overlays upstreamed
     • facilitate suggestions in PR reviews
     • integration test
   • external developers empowered to contribute:
     • NixOS configuration as the core abstraction
     • create integration tests
     • add continuous integration builds to a public cache
     • reproduce required infrastructure
     • continuous deployment
     • separate staging/testing environments
     • facilitate user signup
     • facilitate code reviews
     • get documentation ready
     • upstream to NixOS
3. Exploit our work by facilitating the sample use-case of web hosting organisations using an initial set of services
   • Initial internal user to kick-start feedback process:
     • automated provisioning of:
       • databases
       • object storage
       • virtual machines (and related resources)
       • DNS
       • SMTP service
       • email accounts
       • ephemeral state
     • publish specification using e.g. JSON Schema / OpenAPI
     • facilitate multi-tenancy
     • provision admin accounts
     • ensure users can update their configurations
   • Software ready for web hosts to take into production:
     • garbage collection of unallocated resources
     • Nix-less bootstrap
     • ensure hosts may update users' deployments
     • allow control over version of module deployed
     • security audit
     • automate dependency updates
   • Implemented key features to improve user experience:
     • enqueue deployment syncs
     • application data back-ups
     • migrating application data between hosting providers
     • provide single sign-on
     • upstream configuration options exposed
     • allow disabling application while retaining data
     • aid needed user actions on schema update
     • scaling application resources
     • pooling application instances to shared VMs
     • allow use of external single sign-on
     • on migration, allow reconfiguring monolithic vs distributed

Technologies used

NixOS

NixOS is a Linux distribution with a vibrant, reproducible and security-conscious ecosystem. As such, we see NixOS as the only viable way to reliably create a reproducible outcome for all the work we create.

Considered alternatives include:

• containers: do not by themselves offer the needed reproducibility

npins

Npins is a dependency pinning tool for Nix which leaves recursive dependencies explicit, keeping the consumer in control.

Considered alternatives include:

• Flakes: defaults to implicitly following recursive dependencies, leaving control with the publisher.

OpenTofu

OpenTofu is the leading open-source framework for infrastructure-as-code. This has led it to offer a vibrant ecosystem of 'provider' plugins integrating various programs and services. As such, it can facilitate automated deployment pipelines, including with — relevant to our project — hypervisors and DNS programs.

Considered alternatives include:

• Terraform: not open-source

Proxmox

Proxmox is a hypervisor, allowing us to create VMs for our applications while adhering to our goal of preventing lock-in. In addition, it has been packaged for Nix as well, simplifying our requirements to users setting up our software.

Considered alternatives include:

• OpenNebula: seemed less mature

Garage

Garage is a distributed object storage service. For compatibility with existing clients, it reuses the protocol of Amazon S3.

Considered alternatives include:

• file storage: less centralized for backups

PostgreSQL

PostgreSQL is a relational database. It is used by most of our applications.

Considered alternatives include:

• Sqlite: default option for development in many applications, but less optimized for performance, and less centralized for backups

Valkey

Valkey is a key-value store. It is an open-source fork of Redis.

Considered alternatives include:

• Redis: not open-source

OpenSearch

OpenSearch offers full-text search, and is used for this in many applications. It is an open-source fork of ElasticSearch.

Considered alternatives include:

• ElasticSearch: not open-source

PowerDNS

PowerDNS is a mature DNS server. It further offers an admin front-end.

Considered alternatives include:

• hickory-dns: no front-end
• core-dns: no front-end

Authelia

Authelia is a single sign-on provider that integrates with LDAP.

Considered alternatives include:

• KaniDM: does not do proper LDAP
• Authentik: larger package with focus on many things we do not need
• Keycloak: larger package with focus on many things we do not need

lldap

Lldap is a light LDAP server, allowing to centralize user roles across applications.

Considered alternatives include:

• 389 DS: older larger package
• FreeIPA: wrapper around 389 DS