architecture.md

@@ -1,13 +1,11 @@
-<style>* {font-family: sans-serif;}</style>
-
 ## Actors
 
 - Maintainers
 
-  The group maintaining this repository.
+  The group developing and maintaining this project.
-  We are creating the deployment workflows and service configurations, and curate changes proposed by contributing developers.
+  We are creating the deployment workflows and service configurations, and curate changes proposed by contributors.
 
-- Developers
+- Contributors
 
   People with the technical background to engage with our work, and may contribute back, build on top of, remix, or feel inspired by our work to create something better.
 
@@ -56,7 +54,23 @@
 
 - Migrate
 
-  Move service configurations and deployment (including user data) from one hosting provider to another.
+  Move service configurations and deployments (including user data) from one hosting provider to another.
+
+- Runtime backend
+
+  A type of digital environment one can run operating systems such as NixOS on, e.g. bare-metal, a hypervisor, or a container runtime.
+
+- Runtime environment
+
+  The thing a deployment runs on, an interface against which the deployment is working. See runtime backend.
+
+- Runtime configuration
+
+  A specification for mapping components of a configuration to the runtime environment, e.g. which services to deploy to which virtual machines, or how to access object storage.
+
+- [NixOps4](https://nixops.dev)
+
+  A tool to interact with mutable external resources based on declarations in the [Nix language](https://nix.dev/manual/nix/latest/language/).
 
 - Resource
 
@@ -69,21 +83,12 @@
 
   > Example: We need a resource provider for obtaining deployment secrets from a database.
 
-- Runtime backend
-
-  A type of digital environment one can run operating systems such as NixOS on, e.g. bare-metal, a hypervisor, or a container runtime.
-
-- Runtime environment
-
-  The thing a deployment runs on, an interface against which the deployment is working. See runtime backend.
-
-- Runtime config
-
-  Configuration logic specific to a runtime backend, e.g. how to deploy, how to access object storage.
-
 ## Technologies used
 
-### [NixOS](https://nixos.org/)
+This is an incomplete and evolving list of core components planned to be used in this project.
+It will grow to support more advanced use cases as the framework matures.
+
+### Nix and [NixOS](https://nixos.org/)
 
 NixOS is a Linux distribution with a [vibrant](https://repology.org/repositories/graphs), [reproducible](https://reproducible.nixos.org/) and [security-conscious](https://tracker.security.nixos.org/) ecosystem.
 As such, we see NixOS as the only viable way to reliably create a reproducible outcome for all the work we create.
@@ -92,35 +97,6 @@ Considered alternatives include:
 
 - containers: do not by themselves offer the needed reproducibility
 
-#### [npins](https://github.com/andir/npins)
-
-Npins is a dependency pinning tool for Nix which leaves recursive dependencies explicit, keeping the consumer in control.
-
-Considered alternatives include:
-
-- Flakes: defaults to implicitly following recursive dependencies, leaving control with the publisher.
-
-### [SelfHostBlocks](https://nlnet.nl/project/SelfHostBlocks/)
-
-SelfHostBlocks offers Nix module contracts to decouple application configuration from implementation details, empowering user choice by providing sane defaults yet a [unified interface](https://nlnet.nl/project/SelfHostBlocks/).
-Offered contracts include back-ups, reverse proxies, single sign-on and LDAP.
-In addition, we have been in contact with its creator.
-
-Considered alternatives include:
-
-- nixpkgs-provided NixOS service modules: support far more applications, but tightly coupled with service providers, whereas we expect them to [sooner or later](https://discourse.nixos.org/t/pre-rfc-decouple-services-using-structured-typing/58257) follow suit.
-- NixOS service modules curated from scratch: would support any setup imaginable, but does not seem to align as well with our research-oriented goals.
-
-### [OpenTofu](https://opentofu.org/)
-
-OpenTofu is the leading open-source framework for infrastructure-as-code.
-This has led it to offer a vibrant ecosystem of 'provider' plugins integrating various programs and services.
-As such, it can facilitate automated deployment pipelines, including with — relevant to our project — hypervisors and DNS programs.
-
-Considered alternatives include:
-
-- Terraform: not open-source
-
 ### [Proxmox](https://proxmox.com/)
 
 Proxmox is a hypervisor, allowing us to create VMs for our applications while adhering to our goal of preventing lock-in.
@@ -139,75 +115,15 @@ Considered alternatives include:
 
 - file storage: less centralized for backups
 
-### [PostgreSQL](https://www.postgresql.org/)
-
-PostgreSQL is a relational database.
-It is used by most of our applications.
-
-Considered alternatives include:
-
-- Sqlite: default option for development in many applications, but less optimized for performance, and less centralized for backups
-
-### [Valkey](https://valkey.io/)
-
-Valkey is a key-value store.
-It is an open-source fork of Redis.
-
-Considered alternatives include:
-
-- Redis: not open-source
-
-### [OpenSearch](https://opensearch.org/)
-
-OpenSearch offers full-text search, and is used for this in many applications.
-It is an open-source fork of ElasticSearch.
-
-Considered alternatives include:
-
-- ElasticSearch: not open-source
-
-### [OctoDNS](https://github.com/octodns/octodns)
-
-OctoDNS is a DNS server that may be configured using the Nix-native [NixOS-DNS](https://janik-haag.github.io/NixOS-DNS/).
-
-Considered alternatives include:
-
-- PowerDNS: offers a front-end option, but less geared toward the use-case of configuring by Nix
-
-### [Authelia](https://github.com/authelia/authelia)
-
-Authelia is a single sign-on provider that integrates with LDAP.
-
-Considered alternatives include:
-
-- KaniDM: does not do proper LDAP
-- Authentik: larger package with focus on many things we do not need
-- Keycloak: larger package with focus on many things we do not need
-
-### [lldap](https://github.com/lldap/lldap)
-
-Lldap is a light LDAP server, allowing to centralize user roles across applications.
-
-Considered alternatives include:
-
-- 389 DS: older larger package
-- FreeIPA: wrapper around 389 DS
-
-### [Attic](https://github.com/zhaofengli/attic)
-
-Attic is a multi-tenant Nix cache featuring recency-based garbage collection written in Rust.
-
-Considered alternatives include:
-
-- cache-server: distributed cache written in Python that seems more of a research project than an actively maintained repository.
-
 ## Architecture
 
 At the core of Fediversity lies a NixOS configuration module for a set of selected applications.
-We will support using it with different run-time environments, such as a single NixOS machine or a ProxmoX hypervisor.
+
-Depending on the targeted run-time environment, deployment will further involve OpenTofu as an orchestrator.
+- We will enable using it with **different run-time environments**, such as a single NixOS machine or a ProxmoX hypervisor.
-We further provide a [reference front-end](https://git.fediversity.eu/Fediversity/Fediversity/src/branch/main/panel) to configure applications.
+- Depending on the targeted run-time environment, deployment may involve [NixOps4](https://nixops.dev) or [OpenTofu](https://opentofu.org/) as an **orchestrator**.
-To ensure reproducibility, we also offer Nix packaging for our software.
+- We further provide demo front-end for **configuring applications** and configuring **run-time backends**.
+
+To ensure reproducibility, all software will be packaged with Nix.
 
 To reach our goals, we aim to implement the following interactions between [actors](#actors) (depicted with rounded corners) and system components (see the [glossary](#glossary), depicted with rectangles).
 
@@ -217,41 +133,39 @@ To reach our goals, we aim to implement the following interactions between [acto
 
 The process of migrating one's applications to a different host encompasses:
 
-1. domain registration: involves a (manual) update of DNS records at the registrar
+1. Domain registration: involves a (manual) update of DNS records at the registrar
-1. deployed applications: using the reproducible configuration module
+1. Deploy applications: using the reproducible configuration module
-1. application data:
+1. Copy application data:
-    - back-up/restore scripts [using SelfHostBlocks](https://shb.skarabox.com/contracts.html)
+    - Run back-up/restore scripts
-    - application-specific migration scripts, to e.g. reconfigure of connections/URLs
+    - Run application-specific migration scripts, to e.g. reconfigure connections/URLs
 
 ### Data model
 
-Whereas the bulk of our configuration logic is covered in the configuration schema, our reference front-end application does in fact store data.
+Whereas the bulk of our configuration logic is covered in the configuration schema, our reference front-end applications will store data.
-The design for its data model to support the desired functionality is as follows, using the crow's foot notation to denote cardinality:
+The data model design for the configuration front-end needed support the desired functionality is as follows, using the crow's foot notation to denote cardinality:
 
 <img src="https://git.fediversity.eu/Fediversity/meta/raw/branch/main/architecture-docs/panel-data-model.svg" alt="" style="max-width:600px;"/>
 
 ### Host architecture
 
-Whereas the core abstraction in Fediversity is a NixOS configuration module, a more full-fledged example architecture of the web host use-case we aim to support as part of our exploitation would be as follows, where VMs in question run Fediversity to offer our selected applications:
+Whereas the core abstraction in Fediversity is a NixOS configuration module, a more full-fledged example architecture of the web host use-case we aim to support as part of our exploitation would be as follows, where virtual machines in question run Fediversity to offer our selected applications:
 
 ![](https://git.fediversity.eu/Fediversity/meta/raw/branch/main/architecture-docs/host-architecture.png)
 
-## Break-down of project milestones
+## Breakdown of project milestones and key results
 
 Whereas details of the implementation may need to be decided as the technical challenges involved become clear, we can already give a higher-level planning of relevant milestones and some of their salient features:
 
-1. [implement a way to run online services emphasising user autonomy and data portability](https://git.fediversity.eu/Fediversity/Fediversity/issues/347)
+- Implement a way to run online services emphasizing user autonomy and data portability
-    1. [Finalize architecture doc](https://git.fediversity.eu/Fediversity/Fediversity/issues/39)
+    - Integration tests pass for
-    1. [code-based migration data model](https://git.fediversity.eu/Fediversity/Fediversity/issues/103)
+        - Setting up a fediversity hosting environment from a declarative configuration
-    1. [migrating application data between hosting providers](https://git.fediversity.eu/Fediversity/Fediversity/issues/100)
+        - Configuring, deploying, and migrating a set of dummy applications
-    1. [application offering generalised](https://git.fediversity.eu/Fediversity/Fediversity/issues/369)
+    - Code passes data protection audit
-    1. [NixOS configuration as the core abstraction](https://git.fediversity.eu/Fediversity/Fediversity/issues/339)
+- Disseminate our results by engaging the open-source community to further expand on work in this direction
-1. [disseminate our results by engaging the open-source community to further expand on work in this direction](https://git.fediversity.eu/Fediversity/Fediversity/issues/348)
+    - Present results on at least 3 conferences
-    1. [automated dev-ops workflows](https://git.fediversity.eu/Fediversity/Fediversity/issues/224)
+    - At least 5 applications compatible with Fediversity thanks to external contributions by 2027-03
-    1. [external developers empowered to contribute](https://git.fediversity.eu/Fediversity/Fediversity/issues/288)
+- Exploit our work by enabling reproducible deployments of an initial set of portable applications
-1. [exploit our work by enabling reproducible deployments of an initial set of portable applications](https://git.fediversity.eu/Fediversity/Fediversity/issues/349)
+    - There are 3 fediverse applications available out of the box:
-    1. [applications deployed on command](https://git.fediversity.eu/Fediversity/Fediversity/issues/99)
+        - Mastodon
-    1. [kick-started initial feedback cycle](https://git.fediversity.eu/Fediversity/Fediversity/issues/225)
+        - PeerTube
-    1. [brought into production](https://git.fediversity.eu/Fediversity/Fediversity/issues/228)
+        - Pixelfed
-    1. [nix-less bootstrap](https://git.fediversity.eu/Fediversity/Fediversity/issues/332)
-    1. [key features improving user experience supported](https://git.fediversity.eu/Fediversity/Fediversity/issues/289)