Compare commits

..

30 commits

Author SHA1 Message Date
638f387458
investment questionnaire
Signed-off-by: cinereal <cinereal@riseup.net>
2025-09-09 13:57:10 +02:00
cb403c8a60
add placeholder
Signed-off-by: cinereal <cinereal@riseup.net>
2025-09-09 12:43:08 +02:00
6ecf75b76e
set graph width for print documents
Signed-off-by: cinereal <cinereal@riseup.net>
2025-09-09 12:38:58 +02:00
4820b0dacf Delete Fediverse-investments-an-estimation.md
remove old version, new ones exist in https://git.fediversity.eu/Fediversity/meta/src/branch/main/investment_analysis
2025-09-08 14:52:22 +02:00
cfcb37384a Update investment_analysis/README.md
changed link to version that keeps existing
2025-09-08 14:51:38 +02:00
8d7ab31606 Update investment_analysis/README.md 2025-09-08 14:48:52 +02:00
54fcf0aefe add draft report 2025-09-08 14:47:08 +02:00
c8db81ffe9 Update investment_analysis/readme.md 2025-09-08 14:45:50 +02:00
1f71504de7 new path to point the report to 2025-07-30 17:46:58 +02:00
07f2f9471d 2025-07-22 developer sync 2025-07-22 11:27:13 +02:00
fa2900a0e0
transparent PNGs 2025-07-16 16:03:26 +02:00
9c0606b9a5
add png versions 2025-07-16 15:56:49 +02:00
27d621de66 Merge pull request 'describe CI/CD' (#34) from architecture-ci into main
Reviewed-on: #34
Reviewed-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2025-07-11 09:36:12 +02:00
6b6984ef02
describe CI/CD
see D2.6
2025-07-10 17:20:10 +02:00
e5e608f2c9
in-source architecture images 2025-07-10 17:20:01 +02:00
2b8507ae78
add nix shell with needed packages 2025-07-10 17:19:24 +02:00
798522c747 add REST API outline as discussed 2025-07-10 00:18:31 +02:00
3d55b459c2
add architecture document 2025-07-09 15:41:51 +02:00
82a86f4515 add configuration-to-deployment dataflow diagram
side note: the pile of documents in this directory would benefit from some cleanup,
because it tends to get confusing to navigate it. e.g. we can render images programmatically
from the source rather than checking them in.

currently we're conflating or inconsistently separating two document types:
- meeting notes and design decisions (capturing point-in-time discussion that is append-only)
- technical specification such as architecture diagrams (which change over time to reflect current understanding)

we'll keep piling up for now, but ultimately if we need to do onboarding or hand-over
to new developers or contributors, this should be sorted out.
2025-07-08 12:57:26 +02:00
343f77ef61 2025-07-08 weekly planning 2025-07-08 11:04:18 +02:00
5394b0253f 2025-07-03 data model review 2025-07-03 13:19:44 +02:00
247325db93 2025-07-01 developer sync 2025-07-01 11:29:19 +02:00
7cb91acb3a
fix \n 2025-06-25 17:50:22 +02:00
baf21e1603
add nixos frontend to simple architecture diagram 2025-06-25 17:29:14 +02:00
08bb73aabb
make migration interaction diagram use a legend similar to the fediversity interactions diagram 2025-06-24 16:27:39 +02:00
2c12fc677c
add interactions diagram 2025-06-24 15:40:16 +02:00
6514843244 Merge pull request 'initial gantt chart for proposal rewrite' (#33) from proposal-gantt into main
Reviewed-on: #33
2025-06-12 17:23:37 +02:00
2eb7b855b6 2025-06-12 developer sync 2025-06-11 18:14:14 +02:00
7d68f6fef8 add data model for the reference front-end (#31)
follows up from #31 to offer a data model focused on formalize to the point of incorporating any data attribute types to be stored in the `panel` db.

data model [requirements](https://git.fediversity.eu/Fediversity/meta/src/branch/main/architecture-docs/data-model-requirements.md):

- [x] specifying [entity relations](https://mermaid.js.org/syntax/entityRelationshipDiagram.html#relationship-syntax) e.g. many-to-many
- [x] migrating both deployed and staged configurations
- [x] deploying of applications using the same versions
- [x] retaining relevant application state
- handling of application-specific migration logic, such as to rewrite URLs as needed - note this requirement does not affect the model here, in the sense it is handled through code rather than through the database

closes Fediversity/Fediversity#103.

Reviewed-on: #31
Reviewed-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: cinereal <cinereal@riseup.net>
Co-committed-by: cinereal <cinereal@riseup.net>
2025-06-06 11:05:27 +02:00
ebaf20d558
in interactions rename developers to maintainers
follows
8764276d39
2025-06-04 22:53:18 +02:00
24 changed files with 649 additions and 98 deletions

View file

@ -1,88 +0,0 @@
Outline
> Release a report on the estimated collective current investment on fediverse technology:
>
> Look at qualifiers like:
>- Used system resources (CPU, storage, networking)
>- Amount of software developers actively working on Fediverse software >products
>- Amount of system administration engineers actively working on Fediverse software products
>- Amount of moderation and governance people
* * *
# An estimate of investments in the Fediverse
## Introduction
To provide an estimate of the investments made into the Fediverse, is a very daunting task due to decentralised nature of the Fediverse, the amount of volunteers contributing freely and - in our experience - a lack of research on this topic thus far. Nonetheless, we have taken upon ourselves the task to provide a first glimpse on this subject.
[TODO: describe 'the why' better]
We hope our work will contribute and ignite more serious research into this topic.
In this document we describe our assessment of the investments made in the Fediverse. We start with how we define the Fediverse, describe our methodology for gathering data on the defined Fediverse and share information on the services, platforms and software applications we have deemed relevant to include.
[TODO: need to add more info here ]
* * *
## Define the Fediverse
We need to define what the Fediverse is.
Wikipedia defines Fediverse as "a collection of social networking services that can communicate with each other (formally known as federation) using a common protocol. Users of different websites can send and receive status updates, multimedia files and other data across the network. The term Fediverse is a portmanteau of "federation" and "universe". (Wikipedia contributors, 2024)[^1]".
Erin Kissane & Darius Kazemi in 'Findings Report: Governance on Fediverse Microblogging Servers' define the Fediverse as "a decentralized interoperable network of social media sites, apps, and services built on the ActivityPub protocol.[^2]".
[ TODO: add the other definitions ]
It seems there has not yet formed consensus on what exactly are the attributes which could define a platform, service or software application part of the Fediverse.
## Methodology
* Numbers used & why these numbers and not other numbers
* Services / platforms investigated
* Why these services & platforms?
* Software applications
* Topic opened: https://socialhub.activitypub.rocks/t/any-info-on-the-estimated-collective-investments-time-money-on-fediverse-tech/4844?u=bjornw
## A closer look at the Fediverse platforms
* Name of service/platform
* Description of service/platform
* estimated nr of installations
* estimated nr of users (Monthly Active Users)
* estimated nr of developers working on the service
* estimated nr of moderators
* estimated costs for the service
## Estimating investments in the Fediverse
* Conclusion
## Notes
[^1]: Wikipedia contributors. (2024, December 29). Fediverse. Wikipedia. https://en.wikipedia.org/wiki/Fediverse
[^2]: Kissane, E., & Kazemi, D. (2024, September 4). Findings Report: Governance on Fediverse Microblogging Servers. https://fediverse-governance.github.io/#brief-glossary
## References
* https://en.wikipedia.org/wiki/Fediverse
* https://fedidb.org/
* https://mastodon-analytics.com/
* https://www.thinkimpact.com/mastodon-statistics/
* https://fediverse.observer/
* https://the-federation.info/
* https://fediverse.party
* https://socialhub.activitypub.rocks/t/fep-f1d5-nodeinfo-in-fediverse-software/1190
* https://codeberg.org/fediverse
https://hachyderm.io/@esk/113793277371908181

View file

@ -1,4 +1,4 @@
// usage: nix-shell -p graphviz --command dot -T png -o architecture.png architecture.dot // usage: nix-shell -p graphviz --command 'dot -T png -o architecture.png architecture.dot'
digraph FediversityArchitecture { digraph FediversityArchitecture {
graph[fontname="Liberation Sans"]; graph[fontname="Liberation Sans"];
@ -7,11 +7,13 @@ digraph FediversityArchitecture {
subgraph cluster_frontends { subgraph cluster_frontends {
label="front-ends"; label="front-ends";
nix[label="NixOS module"];
tofu[label="OpenTofu CLI"]; tofu[label="OpenTofu CLI"];
panel[label="FediPanel"]; panel[label="FediPanel"];
protagio[label="NixPanel"]; protagio[label="NixPanel"];
} }
nix -> core;
tofu -> core; tofu -> core;
panel -> core; panel -> core;
protagio -> core; protagio -> core;

Binary file not shown.

Before

Width:  |  Height:  |  Size: 24 KiB

After

Width:  |  Height:  |  Size: 27 KiB

View file

@ -0,0 +1,177 @@
# Fediversity Implementation and planning
## Actors
- Maintainers
The group developing and maintaining this project.
We are creating the deployment workflows and service configurations, and curate changes proposed by contributors.
- Developers
People with the technical background to engage with our work, and may contribute back, build on top of, remix, or feel inspired by our work to create something better.
- Hosting provider
They provide and maintain the physical infrastructure, and run the software in this repository, through which operators interact with their deployments.
Hosting providers are technical administrators for these deployments, ensuring availability and appropriate performance.
We target small- to medium-scale hosting providers with 20+ physical machines.
- Operator
They select the applications they want to run.
They don't need to own hardware or deal with operations.
Operators administer their applications in a non-technical fashion, e.g. as moderators.
They pay the hosting provider for registering a domain name, maintaining physical resources, and monitoring deployments.
- User
They are individuals using applications run by the operators, and e.g. post content.
## Glossary
- [Fediverse](https://en.wikipedia.org/wiki/Fediverse)
A collection of social networking applications that can communicate with each other using a common protocol.
- Application
User-facing software (e.g. from Fediverse) configured by operators and used by users.
- Configuration
A collection of settings for a piece of software.
> Example: Configurations are deployed to VMs.
- Provision
Make a resource, such as a virtual machine, available for use.
- Deploy
Put software onto computers.
The software includes technical configuration that links software components.
- Migrate
Move service configurations and deployments (including user data) from one hosting provider to another.
- Run-time backend
A type of digital environment one can run operating systems such as NixOS on, e.g. bare-metal, a hypervisor, or a container run-time.
- Provider
An interface against which we deploy to a run-time backend.
- Provider configuration
A configuration that specifies resources made available to deploy to and how to access these.
- Resource
A resource is any external entity that we need for our set-up
This may include e.g. hypervisors, file systems, DNS entries, VMs or object storage instances.
## Technologies used
This is an incomplete and evolving list of core components planned to be used in this project.
It will grow to support more advanced use cases as the framework matures.
### Nix and [NixOS](https://nixos.org/)
NixOS is a Linux distribution with a [vibrant](https://repology.org/repositories/graphs), [reproducible](https://reproducible.nixos.org/) and [security-conscious](https://tracker.security.nixos.org/) ecosystem.
As such, we see NixOS as the only viable way to reliably create a reproducible outcome for all the work we create.
Considered alternatives include:
- containers: do not by themselves offer the needed reproducibility
### [Proxmox](https://proxmox.com/)
Proxmox is a hypervisor, allowing us to create VMs for our applications while adhering to our goal of preventing lock-in.
In addition, it has been [packaged for Nix](https://github.com/SaumonNet/proxmox-nixos) as well, simplifying our requirements to users setting up our software.
Considered alternatives include:
- OpenNebula: seemed less mature
### [Garage](https://garagehq.deuxfleurs.fr/)
Garage is a distributed object storage service.
For compatibility with existing clients, it reuses the protocol of Amazon S3.
Considered alternatives include:
- file storage: less centralized for backups
## Architecture
At the core of Fediversity lies a NixOS configuration module for a set of selected applications.
- We will enable using it with **different run-time environments**, such as a single NixOS machine or a ProxmoX hypervisor.
- Depending on the targeted run-time environment, deployment may involve [NixOps4](https://nixops.dev) or [OpenTofu](https://opentofu.org/) as an **orchestrator**.
- We further provide demo front-end for **configuring applications** and configuring **run-time backends**.
To ensure reproducibility, all software will be packaged with Nix.
To reach our goals, we aim to implement the following interactions.
The used legend is as follows:
- Circle: [actor](#actors)
- Angled box: type
- Rectangle: value
- Rounded box: function
- Diamond: state
- Arrow: points towards dependant
For further info on components see the [glossary](#glossary).
<!-- ![](./interactions-migration.svg) -->
![](./interactions-migration.png){ width=100% }
### Configuration data flow
This data flow diagram refines how a deployment is obtained from an operator's application configuration and a hosting provider's runtime setup.
An **application module** specifies operator-facing **application options**, and a **configuration mapping** which determines the application's underlying implementation. Application modules can be supplied by external developers, which would curate application modules against that interface.
For its runtime setup, a hosting provider has to supply a **resource mapping** that would take their self-declared **provider configuration** (which determines the *available* resources) and the output of an application's resource mapping (which determine resource *requirements*) and produce a **configuration**. This configuration ships with a mechanism to be *deployed* to the infrastructure (which is described by the environment, and features the required resources), where it will accumulate **application state**.
Applications and runtime environments thus interface through **resources**, the properties of which are curated by Fediversity maintainers.
<!-- ![](./interactions-fediversity.svg) -->
![](./interactions-fediversity.png){ width=100% }
### Service portability
The process of migrating one's applications to a different host encompasses:
1. Domain registration: involves a (manual) update of DNS records at the registrar
1. Deploy applications: using the reproducible configuration module
1. Copy application data:
- Run back-up/restore scripts
- Run application-specific migration scripts, to e.g. reconfigure connections/URLs
### Data model
Whereas the bulk of our configuration logic is covered in the configuration schema, [implemented here](https://git.fediversity.eu/Fediversity/Fediversity/src/branch/main/deployment/data-model.nix) and [tested here](https://git.fediversity.eu/Fediversity/Fediversity/src/branch/main/deployment/data-model-test.nix), our reference front-end applications will store data.
The data model design for the configuration front-end needed support the desired functionality is as follows, using the crow's foot notation to denote cardinality:
<!-- ![](./panel-data-model.svg) -->
![](./panel-data-model.png){ height=100% }
### Host architecture
Whereas the core abstraction in Fediversity is a NixOS configuration module, a more full-fledged example architecture of the web host use-case we aim to support as part of our exploitation would be as follows, where virtual machines in question run Fediversity to offer our selected applications:
![](./host-architecture.png){ width=100% }
### CI / CD
In our simplest set-up, continuous integration and continuous deployment pipelines are handled using Forgejo's [built-in runner](https://code.forgejo.org/forgejo/runner), with relevant secrets handled using [Forgejo secrets](https://forgejo.org/docs/latest/developer/secrets/). Jobs we handle using CI include linting, formatting, testing, and a periodic life-cycle management job to keep our dependencies up-to-date.
In a future iteration, we may make use of [Gerrit](https://gerrit.googlesource.com/) to better manage our review process for incoming merge requests.

View file

@ -0,0 +1,74 @@
# 2025-06-24 Fediversity configuration data flow
This data flow diagram refines the [global architecture diagram](./interactions-migration.mmd) with respect to how a deployment is obtained from an operator's application configuration and a hosting provider's runtime setup.
## Legend
- Circle: actor
- Angled box: type
- Rectangle: value
- Rounded box: function
- Diamond: state
- Arrow: points towards dependant
## Description
An **application module** specifies operator-facing **application options**, and a **resource mapping** which determines the application's underlying implementation in terms of which resources it requires to be deployed. Application modules can be supplied by external **contributors**, which would package applications against that interface.
For its **runtime environment**, a **hosting provider** has to supply a **policy mapping** that would take their self-declared **resource configuration** (which determines *available* resources, i.e. a description of the infrastructure on which to run the applications) and the output of an application's resource mapping (which determines resource *requirements*) and produce the description of a **deployment**.
An **operator** can supply an **application configuration** and request the hosting provider to let the resulting deployment take effect, such that it will accumulate **application state**, which can later be migrated to other hosting providers.
Applications and runtime environments thus interface through **resources**, the properties of which are curated by Fediversity **maintainers**.
```mermaid
flowchart
subgraph configuration
application-config
end
subgraph resource[resource module]
resource-options
provider-options
end
subgraph application[application module]
application-options{{application-options}} --> application-config
application-config --> config-mapping
resource-options{{consumer-options}} --> config-mapping
config-mapping(resource-mapping)
end
subgraph fediversity[runtime-environment]
config-mapping -->|required resources| resource-mapping(policy-mapping)
provider-options{{resource-options}} --> resource-config --> |available resources| resource-mapping -->|configured resources| deployment[deployment]
deployment --> |deploy| state{application state}
end
maintainer((maintainer)) -->|curates| resource
contributor((contributor)) -->|packages| application
operator((operator)) -->|enters| configuration
hosting-provider((hosting provider)) -->|maintains| fediversity
```
## Discussion
Decoupling of operator-facing application configuration and resource use allows for portability at the infrastructure level: hosting providers determine how applications are deployed to available resources through policies which are agnostic to applications. Additionally it allows hosting providers to isolate applications from each other depending on their deployment model, leveraging the entire toolkit provided by the Nix ecosystem. This puts single-machine and cluster deployments on equal footing, since policies are simply Nix language functions.
At first glance this is an increase in complexity as opposed to configuring NixOS directly, since it adds a layer of indirection. It also requries Fediversity maintainers to declare resource types for use by application packagers and hosting providers. But the layering drastically simplifies application composition over what NixOS can currently offer, frees the hosting provider to choose how to implement required services, and enables security-conscious deployments at very fine granularity.
## Next steps
- Versioning: Each of the components can change; model all supported interactions such that deployments will always be correct.
Primarily, both applications and runtime environments depend on the resource API, and need to be versioned against it. Application packages can be updated on top of that, and will need to provide facilities to migrate between both application versions and resource API versions, both in terms of configurations and application state.
- Provider migrations: Prototype an application-agnostic migration workflow.
Since state migration is application-specific, application packages will need to ship with instructions for how to move their data between hosting providers, also taking version migrations into account. This will likely require extending the resource API with ways to express where state is located, and give hosting providers the tools to expose interfaces though which migrations will take place.

Binary file not shown.

After

Width:  |  Height:  |  Size: 164 KiB

View file

@ -0,0 +1,30 @@
flowchart
subgraph configuration
application-config
end
subgraph resource[resource module]
resource-options
provider-options
end
subgraph application[application module]
application-options{{application-options}} --> application-config
application-config --> config-mapping
resource-options{{resource-options}} --> config-mapping
config-mapping(config-mapping)
end
subgraph fediversity[fediversity setup]
config-mapping -->|required resources| resource-mapping(resource-mapping)
provider-options{{provider-options}} --> provider-config --> |available resources| resource-mapping -->|configuration| deployment{deployment}
end
maintainer((maintainer)) -->|curates| resource
contributor((developer)) -->|curates| application
operator((operator)) -->|enters| configuration
hosting-provider((hosting provider)) -->|maintains| fediversity

Binary file not shown.

After

Width:  |  Height:  |  Size: 50 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 22 KiB

View file

@ -1,9 +1,9 @@
flowchart flowchart
user(user) --> |use| deployment user((user)) --> |use| deployment
configuration1 -->|deploy| deployed1 configuration1 -->|deploy| deployed1
devs(developers) --> |maintain| fediversity maintainers --> |maintain| fediversity
fediversity --> |update| provider1 fediversity --> |update| provider1
subgraph provider1["fediversity setup A"] subgraph provider1["fediversity setup A"]
@ -17,13 +17,13 @@ flowchart
provider-config --> |implement runtime interfaces| configurations1 provider-config --> |implement runtime interfaces| configurations1
subgraph host[runtime environment] subgraph host[runtime environment]
deployment[applications] deployment[applications]
state state{state}
end end
end end
deployment --> |store| state deployment --> |store| state
operator(operator) --> |change| configuration1 operator((operator)) --> |change| configuration1
subgraph provider2["fediversity setup B"] subgraph provider2["fediversity setup B"]
subgraph configurations2[configurations] subgraph configurations2[configurations]
@ -32,16 +32,16 @@ flowchart
end end
subgraph host2[runtime environment] subgraph host2[runtime environment]
deployment2[applications] deployment2[applications]
state2[state] state2{state}
end end
end end
operator --> |trigger| migration operator --> |trigger| migration(migration)
configurations1 & state --> migration configurations1 & state --> migration
migration --> configurations2 & state2 migration --> configurations2 & state2
provider(hosting provider) --> |maintain| provider1 provider((hosting provider)) --> |maintain| provider1
subgraph fediversity[fediversity source code] subgraph fediversity[fediversity source code]
applications[application modules] applications[application modules]
backends[runtime backends] backends[runtime backends]
config[runtime options] config{{runtime options}}
end end

Binary file not shown.

After

Width:  |  Height:  |  Size: 68 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 33 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 32 KiB

View file

@ -0,0 +1,24 @@
---
title: Data model of sample web application
---
erDiagram
operator {
string username
string password_hash
}
deployment {
json deployed_configuration
option[string] staged_configuration
option[string] version
}
backup["back-up"] {
string bucket
string endpoint
}
keypair {
string access_key
string secret_key
}
operator ||--o{ deployment : has
deployment ||--o{ backup : has
backup ||--|{ keypair : authorises

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 12 KiB

14
default.nix Normal file
View file

@ -0,0 +1,14 @@
{
pkgs ? import <nixpkgs> { },
}:
{
shell = pkgs.mkShellNoCC {
packages = with pkgs; [
pandoc
texliveMedium
librsvg
mermaid-cli
plantuml
];
};
}

View file

@ -0,0 +1,204 @@
<!--
> Get insight in the total global investments of the Fediverse (OID)
> To get an overview of the total investment of capital, human resources and other valuables, we will research the usage of these in the (visible) Fediverse globally. A report of this will be released.
> Release a report on the estimated collective current investment on fediverse technology:
> Look at qualifiers like:
>- Used system resources (CPU, storage, networking)
>- Amount of software developers actively working on Fediverse software products
>- Amount of system administration engineers actively working on Fediverse software products
>- Amount of moderation and governance people
-->
# An estimate of investments in the Fediverse
## Introduction
To provide an estimate of the investments made into the Fediverse, defined[^1] as a decentralized interoperable network of social media sites, apps, and services built on the ActivityPub protocol, is a daunting task, owing due to:
1. a lack of research (in our experience) on this topic thus far;
1. the vast number of services part of the Fediverse[^2];
1. the Fediverse's decentralised nature;
1. involvement of voluntary contributions;
1. the third-party nature of some of the integrations.
Nonetheless, we have taken upon ourselves the task to provide a first glimpse on this subject. Given the Fediverse's interoperability empowers users, a better overview on the resources in this technology helps clarify its momentum for key decision-makers interested in adopting, investing in and interfacing with this technology. We further hope our work will contribute and ignite more serious research into this topic.
In this document we describe our assessment of the investments made in the Fediverse. We start with describing our methodology for gathering data on the Fediverse and share information on the services, platforms and software applications we have deemed relevant to include.
## Methodology
In order to better scope our research, we will address the mentioned challenges by:
1. given the lack of research on this topic so far, gathering missing data by contacting representatives of the respective projects;
1. to account for the vast number of services part of the Fediverse, focusing on the major services[^3] part of the Fediverse, in this case the software with at least 10,000 active monthly users as per Fediverse Observer[^2], in total accounting for over 95% of the active users across Fediverse services, i.e.:
* Threads
* Mastodon
* Pixelfed
* NodeBB
* Lemmy
* Peertube
* Loops
* Wordpress
* WriteFreely
1. to account for the Fediverse's decentralised nature, extrapolating for each service to their overall number of (visible[^4]) instances from their flagship instances;
1. to account for the involvement of voluntary contributions, separately citing internal versus overall contributors active on the projects, and extrapolating from the former to estimate the latter;
1. to account for the third-party nature of some of the integrations, explicitly distinguishing these in our overview.
### NodeInfo
Fediversity instances tend to expose data using the NodeInfo protocol (<https://nodeinfo.diaspora.software/>). Data exposed this way is gathered in <https://fediverse.observer/>, which we use as our source for metrics:
* estimated number of installations
* estimated number of users
* estimated number of monthly active users
### Human resources
#### Moderators
<!-- #### Amount of moderation and governance people -->
On users with moderation privileges ActivityPub unfortunately exposes no structured data yet, although this is an outstanding feature request[^5].
In addition, for some single-tenant platforms such as WriteFreely, moderation does not apply. Platforms that support user sign-ups, such as Mastodon, may similarly disable user sign-up, in which case moderation similarly is not relevant.
<!-- TODO: distinguish active user data by whether instances enable user sign-up -->
#### Developers
<!-- #### Amount of software developers actively working on Fediverse software products -->
The amount of active developers we estimate as users:
* as identified by unique emails
* who are not bots, as measured by whether their name contains 'bot',
* who contributed at least **5** commits to the default Git branch at the main forge/fork
* over the past year.
We obtain this metric by running a Nushell query[^6] on the project's respective Git repositories.
Based on project estimations on internal engineers involved measured in FTE, we further estimate the total amount of development involved by extrapolating from these FTE figures to include external contributors based on their relative number of commits --- thereby for the purpose of this estimate presuming similar time and effort required for external versus internal commits.
#### System administrators
<!-- #### Amount of system administration engineers actively working on Fediverse software products -->
### Used system resources
<!-- >- Used system resources (CPU, storage, networking) -->
#### CPU
* minimum/average/peak percentage of (number of) CPU cores used?
<!-- (presumes similar performance across CPU models) -->
* CPU model used
#### storage
in e.g. GB currently used by the instance
#### networking
in e.g. min/max/average MiB/s in/out of the instance, along with physical caps potentially limiting these
<!-- #### RAM -->
### Capital
<!-- * income -->
<!-- note that while most of the Fediverse projects have had grants from NLNet, NLNet does not publish structured information on financial resources committed or reimbursed. -->
<!-- * estimated costs for the service -->
### Questionnaire
In order to gather statistics for applications' flagship instances to extrapolate from, we have set up a questionnaire based on relevant metrics, set up using a LimeSurvey instance hosted by ProcoliX.
### Questions
1. Amount of people actively working on the project in:
1. development of the software
1. governance of the software
1. governance and moderation of the official instance
1. system administration of the official instance
1. Resources used for the official instance in terms of:
1. CPU
* minimum/average/peak percentage of (number of) CPU cores used?
* CPU model used
1. storage: in e.g. GB currently used by the instance
1. networking: in e.g. min/max/average MiB/s in/out of the instance
1. RAM: in e.g. min/max/average GB used by the instance
1. related costs
1. Investments in the project (e.g. donations, self-funded, grants like NLNet, commercial plans):
1. committed so far
1. received so far
1. periodical commitments
## Accompanying introduction
> Hi there,
> On behalf of Fediversity, I'm gathering info on total global [investments in the global Fediverse ecosystem](https://git.fediversity.eu/Fediversity/meta/src/branch/main/investment_analysis), a report on which we will publish on our website <fediversity.eu>. As the structured information available on this topic for instances in the wild is fairly limited, for part of the questions on this I am hoping to make estimations by extrapolating from info on the flagship instances.
> We hope a better overview on the resources in the Fediverse may help clarify its momentum for key decision-makers interested in adopting, investing in and interfacing with this technology.
> Would you maybe have info for me for your application? Questions touch on:
> 1. people active on the project
> 1. resources used for the official instance(s)
> 1. financial investments in the project
> Thanks in advance for helping out - as well as for making the Fediverse great!
## A closer look at the Fediverse platforms
<!--
* Name of service/platform
* Description of service/platform
* capital
* income
* estimated costs for the service
* NodeInfo
* estimated number of installations
* estimated number of users (Monthly Active Users)
* human resources
* [ ] Amount of software developers actively working on Fediverse software products
* estimated number of developers working on the service
* [ ] Amount of system administration engineers actively working on Fediverse software products
* [ ] Amount of moderation and governance people
* estimated number of moderators
* Used system resources
* [ ] CPU
* [ ] storage
* [ ] networking
-->
| Project | Installations | monthly active users |
|-|-|-|
| mastodon | 8033 | 736878 |
| pixelfed | 501 | 126065 |
| nodebb | 43 | 52399 |
| lemmy | 407 | 41341 |
| peertube | 962 | 25804 |
| loops | 1 | 26548 |
| wordpress | 2863 | 11510 |
| writefreely | 492 | 11188 |
## Estimating investments in the Fediverse
We will add results here as we collect and process them.
## Time spent
TODO
## Notes
[^1]: Kissane, E., & Kazemi, D. (2024, September 4). Findings Report: Governance on Fediverse Microblogging Servers. https://fediverse-governance.github.io/#brief-glossary
[^2]: https://fediverse.observer/allsoftwares
[^3]: Note that this focus would exclude work on protocols such as W3C's work on the ActivityPub protocol, as well as on non-core repositories, and ancilliary software whose development is run by others than by the lead developers of the mentioned services, such as bridges, cross-posters, third-party clients, browser extensions, instance indices, etc.
[^4]: Visibility, for our purposes meaning exposing metadata using the NodeInfo protocol, for Mastodon, the Fediverse service with the highest amount of active users, notably excludes an instance with more active users than its biggest visible instance <https://mastodon.social/>, namely Donald Trump's Truth Social.
[^5]: This feature requested may be tracked at: https://github.com/swicg/activitypub-trust-and-safety/issues/25
[^6]: The used query here is: `git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} | where ($it.date > ((date now) - 365day)) | where not ($it.name has 'bot') | group-by name | transpose | upsert column1 {|c| $c.column1 | length} | sort-by column1 | rename name commits | where ($it.commits >= 5) | reverse | length`
## References
* https://en.wikipedia.org/wiki/Fediverse
* https://fedidb.org/
* https://mastodon-analytics.com/
* https://www.thinkimpact.com/mastodon-statistics/
* https://fediverse.observer/
* https://the-federation.info/
* https://fediverse.party
* https://socialhub.activitypub.rocks/t/fep-f1d5-nodeinfo-in-fediverse-software/1190
* https://codeberg.org/fediverse
* https://hachyderm.io/@esk/113793277371908181
## Changes
Older versions of this directory lived here: https://git.fediversity.eu/Fediversity/meta/src/commit/0006758ab7d12c2914f915813815bea89208fc6d/Fediverse-investments-an-estimation.md

View file

@ -0,0 +1,35 @@
# 2025-06-11 Fediversity developer sync
- @kiara has been updating the project proposal to reflect current ideas
- https://git.fediversity.eu/kiara/fedi-goals/src/branch/rewrite/
- need to make sure all stakeholders are on board with the updated vision
- new slated project manager would need to know what they're signing up for
- we reviewed and discussed the draft
- @niols got the full integration test for deploying via the panel to pass
- https://git.fediversity.eu/Fediversity/Fediversity/pulls/361
- we tinkered on reducing the number of hacks involved
- @fricklerhandwerk had some ideas for that, but the sketches need cleanup
- https://git.fediversity.eu/Fediversity/Fediversity/pulls/364
- @kiara and @fricklerhanderk continued editing the adjustments to the project proposal
- @kiara: whether/how to concentrate on external contributors (and if so, how to sensibly reflect that in a key result)
- @kiara: whether keeping the fediverse in even helps communicate our mission, considering fediversity could not fix the fediverse if we wanted to (given speed of adoption imo def isn't about self-hosting)
- @fricklerhandwerk incorporated discussion results into the writeup
- https://git.fediversity.eu/kiara/fedi-goals/pulls/2
- https://git.fediversity.eu/kiara/fedi-goals/pulls/3
## Draft success metrics for the adjusted objectives
- Implement a way to run online services emphasizing user autonomy and data portability
- Integration tests pass for
- Setting up a fediversity hosting environment from a declarative configuration
- Configuring, deploying, and migrating a set of dummy applications
- Code passes data protection audit
- Disseminate our results by engaging the open-source community to further expand on work in this direction
- Present results on at least 3 conferences
- At least 5 applications compatible with Fediversity thanks to external contributions by 2027-03
- Exploit our work by enabling reproducible deployments of an initial set of portable applications
- There are 3 fediverse applications available out of the box:
- Mastodon
- PeerTube
- Pixelfed

View file

@ -0,0 +1,12 @@
Attendees: @niols @kiara @fricklerhandwerk
- updated the project board
- made some clarifications around splitting https://git.fediversity.eu/Fediversity/Fediversity/pulls/389 into manageable chunks
- agreed on a general strategy to get a grip on the project layout:
- migrate everything to modules
- keep using flake-parts but don't use the flake parts except for CI checks
- we may simply swap it out with our custom modules eventually, since most of them will be custom anyway
- eventually we will be able to render most of contributor documentation via a Git hook
- a few small steps already pushed as PRs
- @niols and @kiara will focus on speeding up CI and making the runner reproducible
- @fricklerhandwerk will keep developing the data model with @kiara

View file

@ -0,0 +1,16 @@
Attendees: @kiara @fricklerhandwerk
- Discussed @fricklerhandwerk's WIP branch for implementing https://git.fediversity.eu/Fediversity/Fediversity/issues/103
- The resource policy idea seems sound
- The test sample doesn't look minimal at first glance, but in fact is
- An entire NixOS configuration is only minimal in terms of resource policy -- just put it in a VM
- An application that configures a whole NixOS needs to set many more things to make it run
- The login-shell sample also allows us to explicitly enumerate resource-specific configuration options to demonstrate how they're declared and used
- We re-iterated on the stance not to touch actual applications before we have the whole workflow tested with samples
- Our packaged applications are already tested and in a state where we could translate them to the proper data model mechanically once it's in place and well-understood, so this will be a well-scoped problem for next year
- It emerged again that we need to figure out a notion of compatibility (and therefore versioning) before being able to model migrations
- Once the data model draft type-checks and the tests pass, this will be the next research question
- We need to check if we want or have to make simplifying assumptions, such as not allowing version rollbacks
- Chances are, once we have compatibility clarified, migrations will be trivial
- @kiara will spend some time today debugging the draft
- Likely we'll need the next week to finish that part and play with it a bit, such as by adding a few more test cases.

View file

@ -0,0 +1,28 @@
Attendees: @fricklerhandwerk @niols @kiara
- @fricklerhandwerk
- almost completed the data model up to deployment, including a test
- @kiara fixed up some typos and evaluation errors
- remaining is to mold `nixops4Deployment` into a type that is accepted
- started compiling the mid-term technical report
- will pause work on the data model until that's done (next week)
- @niols worked on extracting a minimal flake for the test
- (discussed how to handle lazy/strict source closures)
- @kiara
- tried containerizing the CI but it didn't work out yet
- setting up a cache may improve turnaround time by a lot
- will pair with @niols the week after next
- tried patching the workflow to enable automatic dependency updates, made small progress
- we can do dependency bumps manually for now
- will pause work on CI until the technical report is done
- todo for end-to-end beta: (REST) API for interacting with a runtime environment (general idea already settled in earlier discussions)
- GET json schema for configuring applications
- GET configuration
- POST configuration
- POST deploy
- GET deployment (version, config, status, ...)
- GET (stream) deployment progress
- Webhook: deployment state updates, e.g. completed, failed, ...
- TODO: workflow for migration
- ...
-

View file

@ -0,0 +1,20 @@
Attendees: @kiara @fricklerhandwerk @niols
- CI still bad: checkout action flaky, secrets exposed to all contributors
- We discussed switching to Gerrit (for review suggestions) and buildbot-nix (for CI)
- It will be a lot of effort, but it's already a lot of effort and we better spend it now than when more people join the project
- Chances are this will solve the secrets issue automatically
- Regressions in Proxmox provisioning
- needs tests and docs, `proxmox-provision.sh` was written ad hoc beginning of the year
- we haven't been exercising our knowledge of provisioning since
- would be nice to rewrite in Python, also as a way to turn it into a NixOps4 resource provider
- NixOps4 future:
- @roberth signaled the path to nested deployments may not be all that long
- @fricklerhandwerk: currently our use case is perfectly suited by the state of affairs (especially having everything one language), but eventually we'll need something more powerful
- @niols: dry run threatens to be a can of worms with nested deployments
- @fricklerhandwerk: we'd need to mock away everything, we don't have a programming pattern for that yet (sigh)
- @niols: for now we expose only the NixOS configurations to get evaluated separately
- @kiara kept dabbling at the data model, fixing small issues
- @fricklerhandwerk will go through the WIP branch and cherry-pick what works
- @niols will add tests to provisioning code to fix regressions from recent refactorings
- will start refactoring `infra/flake-parts.nix` to remove the wild function passing

1
shell.nix Normal file
View file

@ -0,0 +1 @@
(import ./. { }).shell