wip: data model requirements

architecture-docs/data-model-requirements.md

@@ -1,7 +1,13 @@
 # migration data model requirements
 
-To transfer between two providers, the target provider must be able to import the sending provider's versions. (e.g.: a deployment may have latest fediversity, latest pixelfed, but previous mastadon) Thus, for each "realease" of the data model, it needs to be versioned, and applications/APIs also are versioned.
+(updated) deployment incl variables, backup creation/restore
- * (May need a way to show on the front-end which versions are in place, and which migrations are supported. However, for application versions which are completely controlled by the installation and setup, this is "solved".)
+
+<!-- To transfer between two providers, the target provider must be able to import the sending provider's versions. (e.g.: a deployment may have latest fediversity, latest pixelfed, but previous mastodon) Thus, for each "release" of the data model, it needs to be versioned, and applications/APIs also are versioned. -->
+ <!-- * (May need a way to show on the front-end which versions are in place, and which migrations are supported. However, for application versions which are completely controlled by the installation and setup, this is "solved".) -->
+
+Assumptions:
+
+* Our deployment fully controls all versions, bypassing concerns on version mismatches.
 
 for release version 0, focus on known current needs
  * to be expanded later as each new application is added and can be transferred between providers
@@ -10,101 +16,49 @@ for release version 0, focus on known current needs
 Specifically, this suggests scoping to migrating:
 
 - managed infrastructure (rather than managed applications)
-- between servers owned by procolix
+- between servers initially owned by procolix
 - same proxmox version
 - NixOS VMs set up by us so we can guarantee identical application versions
 - hosting limited to a single application (to start)
+- retaining the same domain name
+- migrating the applications rather than also say control of domains
 
 First, a bit of an inventory (list without much structure now, later will create structured form/schema with e.g. many-to-many links, useful for the migration code):
- * clearly mark items that will not be in the first migration as eventually or speculative
+* clearly mark items that will not be in the first migration as planned for later or speculative
- * or reamove them if they would be too far in the future
+* or remove them if they would be too far in the future
- * later we understand what is useful for migration code, we can extract and transform in to a format suitable as data model documentation
+* later we understand what is useful for migration code, we can extract and transform in to a format suitable as data model documentation
 
 Hosting Provider provides:
+
 * proxmox, git
 * hardware
 * filesystem storage
-* DNS automation hooks?
+* DNS automation hooks (RFC 2136, optionally authenticated by TSIG (RFC 2845) or GSS-TSIG (RFC 3645))
 * central/shared garage storage or only hardware+diskspace for the garage VMs to create storage?
   * with central: more efficient but less isolated
 
 FooUniversity (Operator)
-* invoice info
-  * is all info expected to be transferred from provider A to provider B?
-    * May not want to transfer e.g. bank details, because they are already set up at B
-    * May also depend on regulation (which information are you allowed to hand out?)
-* Admins:
-  * credentials
-* persistent identifiers
-  * mappings between them (also need to travel across providers)
-    * e.g. if we can't change content URLs, we may need to create (and from then on carry around) a redirects mapping
-    * those mappings are likely application-specific, but they all belong to the same type class
 * domain(s)
-    * what is needed for DNS management?
+           <!-- * Can we make it a requirement that Garage is behind a predictable URL, eg. `<application>.garage.<customer domain>`? As opposed to something vendor-specific, eg. `pixelfed-university.garage.procolix.com/<customer domain>/<application>` -->
-    * users
-      * display name
-      * email(s)
-      * login id
-        * oauth2 (eventually)
-        * 2fa
-        * password
-        * passkeys (eventually)
-        * LDAP? (eventually?)
-    * all applications
-      * sub domain ( social.example.org vs example.org/social )
-      * info for proxmox setup such as to provision VMs (to reproduce proxmox )
-        * mem
-        * cpus
-        * storage mounts
-        * IPs likely not the same in the target network
-      * storage
-        * filesystem
-           * very well specified per application
-        * blob storage config (garage, s3-like)
-           * index
-           * Can we make it a requirement that Garage is behind a predictable URL, eg. `<application>.garage.<customer domain>`? As opposed to something vendor-specific, eg. `pixelfed-university.garage.procolix.com/<customer domain>/<application>`
            * may need to rewrite URLs to blobs automatically, depending on the underlying URL scheme, which may be per setup or application
         * limits? per application? per user? where are these used/set/enforced?
-        * TODO: what does e.g. borgmatic need to back up storage?
+        * TODO: what does e.g. borgmatic need to back up?
         * out of scope?: focus on actual state, disregarding reconstructable stuff
-      * SQL database
+<!-- * application specifics -->
-        * dump/snapshot
-        * TODO: what does e.g. borgmatic need to back up databases?
-* application specifics
-  * postfix? (is email in version 0?)
   * pixelfed
-    * where is blob storage
+    <!-- * where is blob storage -->
-      * in the specific case of Pixelfed, if blob storage changed URL, we might need to rewrite the pictures URLs in the database (try to avoid this)
+      <!-- * in the specific case of Pixelfed, if blob storage changed URL, we might need to rewrite the pictures URLs in the database (try to avoid this) -->
-    * redis (in the case of pixelfed, it is not just a cache)
-      * misc config: theme, name of instance, email of sysadmin
-    * database
-    * on-disk files
-      * Daniel Supernault is currently making it so evertying can be stored remotely in a garage or sql database
-    * users (login id) (in database? in redis?)
-      * user preferences/settings
-  * peertube
-  * mastodon
-  * matrix? (is it in version 0?)
-* logos
 
-Other considerations:
+ When transforming the data-model code to a deliverable version of the data model as part of the technical architecture document, documenting user-data storage and with respects to security and GDPR
 
-- Put a boundary for what is
+<!-- See also: -->
-  - operator-configurable
-  - needs to get fixed, but at the implementation level
-  - what can be configured dynamically per environment
-- Most importantly we need to preserve persistent identifiers
 
-- When transforming the data-model code to a deliverable version of the data model as part of the technical architecture document, documenting user-data storage and with respects fot security and GDPR
+<!-- - possible overlap/inspiration: Stalw.art [configuration docs](https://stalw.art/docs/server/general) -->
-
-See also:
-
-- possible overlap/inspiration: Stalw.art [configuration docs](https://stalw.art/docs/server/general)
 
 ## MVP scoping ideas
 
 User story 1: New customer
-When a new customer goes to the Fediversity website we want to show that user what Fediversity is all about and what it can give to the customer. This points the customer to a signup form where they can enter all the details that are needed to get it working. Here they can also decide what applications to use (at first no more than three). Details can be, the user/admin login, name, address, bank details, domain, other users, and applications. Than when the customer hits the install/provision/go button everything starts to install automagically. After which the customer is presented with (some) url's to login to.
+When a new customer goes to the Fediversity website we want to show that user what Fediversity is all about and what it can give to the customer. This points the customer to a signup form where they can enter all the details that are needed to get it working. Here they can also decide what applications to use (at first no more than three). Details can be, the admin login, domain, and applications. Then when the customer confirms everything starts to install automagically, after which the customer is presented with (some) url's to login to.
 
 User story 2: Take out / move to other instance
 At any time a customer may wish to change service providers. They can easily go to an admin screen where they can get their configuration and data packaged for transfer. This packaged data can be provided to a new service provider where they will be up-and-running again easily, with minimal downtime.
@@ -114,8 +68,8 @@ proposed MVP scope:
 - blob storage (garage)
 - physical servers
 - proxmox vm management
-- nixops service
+<!-- - nixops service -->
-- nixops scripts
+<!-- - nixops scripts -->
 - 1 to 3 applications packaged in Nix (Mastodon, Peertube, Pixelfed)
 - frontend / website
 - working dns, can be external, but automated

json-schema.yaml

@@ -0,0 +1,12 @@
+---
+type: object
+properties:
+  # version:
+  #   type: number
+  applications:
+    type: object
+    additionalProperties:
+      type: object
+      properties:
+        # version:
+        #   type: number