meta/meeting-notes/2025-01-30nixos_deploy_session.md

Deployment workflow knowledge sharing (disguised as a usability test for our project documentation)

Attendees: Koen, Kiara, Valentin, Nicolas, Kevin Notes&moderation: Valentin

* Kevin drives, Nicolas navigates
* Task: deploy some program to an unsed VM
* Start at Fediversity/Fediversity: README.md
	* Need to add public key to ./keys/contributors/keys
		* TODO: no docs there why we need the keys and how to add them:
			* keys used to decrypt secrets stored in the repo
			* https://git.fediversity.eu/Fediversity/Fediversity/issues/84
		* TODO: document that review process needs rekeying https://git.fediversity.eu/Fediversity/Fediversity/issues/85
		* TODO: use keys for machine access, too https://git.fediversity.eu/Fediversity/Fediversity/issues/83
	* Inspecting ./services
		* TODO: Rewrite the introduction https://git.fediversity.eu/Fediversity/Fediversity/issues/86
		* TODO: update Pixelfed service to use an actual secret https://git.fediversity.eu/Fediversity/Fediversity/issues/87
	* Added an SSH public key and re-keyed
		* Tried redeploying the VM to provide the newly added contributor with machine access
		* TODO: check NixOps4 interfaces in a test https://git.fediversity.eu/Fediversity/Fediversity/issues/90
		* A machine was not available any more and NixOps4 broke for us after an update
			* Workaround: https://git.fediversity.eu/Fediversity/Fediversity/pulls/91
			* TODO: https://git.fediversity.eu/Fediversity/Fediversity/issues/90
	* Adding cowsay to a VM
		* Tested if it's already in the VM's environment: no
		* Added it to the VM's config
		* Ran into a code organisation issue that prevents the change as is
			* TODO: https://git.fediversity.eu/Fediversity/Fediversity/issues/93
		* Had some trouble getting `nixops4` to run
			* TODO: https://git.fediversity.eu/Fediversity/Fediversity/issues/94
			* For some reason NixOps4 is building Nix, takes a while
	* Ran cowsay after a successful deployment
		* Finished in 2:20h!
	* Debrief:
		* Koen: There were a lot of moving parts. Would like this to be more of a linear run-through guided by documentation.
			* Eventually I want to be able to do this myself, I'm exactly the target audience of this tooling
			* Target should be about 30 min to get from nothing to the deployment given solid Linux experience
		* Kiara: Learned the overall flow, should be able to get there a lot faster on Monday
			* I might be close to the target audience, but it's still very geared towards somewhat experienced Nix users
		* Kevin:
			* Most frustrating: Didn't know what I was doing, just following arbitrary-sounding instructions
			* Surprisingly easy: Applying the configuration to the deployment just worked (although getting there was hard)
			* What to change (assuming roadblocks removed): Nothing, seems about right. The problem were the roadblocks and lack of written sequential instructions
		* Nicolas:
			* Process felt painful; may be biased from having it working on my end.
			* Documentation is pretty much lacking, this is the biggest issue; far from being content with the current state
				* Part of it will be NixOps4 docs, part Nix docs
					* Leaking errors from any of those will confuse users not intimately familiar with both
				* Need to figure out how to factor those workflows in documentation; e.g. adding keys, factoring configurations are independent problems, can't put them in a meaningful sequence
		* Valentin:
			* Not surprised it took so long; the whole point was to reveal implicit assumptions
			* Since this project is largely a big integrator, we probably have to (co-)own the UX for each of the underlying tools our users interact with
				* Have to strike a balance with things we can fix upstream or where we have to paper over them with a custom wrapper/interface or documentation
			* We should probably map the entire user story starting with essentially two bare machines (client and deployment target), and then work through and iterate on it until it's smooth
				* Client: Freshly installed Debian machine with one user on it
				* Server: Empty machine
	* Decision: Do this once a week to derive tasks for the following days. Record the process and outcomes.
	* Next week: 
		* Start with two fresh machines: Client (Debian) and server (empty)
		* Install Nix and configure a user environment on the client
			* Pull the Fediversity repo and create a minimal NixOS ISO
		* Install NixOS with Proxmox-nix on the server:
			* Boot the installer ISO
			* Deploy to the machine via NixOps4 from a config in Fediversity repo