meta/meeting-notes/2024-07-16-kubernetes-discussion.md

67 lines
2 KiB
Markdown
Raw Normal View History

2024-07-16 15:30:20 +02:00
Problem
make available lots of open source packages. end-user doesn't have to know about how to do any of these things (install software, resilient storage, etc)
scale to 10k-1m users
we've made some choices: NixOS, s3 storage, stateless as much as possible
we need something that starts services, monitors the services, restarts, spins up extra nodes, etc.
constraint: everything is "real" open-source
"consider kubernetes?"
kubernetes
kubernetes fits
need some extra tools and glueing for making this available to the users
tooling is 90% there
tool recommendations to come
like docker-compose but in the cloud
reconcilliation loop: you specify a state "i want 2 instances of X at Y domain name" kubernetes takes care of everything
applications slot into the reconcilliation loop to enforce the part of the state they manage
most things already exist (DNS, networking, etc)
master/worker: master is the machine that holds state. this is usually managed by the cloud provider
there might be UIs similar to NixPanel already existing for kubernetes
k3s vs. k8s
k8s: kubernetes
k3s: kubernetes but slimed down for playing around at home
alternatives
nomad, but konstantinos does not have experience with it
complexity
it's easy to stay limited to a smaller subset
we don't want to introduce pre-existing components that bring along their own baggage (e.g. "matrix instance, but it's on alpine")
why not? unite things under one tool; can make small changes without breaking everything;
containerization
the pre-existing components are all running under docker containers
can override with a different container
things that might be easier with nix
configuration
rolling updates
integrate with the outside world
manage storage
network configurations
internally ipv4?
address space between nodes
public IPs
namespaces
Kubernetes has the advantage of being familiar to people
disadvantage is that it integrates into existing cloud providers (we want to move people away from google/amazon cloud)