generate kubernetes container from modular service #620

New issue

Closed

opened 2025-12-03 12:59:29 +01:00 by kiara · 0 comments

kiara commented

2025-12-03 12:59:29 +01:00

Owner

As a Fediversity maintainer,
given that applications use modular services (#623),
I want to generate Kubernetes container from such modular services,
so that we may shift to serving applications at the service level.

implementation notes

c.f. nix-to-kubernetes: kubernetes container from service argv
nimi can create a container from a modular service, but does not handle metadata for the likes of k8s (and in their defense, this has barely been exposed)
kubernetes manifests:
- types
  - kubenix: more limited?
  - nixifest: seems more generalized
- content
  - sample metadata implementation + k8s manifest generation
    - ports
    - user
    - directories
    - capabilities
    - environment
    - reload
skipping containers to directly use stuff in nix store
- nix-sidecar, not sure if open-sourced
- nix-snapshotter: older, targets containerd
- nix-storage-plugin: targets podman/CRIO/...
  - needs container-libs patch
- also see Morgan Helton's talks:
  - PlanetNix 2026: From Nix to Kubernetes: No Image Layover
  - Deterministically Built Containers or: How I Learned to Stop Worrying and Love Nix
kubernetes concerns:
- specifying (anti) affinity to ensure/prevent co-location
  - anti-affinity: redundancy (matching consuming regions?), particularly in back-ups
  - affinity
    - one-off: init containers
    - constantly running: sidecar containers
outstanding challenges to facilitate this from modular services

**As** a Fediversity maintainer, **given that** applications use modular services (#623), **I want** to generate Kubernetes container from such modular services, **so that** we may shift to serving applications at the service level. ### implementation notes - c.f. [`nix-to-kubernetes`](https://github.com/DeterminateSystems/nix-to-kubernetes/): kubernetes container from service `argv` - [`nimi`](https://discourse.nixos.org/t/portable-rust-based-service-runner-for-the-experimental-modular-services-spec/74338) can create a container from a modular service, but does not handle metadata for the likes of k8s (and in their defense, this has [barely been exposed](https://github.com/NixOS/nixpkgs/issues/490813)) - kubernetes manifests: - types - [kubenix](https://kubenix.org/): more limited? - [nixifest](https://bitbloxhub.github.io/nixifest/): seems more generalized - content - [sample metadata implementation + k8s manifest generation](https://github.com/KiaraGrouwstra/nixpkgs/pull/new/modular-metadata) - [ ] [ports](https://github.com/NixOS/nixpkgs/issues/490640) - [ ] user - [ ] directories - [ ] capabilities - [ ] [environment](https://github.com/NixOS/nixpkgs/pull/518860) - [ ] [reload](https://github.com/NixOS/nixpkgs/pull/518717) - skipping containers to directly use stuff in nix store - [`nix-sidecar`](https://www.socallinuxexpo.org/scale/22x/presentations/docker-was-too-slow-so-we-replaced-it-nix-production/), not sure if open-sourced - [`nix-snapshotter`](https://github.com/pdtpartners/nix-snapshotter): older, targets containerd - [`nix-storage-plugin`](https://github.com/bitbloxhub/nix-storage-plugin): targets podman/CRIO/... - [needs](https://discourse.nixos.org/t/nixifest-generate-kubernetes-manifests-from-nix/79687/11) [`container-libs patch`](https://github.com/podman-container-tools/container-libs/compare/main...bitbloxhub:container-libs:als-first) - also see [Morgan Helton's talks](https://www.socallinuxexpo.org/scale/speaker/morgan-helton): - [PlanetNix 2026: From Nix to Kubernetes: No Image Layover](https://www.youtube.com/watch?v=_Y40JOcWgOw) - [Deterministically Built Containers or: How I Learned to Stop Worrying and Love Nix](https://www.youtube.com/watch?v=QkI6OhJSoI8) - kubernetes concerns: - specifying (anti) affinity to ensure/prevent co-location - anti-affinity: redundancy (matching consuming regions?), particularly in back-ups - affinity - one-off: [init](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) containers - constantly running: [sidecar](https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/) containers - [outstanding challenges to facilitate this from modular services](https://github.com/NixOS/nixpkgs/issues/428084#issuecomment-3904556062)

Rows
Columns