390 lines
7.4 KiB
Markdown
390 lines
7.4 KiB
Markdown
---
|
|
marp: true
|
|
# theme: gaia
|
|
class: invert
|
|
headingDivider: 1
|
|
---
|
|
|
|
|
|
# NixOps4
|
|
|
|
|
|
|
|
|
|
<!--
|
|
_class: invert lead
|
|
-->
|
|
|
|
- Why
|
|
- What
|
|
- How
|
|
|
|
Robert Hensing
|
|
@roberth
|
|
|
|
fediversity.eu
|
|
|
|
# @roberth
|
|
|
|
|
|
Robert Hensing
|
|
|
|
- Hercules CI
|
|
- Nix Steering Committee
|
|
- Nix team
|
|
- Module System / flake-parts / modular services / single package fixpoint
|
|
- NixOps4
|
|
|
|
# Fediversity.eu
|
|
|
|
|
|
|
|
- NLNet
|
|
- NORDUnet
|
|
- Tweag
|
|
- Open Internet Discourse Foundation
|
|
|
|
# Fediversity.eu
|
|
|
|
Quotes, emphasis mine:
|
|
|
|
The Fediversity Project is a comprehensive effort to bring easy-to-use, ***hosted cloud services*** that have service portability and personal freedom at their core to everyone.
|
|
|
|
|
|
The Fediversity Project enables easy hosting for a wide variety of fediverse platforms, all based on ***NixOS***.
|
|
|
|
helps with decentralisation of the internet, a core principle of the NGI, by making it easier for people to participate in the Open Social Web ***on their own terms***.
|
|
|
|
# NixOps4
|
|
|
|
- Deployment tool
|
|
- WIP
|
|
- Successor to NixOps
|
|
|
|
# Why
|
|
|
|
<!--
|
|
What is the problem NixOps solves?
|
|
-->
|
|
|
|
2013 - 2020
|
|
|
|
- NixOps 1 is a tool to deploy NixOS systems
|
|
- Provisioning, secrets
|
|
- Also resources, e.g. AWS Route53
|
|
|
|
# Why
|
|
|
|
<!-- Also _has_ problems
|
|
|
|
Testing and CI are underdeveloped
|
|
|
|
Python is probably a fine language, but it did turn away contributors, and the way the code base was set up made it hard
|
|
|
|
Calling the evaluator twice is both too often and too little, leading to bad workarounds.
|
|
-->
|
|
- Testing and CI
|
|
- Python (*)
|
|
- Call Nix evaluator in batch, twice (bad)
|
|
|
|
|
|
# Why
|
|
|
|
NixOps 2
|
|
|
|
2020 - ...
|
|
|
|
- Plugins
|
|
- Polyrepo
|
|
|
|
# Why
|
|
|
|
NixOps 2
|
|
|
|
2020 - ...
|
|
|
|
|
|
|
|
<!--
|
|
|
|
I sincerely apologize to the authors and previous maintainers.
|
|
They did a good job with the architecture they had.
|
|
|
|
- Plugin system
|
|
- Ossified the architecture
|
|
-->
|
|
|
|
# Why
|
|
|
|
2021
|
|
|
|
@roberth
|
|
|
|
- Still the only tool that integrates provisioning
|
|
|
|
|
|
# Step back
|
|
|
|
Nix
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
┌────────────┐ ┌────────────────────┐
|
|
│ Nix │---- instantiate ---->│ Derivations │ ↺ store path
|
|
│ expression │ └────────────────────┘
|
|
│ language │ ⇑ builds
|
|
│ │ ┌────────────────────┐
|
|
│ │ │ Nix sandbox, store │
|
|
└────────────┘ └────────────────────┘
|
|
```
|
|
|
|
<!--
|
|
TODO: Explain firmly
|
|
-->
|
|
|
|
# Architecture
|
|
|
|
NixOps4
|
|
|
|
```
|
|
┌────────────┐ ┌────────────────────┐
|
|
│ Nix │---- configure ------>│ Resources │ ↺ nix value
|
|
│ expression │ └────────────────────┘
|
|
│ language │ ⇑ run output
|
|
│ │ ┌────────────────────┐
|
|
│ │---- instantiate ---->│ Derivations │ ↺ store path
|
|
│ │ └────────────────────┘
|
|
│ │ ⇑ builds
|
|
│ │ ┌────────────────────┐
|
|
│ │ │ Nix sandbox, store │
|
|
└────────────┘ └────────────────────┘
|
|
```
|
|
|
|
<!--
|
|
|
|
Adds new layer on top, analogous to Nix
|
|
|
|
Focus on `nix value` => precisely that; no tight coupling between NixOps and its resources
|
|
|
|
NixOps4 just manages the data flows generically
|
|
|
|
Another benefit
|
|
- resource can be implemented in any language, with any library
|
|
|
|
Not comparable to NixOps 2 architecture image. NixOps 2 is "just a script" that grew until it failed to scale and then ossified with plugins.
|
|
|
|
-->
|
|
|
|
|
|
# Resource
|
|
|
|
- Declares the existence of a real world object
|
|
- Operations
|
|
- Create
|
|
- Read
|
|
- Update
|
|
- Delete
|
|
|
|
# Deployment
|
|
|
|
Collection of resources
|
|
- wired together with Nix expressions
|
|
- reflecting some area of the real world
|
|
|
|
# Operations
|
|
|
|
- CRUD
|
|
|
|
- "`nix run`"
|
|
- backup
|
|
- key rotation
|
|
|
|
<!--
|
|
a. Arbitrary callable methods in resource provider
|
|
b. Scripts depending on resource outputs
|
|
-->
|
|
|
|
# Resource Provider
|
|
|
|
- Program built with Nix
|
|
- Called by NixOps
|
|
- JSON over stdio
|
|
|
|
|
|
# Expressions
|
|
|
|
Simplified
|
|
|
|
```nix
|
|
{ # flake.nix
|
|
outputs = inputs: {
|
|
nixops4Deployments.default = { resources, ... }: {
|
|
resources = {
|
|
<resource name> = {
|
|
...
|
|
};
|
|
};
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
<!-- These are very abstract. Clarify why. -->
|
|
|
|
<!-- Next: zoom in on the function -->
|
|
|
|
# Expressions
|
|
|
|
```nix
|
|
{ resources, ... }: {
|
|
resources = {
|
|
"nixos" = {
|
|
imports = [ inputs.nixos.modules.nixops4Resource.nixos ];
|
|
inputs = {
|
|
ssh.privateKey = resources.sshkeypair.privateKey;
|
|
ssh.host = resources.host;
|
|
module = ./configuration.nix;
|
|
};
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
# Expressions
|
|
|
|
```nix
|
|
{ resources, ... }: {
|
|
resources = {
|
|
"nixos" = ...;
|
|
"sshkeypair" = {
|
|
type = "ssh.keypair";
|
|
inputs = {
|
|
state = resources.state;
|
|
};
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
# Expressions
|
|
|
|
```nix
|
|
{ resources, ... }: {
|
|
resources = {
|
|
"nixos" = ...;
|
|
"sshkeypair" = ...;
|
|
"state" = {
|
|
type = "s3.object";
|
|
inputs = {
|
|
endpoint = "https://garage.example.com";
|
|
bucket = "nixops4-my-project";
|
|
};
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
# Expressions
|
|
|
|
```nix
|
|
{ config, resources, ... }: {
|
|
options.customers = mkOption {
|
|
type = attrsOf (submodule ./customer.nix);
|
|
};
|
|
config.resources = {
|
|
"state" = ...;
|
|
"sshkeypair" = ...;
|
|
"nixos" = ... (foo config.customers) ...;
|
|
};
|
|
}
|
|
```
|
|
|
|
# Expressions
|
|
|
|
```nix
|
|
{ resources, ... }: {
|
|
imports = [
|
|
./data-model.nix
|
|
./applications/pixelfed.nix
|
|
./applications/mastodon.nix
|
|
./applications/peertube.nix
|
|
];
|
|
}
|
|
```
|
|
|
|
# Expressions
|
|
|
|
- `resources` monoid in the category of endofunctors :wink:
|
|
- Structural composition like `attrsOf` or `submodule`
|
|
- `imports` is mix-ins
|
|
|
|
```nix
|
|
top@{ resources, ... }: {
|
|
resources = {
|
|
"state" = ...;
|
|
"my-host" = mkSequence ({ resources, ... }: {
|
|
"sshkeypair" = ... top.resources.state.handle ...;
|
|
"nixos" = ... resources.sshkeypair.privateKey ...;
|
|
});
|
|
};
|
|
}
|
|
```
|
|
|
|
# Module author benefits
|
|
|
|
- All-Nix development experience
|
|
- No glue code
|
|
- All declarative
|
|
|
|
# Application benefits
|
|
|
|
"NixPanel"
|
|
|
|
- Structured logging
|
|
|
|
- Separate evaluator for stability
|
|
|
|
# Operator benefits
|
|
|
|
CLI for the backend
|
|
|
|
Integrate arbitrary scripts, no glue code
|
|
|
|
# Operator benefits
|
|
|
|
# Progress
|
|
|
|
Built, but subject to fixes and improvements
|
|
- `nixops4`/`nixops4-eval` process architecture
|
|
- Nix C API, Rust bindings
|
|
- Documentation tooling
|
|
- Testing
|
|
|
|
# Progress
|
|
|
|
TBD
|
|
- `mkSequence` nesting / data dependencies
|
|
- Read, Update, Delete
|
|
- State management
|
|
- More resources, OpenTofu
|
|
|
|
# Questions
|
|
|
|
# Not discussed
|
|
|
|
- Resource naming within the state
|
|
- read multiple => migrations
|
|
|
|
- `resourceProviderSystem`
|
|
|
|
# Process Architecture
|
|
|
|
- `nixops4`
|
|
- `nixops4-eval` -> `libnixexpr` etc (internal)
|
|
- resource providers
|
|
- `nixops4-resources-local`
|
|
- `nixops4-resources-opentofu` (planned)
|
|
- ...
|