nixops4-quick-intro/slides.md

---
marp: true
# theme: gaia
class: invert
headingDivider: 1
---


# NixOps4


![bg left:40% 80%](nixops4.svg)

<!--
_class: invert lead
-->

- Why
- What
- How

Robert Hensing
@roberth

fediversity.eu

# @roberth

![bg left:30% 40%](roberth.jpeg)
Robert Hensing

- Hercules CI
- Nix Steering Committee
- Nix team
- Module System / flake-parts / modular services / single package fixpoint
- NixOps4

# Fediversity.eu

![bg left:50% 70%](ngi_fedi_full.svg)

- NLNet
- NORDUnet
- Tweag
- Open Internet Discourse Foundation

# Fediversity.eu

Quotes, emphasis mine:

The Fediversity Project is a comprehensive effort to bring easy-to-use, ***hosted cloud services*** that have service portability and personal freedom at their core to everyone.


The Fediversity Project enables easy hosting for a wide variety of fediverse platforms, all based on ***NixOS***.

helps with decentralisation of the internet, a core principle of the NGI, by making it easier for people to participate in the Open Social Web ***on their own terms***.

# NixOps4

- Deployment tool
- WIP
- Successor to NixOps

# Why

<!-- 
What is the problem NixOps solves?
-->

2013 - 2020

- NixOps 1 is a tool to deploy NixOS systems
- Provisioning, secrets
- Also resources, e.g. AWS Route53

# Why

<!-- Also _has_ problems

Testing and CI are underdeveloped

Python is probably a fine language, but it did turn away contributors, and the way the code base was set up made it hard

Calling the evaluator twice is both too often and too little, leading to bad workarounds.
  -->
- Testing and CI
- Python (*)
- Call Nix evaluator in batch, twice (bad)


# Why

NixOps 2

2020 - ...

- Plugins
- Polyrepo

# Why

NixOps 2

2020 - ...

![bg right:66% height:80%](nixops2.png)

<!--

I sincerely apologize to the authors and previous maintainers.
They did a good job with the architecture they had.

- Plugin system
- Ossified the architecture
-->

# Why

2021

@roberth

- Still the only tool that integrates provisioning


# Step back

Nix

```




┌────────────┐                      ┌────────────────────┐
│ Nix        │---- instantiate ---->│ Derivations        │ ↺ store path
│ expression │                      └────────────────────┘
│ language   │                                ⇑ builds
│            │                      ┌────────────────────┐
│            │                      │ Nix sandbox, store │
└────────────┘                      └────────────────────┘
```

<!--
  TODO: Explain firmly
-->

# Architecture

NixOps4

```
┌────────────┐                      ┌────────────────────┐
│ Nix        │---- configure ------>│ Resources          │ ↺ nix value
│ expression │                      └────────────────────┘
│ language   │                                ⇑ run output
│            │                      ┌────────────────────┐
│            │---- instantiate ---->│ Derivations        │ ↺ store path
│            │                      └────────────────────┘
│            │                                ⇑ builds
│            │                      ┌────────────────────┐
│            │                      │ Nix sandbox, store │
└────────────┘                      └────────────────────┘
```

<!--

Adds new layer on top, analogous to Nix

Focus on `nix value` => precisely that; no tight coupling between NixOps and its resources

NixOps4 just manages the data flows generically

Another benefit
 - resource can be implemented in any language, with any library

Not comparable to NixOps 2 architecture image. NixOps 2 is "just a script" that grew until it failed to scale and then ossified with plugins.

-->


# Resource

- Declares the existence of a real world object
- Operations
  - Create
  - Read
  - Update
  - Delete

# Deployment

Collection of resources
- wired together with Nix expressions
- reflecting some area of the real world

# Operations

- CRUD

- "`nix run`"
  - backup
  - key rotation

<!--
  a. Arbitrary callable methods in resource provider
  b. Scripts depending on resource outputs 
-->

# Resource Provider

- Program built with Nix
- Called by NixOps
- JSON over stdio


# Expressions

Simplified

```nix
{ # flake.nix
  outputs = inputs: {
    nixops4Deployments.default = { resources, ... }: {
      resources = {
        <resource name> = {
          ...
        };
      };
    };
  };
}
```

<!-- These are very abstract. Clarify why. -->

<!-- Next: zoom in on the function -->

# Expressions

```nix
{ resources, ... }: {
  resources = {
    "nixos" = {
      imports = [ inputs.nixos.modules.nixops4Resource.nixos ];
      inputs = {
        ssh.privateKey = resources.sshkeypair.privateKey;
        ssh.host = resources.host;
        module = ./configuration.nix;
      };
    };
  };
}
```

# Expressions

```nix
{ resources, ... }: {
  resources = {
    "nixos" = ...;
    "sshkeypair" = {
      type = "ssh.keypair";
      inputs = {
        state = resources.state;
      };
    };
  };
}
```

# Expressions

```nix
{ resources, ... }: {
  resources = {
    "nixos" = ...;
    "sshkeypair" = ...;
    "state" = {
      type = "s3.object";
      inputs = {
        endpoint = "https://garage.example.com";
        bucket = "nixops4-my-project";
      };
    };
  };
}
```

# Expressions

```nix
{ config, resources, ... }: {
  options.customers = mkOption {
    type = attrsOf (submodule ./customer.nix);
  };
  config.resources = {
    "state" = ...;
    "sshkeypair" = ...;
    "nixos" = ... (foo config.customers) ...;
  };
}
```

# Expressions

```nix
{ resources, ... }: {
  imports = [
    ./data-model.nix
    ./applications/pixelfed.nix
    ./applications/mastodon.nix
    ./applications/peertube.nix
  ];
}
```

# Expressions

- `resources` monoid in the category of endofunctors :wink:
- Structural composition like `attrsOf` or `submodule`
  - `imports` is mix-ins

```nix
top@{ resources, ... }: {
  resources = {
    "state" = ...;
    "my-host" = mkSequence ({ resources, ... }: {
      "sshkeypair" = ... top.resources.state.handle ...;
      "nixos" = ... resources.sshkeypair.privateKey ...;
    });
  };
}
```

# Module author benefits

- All-Nix development experience
- No glue code
- All declarative

# Application benefits

"NixPanel"

- Structured logging

- Separate evaluator for stability

# Operator benefits

CLI for the backend

Integrate arbitrary scripts, no glue code

# Operator benefits

# Progress

Built, but subject to fixes and improvements
- `nixops4`/`nixops4-eval` process architecture
- Nix C API, Rust bindings
- Documentation tooling
- Testing

# Progress

TBD
- `mkSequence` nesting / data dependencies
- Read, Update, Delete
- State management
- More resources, OpenTofu

# Questions

# Not discussed

- Resource naming within the state
  - read multiple => migrations

- `resourceProviderSystem`

# Process Architecture

- `nixops4`
  - `nixops4-eval` -> `libnixexpr` etc (internal)
  - resource providers
    - `nixops4-resources-local`
    - `nixops4-resources-opentofu` (planned)
    - ...