infrastructure automatically deployed using continuous deployment #177

Closed
opened 2025-02-24 09:03:28 +01:00 by kiara · 6 comments
Owner

As a developer,
I want for commits to main to be deployed automatically,
so that we can more easily demonstrate our progress to non-developers.

Acceptance criteria

Given a PR change to our infrastructure code,
When I merge the PR,
The infrastructure is realized as described by that code.

**As** a developer, **I want** for commits to main to be deployed automatically, **so that** we can more easily demonstrate our progress to non-developers. ## Acceptance criteria Given a PR change to our infrastructure code, When I merge the PR, The infrastructure is realized as described by that code.
kiara added this to the Fediversity project 2025-03-18 11:30:54 +01:00
kiara removed this from the Fediversity project 2025-04-14 11:13:11 +02:00
kiara added this to the Fediversity project 2025-04-18 10:04:04 +02:00
kiara changed title from continuous deployment to infrastructure automatically deployed using continuous deployment 2025-06-10 18:54:16 +02:00
kiara removed this from the Fediversity project 2025-06-10 19:06:53 +02:00
kiara closed this issue 2025-07-11 11:23:35 +02:00
Author
Owner

as per run https://git.fediversity.eu/Fediversity/Fediversity/actions/runs/1017#jobstep-3-102, this seems to get stuck now at:

nixops| building '/nix/store/9is7h345i7yxmp0n59gdrkpq0gzfb4gh-nixos-system-vm02187-25.05.19700101.dirty.drv'...

i presume this would be resolved with #309, tho the TF thread we had de-prioritized in favor of #103.

as per run https://git.fediversity.eu/Fediversity/Fediversity/actions/runs/1017#jobstep-3-102, this seems to get stuck now at: > nixops| building '/nix/store/9is7h345i7yxmp0n59gdrkpq0gzfb4gh-nixos-system-vm02187-25.05.19700101.dirty.drv'... i presume this would be resolved with #309, tho the TF thread we had de-prioritized in favor of #103.

But is it stuck consistently? And maybe it's some resource bottleneck like thrashing due to lack of swap? It doesn't look like a failure mode that has anything to do with nixops.

But is it stuck consistently? And maybe it's some resource bottleneck like thrashing due to lack of swap? It doesn't look like a failure mode that has anything to do with nixops.
Author
Owner

to be fair now it got stuck at some different part.
top and df queries also revealed no weird resource usage, nor did systemctl or journalctl -xe show anything off on the forgejo-ci node.

to be fair now it got stuck at some different part. `top` and `df` queries also revealed no weird resource usage, nor did `systemctl` or `journalctl -xe` show anything off on the `forgejo-ci` node.
Author
Owner

okay, there may be a few different things on our roadmap that could impact this:

  • #309
  • #366
  • #463
  • move of the node from office to the data center (offered by Richard)

my procolix colleague Jonathan further offered assistance in debugging this CI flaking. potentially straceing the process could help with this.

okay, there may be a few different things on our roadmap that could impact this: - #309 - #366 - #463 - move of the node from office to the data center (offered by Richard) my procolix colleague Jonathan further offered assistance in debugging this CI flaking. potentially `strace`ing the process could help with this.
kiara referenced this issue from a commit 2025-07-16 14:14:08 +02:00
Author
Owner

an htop attempt by Jonathan showed it as stuck on:

|-- ssh-session
  |-- nix-daemon --stdio

to debug further we used nixops4's verbose mode, which in turn revealed it as hanging on:

nixops| Couldn't execute bash -c "echo started": No such file or directory

that echo command appears part of at least nix copy, and seems to have hit various other users before.

some clues i'm gleaning from others affected:

  • have env var SHELL set to e.g. ${pkgs.bash}/bin/bash from the deploying node
  • something something Nix versions
an `htop` attempt by Jonathan showed it as stuck on: ``` |-- ssh-session |-- nix-daemon --stdio ``` to debug further we used `nixops4`'s verbose mode, which in turn revealed it as hanging on: ``` nixops| Couldn't execute bash -c "echo started": No such file or directory ``` that echo command appears [part of at least `nix copy`](https://github.com/NixOS/nix/blob/d7c340afbfda28806e9ac6b6264a2b598933c7c2/src/libstore/ssh.cc#L72), and seems [to](https://github.com/NixOS/nix/issues/8292) [have](https://github.com/NixOS/nix/issues/8750) [hit](https://github.com/NixOS/nix/issues/8329) [various](https://github.com/nix-community/nixos-anywhere/issues/147) [other](https://github.com/nix-community/nixos-anywhere/issues/251) [users](https://github.com/nix-community/nixos-anywhere/issues/280) [before](https://github.com/nix-community/nixos-anywhere/issues/437). some clues i'm gleaning from others affected: - have env var `SHELL` set to e.g. `${pkgs.bash}/bin/bash` from the deploying node - something something Nix versions
Author
Owner
CD went thru 🥳 https://git.fediversity.eu/Fediversity/Fediversity/actions/runs/1031
kiara closed this issue 2025-07-17 10:01:52 +02:00
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference: fediversity/fediversity#177
No description provided.