Fediversity/Fediversity
13
2
Fork 6
Code Issues 111 Pull requests 15 Projects 1 Activity Actions

Unflakify deployment tests #449

Merged
Niols merged 6 commits from Niols/Fediversity:unflakify-tests-3 into main 2025-07-09 15:07:04 +02:00
Conversation 0 Commits 6 Files changed 17 +338 -247
Niols commented 2025-07-08 15:58:11 +02:00
Owner
Copy link

This PR builds on top of #447 and #448. Since these might be rejected, there will be some changes needed for this PR as well. Let's see how the discussions go in #447.

In the meantime, @fricklerhandwerk, would you mind (in)validating the core idea of this PR? You only need to look at 7cf43c4041, really.

This PR builds on top of #447 and #448. Since these might be rejected, there will be some changes needed for this PR as well. Let's see how the discussions go in #447. In the meantime, @fricklerhandwerk, would you mind (in)validating the core idea of this PR? You only need to look at https://git.fediversity.eu/Niols/Fediversity/commit/7cf43c404178dbf3501b955349836fc8b5a4ec5c, really.
Niols added 8 commits 2025-07-08 15:58:13 +02:00
Get rid of flake dependency on flake-parts - grab it from npins
Some checks failed
/ check-data-model (pull_request) Successful in 31s
Details
/ check-peertube (pull_request) Successful in 21s
Details
/ check-panel (pull_request) Successful in 1m28s
Details
/ check-deployment-basic (pull_request) Successful in 11m37s
Details
/ check-deployment-cli (pull_request) Successful in 40m29s
Details
/ check-pre-commit (pull_request) Successful in 13s
Details
/ check-deployment-panel (pull_request) Has been cancelled
Details
e9f7409ed0
Also define systems in mkFlake.nix
All checks were successful
/ check-pre-commit (pull_request) Successful in 14s
Details
/ check-data-model (pull_request) Successful in 30s
Details
/ check-peertube (pull_request) Successful in 21s
Details
/ check-panel (pull_request) Successful in 1m31s
Details
/ check-deployment-basic (pull_request) Successful in 11m14s
Details
/ check-deployment-cli (pull_request) Successful in 39m1s
Details
/ check-deployment-panel (pull_request) Successful in 43m7s
Details
f7a55963ec
Grab git-hooks from npins
Some checks failed
/ check-pre-commit (pull_request) Successful in 17s
Details
/ check-data-model (pull_request) Successful in 37s
Details
/ check-peertube (pull_request) Successful in 25s
Details
/ check-panel (pull_request) Successful in 1m35s
Details
/ check-deployment-basic (pull_request) Failing after 7m10s
Details
/ check-deployment-cli (pull_request) Failing after 7m37s
Details
/ check-deployment-panel (pull_request) Failing after 8m23s
Details
a6b6d66292
Panel deployment test: same
Some checks failed
/ check-pre-commit (pull_request) Successful in 13s
Details
/ check-data-model (pull_request) Successful in 30s
Details
/ check-peertube (pull_request) Successful in 20s
Details
/ check-panel (pull_request) Successful in 1m30s
Details
/ check-deployment-cli (pull_request) Has been cancelled
Details
/ check-deployment-panel (pull_request) Has been cancelled
Details
/ check-deployment-basic (pull_request) Has been cancelled
Details
1190a54e0a
Niols added 1 commit 2025-07-08 16:06:35 +02:00
Plug everything directly from deployment/flake-part.nix
All checks were successful
/ check-pre-commit (pull_request) Successful in 14s
Details
/ check-data-model (pull_request) Successful in 30s
Details
/ check-peertube (pull_request) Successful in 20s
Details
/ check-panel (pull_request) Successful in 1m30s
Details
/ check-deployment-basic (pull_request) Successful in 11m5s
Details
/ check-deployment-cli (pull_request) Successful in 39m39s
Details
/ check-deployment-panel (pull_request) Successful in 44m28s
Details
aba96a84d5
kiara approved these changes 2025-07-08 17:59:46 +02:00
kiara left a comment
Owner
Copy link

for what it's worth, i don't currently have strong opinions on this, so i'll leave it to y'all

for what it's worth, i don't currently have strong opinions on this, so i'll leave it to y'all
fricklerhandwerk approved these changes 2025-07-09 08:34:52 +02:00
fricklerhandwerk left a comment
Owner
Copy link

LGTM conceptually! More comments on mkFlake in the respective PR.

LGTM conceptually! More comments on mkFlake in the respective PR.
fricklerhandwerk reviewed 2025-07-09 10:19:32 +02:00
deployment/check/common/nixosTest.nix Outdated
@ -136,2 +136,3 @@
## instance by allowing to grab things directly from the host's store.
with subtest("Override the lock"):
with subtest("Override the flake and its lock"):
deployer.succeed("cp ${config.pathFromRoot}/flake-under-test.nix flake.nix")
fricklerhandwerk commented 2025-07-09 10:19:32 +02:00
Owner
Copy link

It may be worth noting in a comment here that we're otherwise using the entire repository as is. Otherwise it's slightly too magical when you try to trace it through in your head.

It may be worth noting in a comment here that we're otherwise using the entire repository as is. Otherwise it's slightly too magical when you try to trace it through in your head.
Niols commented 2025-07-09 10:36:28 +02:00
Author
Owner
Copy link

Fair enough. I have added a comment explaining this. (Only locally on my machine for now, as I don't want to overwhelm our CI.)

Fair enough. I have added a comment explaining this. (Only locally on my machine for now, as I don't want to overwhelm our CI.)
😆 1
fricklerhandwerk commented 2025-07-09 11:00:16 +02:00
Owner
Copy link

CI-killing comments...

CI-killing comments...
Niols commented 2025-07-09 11:12:50 +02:00
Author
Owner
Copy link

Yeah, frustrating! It will come as soon as I rebase after the merge of #448.

Yeah, frustrating! It will come as soon as I rebase after the merge of #448.
fricklerhandwerk marked this conversation as resolved
fricklerhandwerk reviewed 2025-07-09 10:22:59 +02:00
deployment/check/common/nixosTest.nix Outdated
@ -138,4 +139,3 @@
deployer.succeed("""
nix flake lock --extra-experimental-features 'flakes nix-command' \
--offline -v \
--override-input flake-parts ${inputs.flake-parts} \
fricklerhandwerk commented 2025-07-09 10:22:59 +02:00
Owner
Copy link

Arguably we don't need flake inputs here at all as we can pass store paths via sources here, which means we don't need flakes at all at the project level (unless flake-parts wants a root flake to do its thing). Not saying we need to pull through with that now, just trying to separate accidental from essential complexity.

Arguably we don't need flake inputs here at all as we can pass store paths via `sources` here, which means we don't need flakes at all at the project level (unless flake-parts wants a root flake to do its thing). Not saying we need to pull through with that now, just trying to separate accidental from essential complexity.
Niols commented 2025-07-09 10:30:21 +02:00
Author
Owner
Copy link

Is this comment different from #449 (comment)?

Is this comment different from https://git.fediversity.eu/Fediversity/Fediversity/pulls/449#issuecomment-8697?
fricklerhandwerk commented 2025-07-09 11:00:42 +02:00
Owner
Copy link

I thought it was but seems like it's not.

I thought it was but seems like it's not.
fricklerhandwerk marked this conversation as resolved
fricklerhandwerk reviewed 2025-07-09 10:25:13 +02:00
deployment/check/common/nixosTest.nix Outdated
@ -135,3 +135,3 @@
## NOTE: This is super slow. It could probably be optimised in Nix, for
## instance by allowing to grab things directly from the host's store.
with subtest("Override the lock"):
with subtest("Override the flake and its lock"):
fricklerhandwerk commented 2025-07-09 10:25:13 +02:00
Owner
Copy link

Regarding the slowness -- do we even need the flake-under-test to have flake inputs? If we make the right store paths available in the test VM (e.g. by mounting the host store) we may as well get the sources in that flake-under-test from npins and save ourselves the lock override.

Regarding the slowness -- do we even *need* the flake-under-test to have flake inputs? If we make the right store paths available in the test VM (e.g. by mounting the host store) we may as well get the sources in that flake-under-test from npins and save ourselves the lock override.
Niols commented 2025-07-09 10:30:03 +02:00
Author
Owner
Copy link

Well in my head we were aiming to have our flake.nix without inputs, grabbing everything from npins. This would then also give this for free: our flake-under-test.nix files would indeed be simple, grab everything from npins as well, and we would not have this slowness anymore.

Of course, we can still have our flake-under-test.nix have no flake inputs and grab everything from npins, while flake.nix still has some regular flake inputs, but then we have the problem of aligning the npins and flake dependencies.

I think we shouldn't tackle this in this PR. We should have a subsequent PR showing a way to grab nixops4 and nixops4-nixos from npins, and using that to replace the flake inputs of the flake-under-test.nix files, but maybe also from the flake.nix file.

Well in my head we were aiming to have our `flake.nix` without inputs, grabbing everything from npins. This would then also give this for free: our `flake-under-test.nix` files would indeed be simple, grab everything from npins as well, and we would not have this slowness anymore. Of course, we can still have our `flake-under-test.nix` have no flake inputs and grab everything from npins, while `flake.nix` still has some regular flake inputs, but then we have the problem of aligning the npins and flake dependencies. I think we shouldn't tackle this in this PR. We should have a subsequent PR showing a way to grab `nixops4` and `nixops4-nixos` from npins, and using that to replace the flake inputs of the `flake-under-test.nix` files, but maybe also from the `flake.nix` file.
fricklerhandwerk commented 2025-07-09 10:35:27 +02:00
Owner
Copy link

Yes agreed. And I didn't imply we should do anything about that in this PR.

Yes agreed. And I didn't imply we should do anything about that in this PR.
👍 1
Niols marked this conversation as resolved
Niols force-pushed unflakify-tests-3 from aba96a84d5 to e0a4d652b4 2025-07-09 13:23:15 +02:00 Compare
Niols merged commit de38611572 into main 2025-07-09 15:07:04 +02:00
Niols deleted branch unflakify-tests-3 2025-07-09 15:07:04 +02:00
Niols referenced this pull request from a commit 2025-07-09 15:07:05 +02:00
Unflakify deployment tests (#449)
Sign in to join this conversation.
Reviewers
No reviewers
kiara
fricklerhandwerk
Labels
Clear labels   
api service

features that may relate more to a (to be created) API service

  
blocked

   
bug

Anything that doesn't work as advertised

  
component: fediversity panel

   
component: nixops4

   
documentation

   
estimation high: >3d

   
estimation low: <2h

   
estimation mid: <8h

   
productisation

features out-scoped with the decoupling of the productisation effort

  
project-management

Project management related tasks

  
question

   
role: sysadmin

   
security

   
technical debt

   
testing

   
type unclear

   
type: key result

   
type: objective

   
type: task

   
type: user story

   
user experience
No labels
api service
blocked
bug
component: fediversity panel
component: nixops4
documentation
estimation high: >3d
estimation low: <2h
estimation mid: <8h
productisation
project-management
question
role: sysadmin
security
technical debt
testing
type unclear
type: key result
type: objective
type: task
type: user story
user experience
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Fediversity/Fediversity#449
No description provided.
Delete branch "Niols/Fediversity:unflakify-tests-3"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?