Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 27 additions & 27 deletions scripts/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
# Nixpkgs Committer management script testing

The recommended way to test these scripts is to run them in a GitHub test organisation with the right setup.
Since creating your own takes some time, you can just ask @infinisil to get added to his test setup instead,
Since creating your own takes some time, you can just ask @philiptaron to get added to the nixos-test-org test setup instead,
whose identifiers will be used here.

## Setup

### One-time

- [infinisil-test-org](https://github.com/infinisil-test-org): A GitHub organisation you're part of
- [nixos-test-org](https://github.com/nixos-test-org): A GitHub organisation you're part of
- Repositories:
- [infinisil-test-org/empty](https://github.com/infinisil-test-org/empty): A repository with only one commit, not attributed to your GitHub user
- [infinisil-test-org/active](https://github.com/infinisil-test-org/active): One where you have some activity
- [infinisil-test-org/nixpkgs-committers](https://github.com/infinisil-test-org/nixpkgs-committers): A fork of the upstream repo
- [nixos-test-org/empty](https://github.com/nixos-test-org/empty): A repository with only one commit, not attributed to your GitHub user
- [nixos-test-org/active](https://github.com/nixos-test-org/active): One where you have some activity
- [nixos-test-org/nixpkgs-committers](https://github.com/nixos-test-org/nixpkgs-committers): A copy of the upstream repo with the `nomination` and `retirement` labels, and with GitHub Actions disabled so the scheduled workflows don't run

Useful to keep the first two separate for testing, because it's not possible to "undo" activity on a repo.
- [@infinisil-test-org/actors](https://github.com/orgs/infinisil-test-org/teams/actors): A team you're part of, needs write access to the `active` and `nixpkgs-committers` repository
- [@nixos-test-org/actors](https://github.com/orgs/nixos-test-org/teams/actors): A team you're part of, needs write access to the `active` and `nixpkgs-committers` repository

### Per-user

Once you have the above setup (or got @infinisil to add yourself to his), you have to prepare the following:
Once you have the above setup (or got added to nixos-test-org), you have to prepare the following:

- Add some activity of yours to the `active` repo.
To match Nixpkgs, it’s recommended to do this by setting the “Default commit message” for merge commits to “Pull request title”, then creating and merging a PR.
Expand All @@ -28,51 +28,51 @@ Once you have the above setup (or got @infinisil to add yourself to his), you ha
- A local Git clone of this repository with the `origin` remote set to the test repository:
```bash
git remote add upstream git@github.com:NixOS/nixpkgs-committers.git
git remote set-url origin git@github.com:infinisil-test-org/nixpkgs-committers.git
git remote set-url origin git@github.com:nixos-test-org/nixpkgs-committers.git
```

## Testing `sync.sh`

This script has no external effects and as such can be easily tested by running:

```bash
scripts/sync.sh infinisil-test-org actors members-test
scripts/sync.sh nixos-test-org actors members-test
```

Check that it synchronises the files in the `members-test` directory with the team members of the `actors` team.

## Testing `nomination.sh`

This script does not depend on the current repository, but has some external effects.
For testing, we'll use [PR #33](https://github.com/infinisil-test-org/nixpkgs-committers/pull/33) and [issue #30](https://github.com/infinisil-test-org/nixpkgs-committers/issues/30).
For testing, we'll use [PR #3](https://github.com/nixos-test-org/nixpkgs-committers/pull/3) and [issue #2](https://github.com/nixos-test-org/nixpkgs-committers/issues/2).

To test:
1. Delete all labels of the PR and reset the title:
```bash
gh api --method DELETE /repos/infinisil-test-org/nixpkgs-committers/issues/33/labels
gh api --method PATCH /repos/infinisil-test-org/nixpkgs-committers/pulls/33 -f title="A non-conforming title"
gh api --method DELETE /repos/nixos-test-org/nixpkgs-committers/issues/3/labels
gh api --method PATCH /repos/nixos-test-org/nixpkgs-committers/pulls/3 -f title="A non-conforming title"
```
1. Run the script while simulating that a non-nomination PR was opened:
```bash
scripts/nomination.sh members infinisil-test-org/nixpkgs-committers 33 30 <<< "removed members/infinisil"
scripts/nomination.sh members nixos-test-org/nixpkgs-committers 3 2 <<< "removed members/ghost"
```

Ensure that it exits with 0 and wouldn't run any effects.
1. Run the script while simulating that multiple users were nominated together:
```bash
scripts/nomination.sh members infinisil-test-org/nixpkgs-committers 33 30 <<< "removed members/foo"$'\n'"added members/bar"
scripts/nomination.sh members nixos-test-org/nixpkgs-committers 3 2 <<< "removed members/foo"$'\n'"added members/bar"
```

Ensure that it exits with non-0 and wouldn't run any effects.
1. Run the script simulating a successful nomination
```bash
scripts/nomination.sh members infinisil-test-org/nixpkgs-committers 33 30 <<< "added members/infinisil"
scripts/nomination.sh members nixos-test-org/nixpkgs-committers 3 2 <<< "added members/ghost"
```

Ensure that it exits with 0 and would run effects to label the PR, change the title and post a comment in the issue.
1. Rerun with effects
```bash
PROD=1 scripts/nomination.sh members infinisil-test-org/nixpkgs-committers 33 30 <<< "added members/infinisil"
PROD=1 scripts/nomination.sh members nixos-test-org/nixpkgs-committers 3 2 <<< "added members/ghost"
```

## Testing `retire.ts`
Expand Down Expand Up @@ -115,62 +115,62 @@ In CI it is set to `7 days ago` instead, which together with the `CACHE_FILE` ac

1. Run the script with the `active` repo argument to simulate CI running without inactive users:
```bash
scripts/retire-local.ts infinisil-test-org active nixpkgs-committers members-test 'yesterday 1 month ago' now now
scripts/retire-local.ts nixos-test-org active nixpkgs-committers members-test 'yesterday 1 month ago' now now
```

Check that no PR would be opened.
1. Run the previous command again with `CACHE_FILE` set:
```bash
CACHE_FILE=.tmp-cache.json scripts/retire-local.ts infinisil-test-org active nixpkgs-committers members-test 'yesterday 1 month ago' now now
CACHE_FILE=.tmp-cache.json scripts/retire-local.ts infinisil-test-org active nixpkgs-committers members-test 'yesterday 1 month ago' now now
CACHE_FILE=.tmp-cache.json scripts/retire-local.ts nixos-test-org active nixpkgs-committers members-test 'yesterday 1 month ago' now now
CACHE_FILE=.tmp-cache.json scripts/retire-local.ts nixos-test-org active nixpkgs-committers members-test 'yesterday 1 month ago' now now
rm .tmp-cache.json
```

Check that the first run writes your activity to the cache file and that the second run skips the `/commits` query for your user because of it.
1. Run the script with the `empty` repo argument to simulate CI running with inactive users:

```bash
scripts/retire-local.ts infinisil-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now now
scripts/retire-local.ts nixos-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now now
```

Check that it would only create a PR for your own user and not the "new-committer-1" or "new-committer-2" user.
Also check that with a `CONFIRM_CUTOFF` in the past (as used in CI), no PR would be opened yet:

```bash
scripts/retire-local.ts infinisil-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now '7 days ago'
scripts/retire-local.ts nixos-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now '7 days ago'
```

Then run it again with `PROD=1` to actually do it:

```bash
PROD=1 scripts/retire-local.ts infinisil-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now now
PROD=1 scripts/retire-local.ts nixos-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now now
```

Check that it created the PR appropriately, including assigning the "retirement" label.
You can undo this step by closing the PR.
1. Run it again to simulate CI running again later:
```bash
PROD=1 scripts/retire-local.ts infinisil-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now now
PROD=1 scripts/retire-local.ts nixos-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now now
```
Check that no other PR is opened.
1. Run it again with `now` as the notice cutoff date to simulate the time interval passing:
```bash
PROD=1 scripts/retire-local.ts infinisil-test-org empty nixpkgs-committers members-test now now now
PROD=1 scripts/retire-local.ts nixos-test-org empty nixpkgs-committers members-test now now now
```
Check that it undrafted the previous PR and posted an appropriate comment.
1. Run it again to simulate CI running again later:
```bash
PROD=1 scripts/retire-local.ts infinisil-test-org empty nixpkgs-committers members-test now now now
PROD=1 scripts/retire-local.ts nixos-test-org empty nixpkgs-committers members-test now now now
```
Check that no other PR is opened.
1. Reset by marking the PR as a draft again, then run it again with the `active` repo argument to simulate activity during the time interval:
```bash
PROD=1 scripts/retire-local.ts infinisil-test-org active nixpkgs-committers members-test now now now
PROD=1 scripts/retire-local.ts nixos-test-org active nixpkgs-committers members-test now now now
```
Check that it gets undrafted with a comment listing the new activity.
1. Close the PR, then run the script again with no activity and for an earlier close cutoff, simulating that the retirement was delayed:
```bash
PROD=1 scripts/retire-local.ts infinisil-test-org empty nixpkgs-committers members-test now '1 day ago' now
PROD=1 scripts/retire-local.ts nixos-test-org empty nixpkgs-committers members-test now '1 day ago' now
```

Check that no other PR is opened.
Loading