Skip to content

Make the rules guard catch packages leaving an enforced rule and new ratcheted rules, and give each downgrade an open tracking issue #912

Closed

Description

@anth-volk

Problem

The rules guard compares each rule that exists on the base branch with the same rule on the head (check_baselines in scripts/rules/baseline.py). It fails when a baseline count grows, and when a rule moves from enforced or ratcheted to advisory or doc-only, or from enforced to ratcheted, without a [rule.downgrade] reason. Three ways of weakening a rule still pass:

  1. It doesn't compare a rule's enforcer.projects. Moving a package out of an enforced Pyright rule into a new ratcheted rule passes, because a new rule only produces the note "rule … is new".
  2. A new rule can start ratcheted, with a baseline that tolerates violations, and no [rule.downgrade].
  3. A downgrade's reason is free text. Nothing records the work that restores the rule, so a downgrade has no follow-up.

docs/engineering/skills/type-checking.md lists the first gap under "It doesn't check" (#877).

Proposal

  • A package that an enforced Pyright rule lists on the base branch, and that still exists, must be listed by an enforced Pyright rule on the head. Otherwise the rule that lists it now needs a [rule.downgrade] section. Moving a package between two enforced rules, and deleting the package, pass.
  • A rule that is new on the head, ratcheted, with a baseline count above 0, needs a [rule.downgrade] section.
  • [rule.downgrade] requires issue, the number of an open issue in this repository that tracks restoring the rule, next to reason. The guard always checks that it is a positive integer, and checks through the GitHub API that the issue is open. In CI, a missing token or a failed lookup fails the guard. Locally, without a token, the guard prints a note that it didn't check the issue.
  • The quality-guards job in .github/workflows/pr.yml gives the guard the job's token, with read access to issues.
  • docs/engineering/skills/type-checking.md and docs/engineering/skills/rules-registry.md describe what the guard now checks.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions