Skip to content

#65845 Increase the specificity of workflow path filtering - #9457

Open
johnbillion wants to merge 10 commits into
WordPress:trunkfrom
johnbillion:63170-paths
Open

#65845 Increase the specificity of workflow path filtering#9457
johnbillion wants to merge 10 commits into
WordPress:trunkfrom
johnbillion:63170-paths

Conversation

@johnbillion

@johnbillion johnbillion commented Aug 12, 2025

Copy link
Copy Markdown
Member

Note from @lancewillett September 4, 2026: I’m taking over this PR from @johnbillion to finish validation and shepherd it through commit.


Purpose

Reduce unnecessary pull-request workflow runs without silently losing coverage.

Today, a PHPUnit-only change can also trigger end-to-end, performance, and build-process testing. Narrow include lists save runs, but new source paths can remain untested until someone updates each list.

Approach

This PR updates nine workflow filters:

  • Seven broad filters now begin with all paths and exclude known irrelevant paths. New source file types therefore run by default.
  • Relevant tests, tools, configuration, and workflow files are re-included after broader exclusions.
  • Local Docker testing now includes wp-cli.yml.
  • Upgrade testing now includes PHP files in the repository root.

This also closes an existing Sass gap. A pull request changing only src/wp-admin/css/colors/*/colors.scss now triggers build-process testing.

Review focus

  • Confirm each excluded path cannot affect the corresponding workflow.
  • Confirm positive workflow and configuration patterns follow any broader negative pattern.
  • Confirm each workflow still covers its source, tests, tools, and configuration dependencies.

Validation

Historical replay and targeted checks

For each of 1,789 first-parent trunk commits from July 25th, 2025 through July 25th, 2026, I evaluated its changed paths using GitHub’s documented ordered path-filter behavior.

Workflow Includes saved Exclusions saved Extra runs
Coding standards 15 15 0
End-to-end tests 156 147 9
JavaScript tests 6 3 3
Performance tests 155 146 9
PHP compatibility 159 159 0
PHPUnit tests 7 0 7
Build-process tests 168 162 6
Total PR 665 631 34

No historical commit or currently tracked path covered by the include version was missed. The 34 additional runs cover Sass, certificates, fonts, and environment files—the intended safer failure mode.

Targeted fixtures also confirmed:

  • Sass triggers the build workflow.
  • New source file types run by default.
  • Documentation and unrelated workflows remain excluded.
  • Relevant workflow files are re-included after .github/**.
  • All changed workflow files pass actionlint.

Definition of done

  • Use exclusion-first filters so new source paths run by default.
  • Preserve all coverage provided by the previous include lists.
  • Ensure Sass-only changes trigger the build test.
  • Re-include each affected workflow after excluding .github/**.
  • Replay both approaches against 1,789 trunk commits.
  • Retain most savings: 631 avoided runs versus 665 with includes.
  • Pass actionlint and targeted path-filter tests.
  • Pass GitHub Actions.
  • Receive final reviewer approval.
  • Commit to WordPress trunk under Trac #65845.

Use of AI tools

Claude Code (Opus 5) and Codex (GPT-5) helped verify build inputs, replay historical filters, and validate the changes.

@johnbillion
johnbillion marked this pull request as ready for review August 12, 2025 22:57
@github-actions

github-actions Bot commented Aug 12, 2025

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props johnbillion, lancewillett, peterwilsoncc, desrosj, adrianmoldovanwp.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@github-actions

This comment was marked as off-topic.

peterwilsoncc

This comment was marked as resolved.

Comment thread .github/workflows/coding-standards.yml Outdated
Comment on lines +23 to +24
- 'src/**.php'
- 'tests/**.php'

This comment was marked as resolved.

Comment thread .github/workflows/end-to-end-tests.yml Outdated
Comment thread .github/workflows/coding-standards.yml Outdated
Comment on lines +23 to +24
- 'src/**.php'
- 'tests/**.php'

This comment was marked as outdated.

Comment thread .github/workflows/performance.yml Outdated
Comment thread .github/workflows/phpunit-tests.yml Outdated
Comment thread .github/workflows/phpunit-tests.yml Outdated
- 'src/**.css'
- 'src/**.js'
- 'src/**.json'
- 'src/**.php'

This comment was marked as outdated.

@johnbillion johnbillion changed the title #63170 Increase the specificity of workflow path filtering #64227 Increase the specificity of workflow path filtering Dec 16, 2025
@johnbillion
johnbillion requested a review from lancewillett July 25, 2026 20:15
@lancewillett

This comment was marked as outdated.

@lancewillett lancewillett changed the title #64227 Increase the specificity of workflow path filtering #65845 Increase the specificity of workflow path filtering Sep 2, 2026
@lancewillett

Copy link
Copy Markdown
Member

I'm working on reviving this one for 7.2 release.

@lancewillett

Copy link
Copy Markdown
Member

@adimoldovan Could you please take a look at the approach?

@lancewillett

Copy link
Copy Markdown
Member

This PR is ready for re-reviews.

@lancewillett
lancewillett requested a review from desrosj September 2, 2026 21:13
@lancewillett lancewillett self-assigned this Sep 2, 2026

@adimoldovan adimoldovan left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed against 6f35729. I left an inline suggestion based on the current approach.

But I suggest changing the approach to use exclude paths instead of include.
The two directions do not fail the same way: a list of includes that misses a path skips the check and we miss coverage. In time this list will become stale and someone needs to know to add new code paths in this list.

GitHub reads paths in order, and a positive pattern after a negative one puts the path back, so this fits in a single list. For coding-standards.yml:

    paths:
      - '**'
      # Documentation and repository metadata.
      - '!**.md'
      - '!.editorconfig'
      - '!.gitignore'
      - '!.git-blame-ignore-revs'
      - '!.mailmap'
      # Local environment and editor configuration.
      - '!.devcontainer/**'
      - '!docker-compose.yml'
      - '!wp-cli.yml'
      - '!.env.example'
      - '!jsdoc.conf.json'
      - '!tsconfig.json'
      # Other workflows.
      - '!.github/**'
      - '.github/workflows/coding-standards.yml'
      - '.github/workflows/reusable-coding-standards-*.yml'

Comment thread .github/workflows/test-build-processes.yml Outdated
@lancewillett

Copy link
Copy Markdown
Member

Thanks @adimoldovan—the Sass gap is real. I verified that sass is in the build chain, sass:colors compiles src/wp-admin/css/colors/*/colors.scss, 157 .scss files are tracked, and reusable-test-core-build-process.yml runs npm run build. A Sass-only pull request therefore skips the build test today, so broken Sass could land. This is an existing gap, not one introduced by this PR.

I also agree with the failure-mode argument for exclusions. A missing include silently loses coverage. A missing exclude causes one unnecessary run, which is visible.

Before changing the PR’s shape, I want to replay the proposed exclusion lists against the same 12 months of trunk history—1,800 commits—used to measure the current version. The include lists avoided roughly 790 runs per year. If exclusions save about the same amount, they win on safety without a meaningful efficiency trade-off.

GitHub’s documentation confirms that ordered negative and positive patterns work for both push and pull_request. It defines ** as matching all file paths, but does not separately document dot-prefixed paths. That ambiguity should not block the approach: we can keep the explicit !.github/** exclusion, re-include the relevant workflow files afterward, and cover those cases in the replay and targeted validation.

Transparency in AI usage: I used Claude Code Opus 5 to verify the Sass claim against the Gruntfile and workflow files.

@lancewillett

lancewillett commented Sep 4, 2026

Copy link
Copy Markdown
Member

@adimoldovan I switched the seven broad filters to exclusion-first in 70b7fc1.

I replayed both versions across the same 1,789 trunk commits from July 25th, 2025 through July 25th, 2026:

  • Current include version: 665 avoided workflow runs.
  • Exclusion version: 631 avoided workflow runs.
  • Difference: 34 additional runs for the exclusion version—fewer than three per month.
  • Coverage: no commit or currently tracked path covered by the include version was missed by the exclusion version.

The earlier estimate of roughly 790 avoided runs came from an earlier PR revision. The coverage fixes added since then reduced the current comparison baseline to 665.

The 34 extra runs are the intended safety margin. They cover changes such as Sass, certificates, fonts, and environment files. A Sass-only change now triggers the build test, and a new source file type runs the seven converted workflows by default.

I also verified the workflow files with actionlint and targeted path fixtures, including dot-prefixed workflow paths and positive patterns after exclusions.


AI use: gpt-5

@lancewillett

Copy link
Copy Markdown
Member

I updated the original PR body to make this change easier to understand and review.

@adimoldovan I'd love another review please

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

5 participants