Skip to content

Fix byte-string Unicode semantics for regex word classes #1189

Description

@fglock

Summary

PerlOnJava regressed byte/UTF-8-sensitive regular-expression behavior exercised by Test::Regexp.

In the Test::Regexp UTF-8 matrix, cases involving the byte form of F\x{F8}o and \w+ report a successful match when the upstream test expects no match. This causes follow-on failures for match completeness and produces 36 semantic assertion failures in t/140_utf8.t.

CPAN impact

Test::Regexp 2017040101 regressed in CPAN tester run 20260828-172308-14550 after its prior recorded pass on cc69cbb4c (2026-08-12).

The run fails 55 of 2,253 subtests:

  • t/140_utf8.t: 46 failures, of which 36 are the byte/UTF-8 regex-semantic failures.
  • t/120_keep_pattern.t: 9 named-capture pattern-rendering failures. Those are tracked separately from the semantic problem here because capture results themselves are correct.

Oracle and scope

System Perl 5.42 passes the complete unchanged distribution suite: 23 files and 2,251 tests (with its ordinary author-test skips).

Both PerlOnJava JVM and interpreter backends reproduce t/140_utf8.t failures. The failures occur through Test::Regexp::match while it varies UTF-8 flag state with utf8::upgrade and utf8::downgrade; they are not a source-filter or native dependency problem.

The byte-mode capture corruption tracked in #1183 may be related at the representation boundary, but this case is a direct \w matching/flag-semantics regression and needs an independent minimal reproducer before sharing an implementation fix.

Acceptance criteria

  • Add a focused project-owned regression based on the smallest system-Perl-validated \w case that distinguishes upgraded and downgraded Latin-1 strings in the same way as Test::Regexp.
  • Match system Perl's result and match-completeness behavior on JVM and interpreter backends.
  • Preserve existing Unicode behavior for UTF-8-flagged strings.
  • Test::Regexp 2017040101 t/140_utf8.t passes on both backends.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions