Skip to content

re.IGNORECASE | re.LOCALE: a negated character set matches the other case #156444

Description

@serhiy-storchaka

Under re.IGNORECASE | re.LOCALE, [bc] matches b'B', but so does [^bc].

import re

W = re.IGNORECASE | re.LOCALE
print(re.fullmatch(rb'[bc]', b'B', W))
print(re.fullmatch(rb'[^bc]', b'B', W))
<re.Match object; span=(0, 1), match=b'B'>
<re.Match object; span=(0, 1), match=b'B'>

Expected: the second line is None, which is what 3.6 and earlier print.

charset_loc_ignore() in Modules/_sre/sre_lib.h tests the whole set once per locale case and returns true if either test matches. For a set carrying NEGATE the negation has to be applied outside that disjunction: b'B' matches because 'B' itself is not in {b, c}, though its lowercase is.

A one-member set escapes this, since it compiles to NOT_LITERAL_LOC_IGNORE and compares against both cases correctly, so [^b] is fine and two members are needed to see it.

IN_LOC_IGNORE and charset_loc_ignore() were added in 3.7 by bpo-30215 (898ff03), which moved locale case folding from compile time to match time. 3.2–3.6 are correct, 3.7 through main are affected.

The set-difference fusion new in 3.16 hits the same limitation from another direction: it rewrites A(?<![B]) into a single NEGATE-bearing set, which then compiles to IN_LOC_IGNORE. See gh-155985 and GH-155993, which avoids the fusion under these flags.

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13bugs and security fixes3.14bugs and security fixes3.15pre-release feature fixes, bugs and security fixes3.16new features, bugs and security fixesstdlibStandard Library Python modules in the Lib/ directorytopic-regextype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions