Skip to content

fix(catalog): fix PYTHON_FORMAT regex false positives for percentage text - #1325

Open
goodluck-ry wants to merge 1 commit into
python-babel:masterfrom
goodluck-ry:error_to_warning
Open

fix(catalog): fix PYTHON_FORMAT regex false positives for percentage text#1325
goodluck-ry wants to merge 1 commit into
python-babel:masterfrom
goodluck-ry:error_to_warning

Conversation

@goodluck-ry

@goodluck-ry goodluck-ry commented Aug 17, 2026

Copy link
Copy Markdown

Fixes #1268
Supersedes #1321

Hi @akx,

I've reverted the changes in frontend.py and refactored the PR to focus entirely on the regex rule in catalog.py.

The root cause of the false positive (such as '10% of' breaking builds) was that line 72 of catalog.py matched the space following '%' as a Python format flag in PYTHON_FORMAT. Removing the space flag from the regex prevents strings like '10% of' from being recognized as placeholders in the first place.

The original issue suggested downgrading errors from ERROR to WARNING. However, applying a downgrade in frontend.py would silently swallow real format mismatches. Fixing the regex at the source ensures that true syntax errors are still safely caught and exit with status 1, while the false positives are eliminated without touching error severity.

Trade-off note: Since explicit space-padded specifiers (like '% d') and text like '10% der' are textually identical at the static regex level, removing the space flag means '% d' won't be flagged. However, space-padded specifiers are extremely rare in translations compared to percentage text, making this a clean and pragmatically minimal fix.

I have updated the branch with corresponding tests for this case. Let me know what you think!

@Mukller Mukller left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling this — the \100 % done\ false positives are real and annoying.

I verified the change empirically against both regex versions:

input master this PR
\100 % done\ (prose) matched (FP) no match
\50 % off\ (prose) matched (FP) no match
\% d\ (valid placeholder) matched no match
\% 5d, \% f, \% e\ matched no match

The problem: the space character is a legitimate printf flag ('% d' % 3 == ' 3', Python docs), so dropping \\ \\ from the flags class turns current false positives into false negatives for translations that legitimately use spaced placeholders.

A middle ground that keeps both cases working: allow the space flag only when explicit width/precision digits follow (bare % \ is almost always prose):

\\python
PYTHON_FORMAT = re.compile(
r'''
\%
(?:\(([\w])\))?
(
(?:[-#0+]?(?:\
|[\d]+)?(?:\.(?:\*|\[\d]+))?)
|
(?:\ +[\d]+(?:\.[\d]+)?)
)
[hlL]?
([diouxXeEfFgGcrs%])
''',
re.VERBOSE,
)
\\

With this variant: % done\ / % off\ don't match, while % 5d, % f-style placeholders still do. Related open issue about PYTHON_FORMAT quirks: #865. Could you add test cases for both directions to the PR? Happy to re-review.

@Mukller

Mukller commented Aug 22, 2026

Copy link
Copy Markdown

(Reposting with fixed formatting — the review above got mangled by my tooling, apologies.)

Thanks for tackling this — the 100 % done false positives are real and annoying.

I verified the change empirically against both regex versions:

input master this PR
100 % done (prose) matched (FP) no match ✅
50 % off (prose) matched (FP) no match ✅
% d (valid placeholder) matched no match ❌
% 5d, % f, % e matched no match ❌

The problem: the space character is a legitimate printf flag ('% d' % 3 == ' 3', see Python docs), so dropping \ from the flags class turns current false positives into false negatives for translations that legitimately use spaced placeholders.

A middle ground that keeps both cases working — allow the space flag only when explicit width digits follow (a bare % <letter> is almost always prose):

PYTHON_FORMAT = re.compile(
    r'''
    \%
        (?:\(([\w]*)\))?
        (
            (?:[-#0+]?(?:\*|[\d]+)?(?:\.(?:\*|[\d]+))?)
            |
            (?:\ +[\d]+(?:\.[\d]+)?)
        )
        [hlL]?
        ([diouxXeEfFgGcrs%])
''',
    re.VERBOSE,
)

With this variant: % done / % off don't match, while % 5d-style placeholders still do.

Related open issue about PYTHON_FORMAT quirks: #865.

Could you add test cases covering both directions (prose false positives and spaced valid placeholders) to the PR? Happy to re-review.

@Mukller

Mukller commented Aug 22, 2026

Copy link
Copy Markdown

Self-correction before anyone wastes time on my earlier suggestion: I verified my proposed pattern empirically and it had a gap — the second branch required digits after the space flag, so % .2f (space + precision, no width) would not match, even though '% .2f' % 1.5 == ' 1.50' is valid Python.

Corrected alternative — space flag is allowed only when followed by width digits or a precision dot:

(
    (?:[-#0+]?(?:\*|[\d]+)?(?:\.(?:\*|[\d]+))?)   # current form, minus space flag
    |
    (?:\ +(?:\.[\d]+|[\d]+(?:\.[\d]+)?))          # space flag requires explicit width/precision
)

Verified against: 100 % done ✗ / 50 % off ✗ (prose stays out), % 5d ✓ / % .2f ✓ / % 5.2f ✓ / %s, %5d ✓.

Remaining known limitation, which I think is fundamental rather than fixable: bare % f / % e (space flag + conversion, no width/precision) stay unmatched — they're indistinguishable from prose like % done at the regex level. Master matches them today, so this trade-off should be an explicit maintainer decision; if losing bare % f is unacceptable, the PR's current approach (drop the space flag entirely) vs. this middle ground vs. status quo are the three real options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reduce compile translation checks from ERROR to WARNING

2 participants