gh-152204: Validate date fields in pure-Python date.fromisoformat by tonghuaroot · Pull Request #152205 · python/cpython

tonghuaroot · 2026-06-25T14:03:46Z

_pydatetime._parse_isoformat_date reads each fixed-width field with int() on a slice, without checking that the slice is exactly N ASCII digits. int() accepts a leading +/-/whitespace and a short string, so several malformed ISO 8601 basic-format dates are silently parsed into a wrong-but-plausible date instead of raising ValueError:

>>> import _pydatetime
>>> _pydatetime.date.fromisoformat('2020+12')
datetime.date(2020, 1, 2)
>>> _pydatetime.date.fromisoformat('+020-06-15')
datetime.date(20, 6, 15)
>>> _pydatetime.date.fromisoformat('2020061')   # 7 chars: day slice reads '1'
datetime.date(2020, 6, 1)
>>> _pydatetime.date.fromisoformat('2020-W2')   # 1-digit week number
datetime.date(2020, 1, 6)

The C accelerator rejects all of these via parse_digits() (which requires the exact field width and digit-only content), so this is a C-vs-pure-Python divergence. The pure-Python path is used when the _datetime C extension is unavailable, and directly via _pydatetime.

This validates each field slice (year / month / day / weekno / weekday) to be exactly N ASCII digits before converting, mirroring the C parse_digits(), and extends datetimetester's test_fromisoformat_fails with the affected inputs (the new cases now reject on both implementations).

Fixes #152204.

Prepared with AI assistance (Claude Code) and verified by hand against a debug build, against both the C and pure-Python implementations.

Issue: Pure-Python date.fromisoformat silently mis-parses malformed basic-format dates #152204

The pure-Python _parse_isoformat_date read each fixed-width field with int() on a slice, which silently accepts a leading sign or whitespace, or a short slice that runs off the end of the string. Malformed basic-format inputs such as '2020+12' or '2020061' were therefore parsed into a wrong-but-plausible date instead of raising, while the C accelerator rejects them via parse_digits(). Validate that each field slice is exactly N ASCII digits before converting.

serhiy-storchaka

This change has significant impact on performance.

Would not it be simpler to use .isascii() at the beginning and .isdigit() for each fragment?

tonghuaroot · 2026-08-19T02:53:41Z

Done in 46f6881: one dtstr.isascii() up front (also covers the datetime path, which lacked it) + s.isdigit() per fragment — ~10x faster than the per-character check, and the same rejection.

StanFromIreland · 2026-08-19T11:39:04Z

+            '2020061',          # 7 chars: day slice reads a 1-character tail
+            '2020123',          # 7 chars: day slice reads a 1-character tail
+            '9999121',          # 7 chars: day slice reads a 1-character tail


All of these cases are identical?

They duplicates can be removed.

StanFromIreland · 2026-08-19T11:40:23Z

-    year = int(dtstr[0:4])
+    if not dtstr.isascii():
+        raise ValueError(f"Invalid isoformat string: {dtstr!r}")
+    def _read(s, n):


Let's move this out of the hotpath to reduce overhead.

StanFromIreland · 2026-08-19T11:41:18Z

+    if not dtstr.isascii():
+        raise ValueError(f"Invalid isoformat string: {dtstr!r}")
+    def _read(s, n):
+        # dtstr is ASCII, so isdigit() matches only ASCII digits.


We already have a helper for this:

cpython/Lib/_pydatetime.py

Lines 298 to 299 in 20e6c2f

def _is_ascii_digit(c):

return c in "0123456789"

It was in the the initial variant. It is slower.

StanFromIreland · 2026-08-19T11:41:59Z

        raise ValueError("Invalid isoformat string")
-    year = int(dtstr[0:4])
+    if not dtstr.isascii():
+        raise ValueError(f"Invalid isoformat string: {dtstr!r}")


Keep it the same as the other message, and these won't surface anyway I think.

StanFromIreland · 2026-08-19T11:44:17Z

            return 8


 def _parse_isoformat_date(dtstr):


What about the time?

>>> import _pydatetime >>> _pydatetime.datetime.fromisoformat('20201212T0102٣٤') datetime.datetime(2020, 12, 12, 1, 2, 34)

StanFromIreland · 2026-08-19T11:44:51Z

+            '2020123',          # 7 chars: day slice reads a 1-character tail
+            '9999121',          # 7 chars: day slice reads a 1-character tail
+            '2020-W2',          # 1-digit week number
            '٢025-03-09'        # Unicode characters


Please add the missing comma.

serhiy-storchaka

LGTM. 👍

serhiy-storchaka · 2026-08-19T14:29:39Z

+    if not dtstr.isascii():
+        raise ValueError(f"Invalid isoformat string: {dtstr!r}")
+    def _read(s, n):
+        # dtstr is ASCII, so isdigit() matches only ASCII digits.


It was in the the initial variant. It is slower.

tonghuaroot · 2026-08-19T15:10:38Z

Done in a025467: moved the reader to a module-level _read_isoformat_component, added the same isascii()+isdigit() guard to the time path (fixes the T0102٣٤ case), and fixed the missing comma. Kept isdigit() per Serhiy.

StanFromIreland · 2026-08-19T18:56:11Z

There are conflicts now.

tonghuaroot · 2026-08-20T00:56:28Z

Done in aed8e2d: merged main (conflicts resolved) and removed the duplicate 7-char cases.

tonghuaroot requested review from StanFromIreland and pganssle as code owners June 25, 2026 14:03

bedevere-app Bot mentioned this pull request Jun 25, 2026

Pure-Python date.fromisoformat silently mis-parses malformed basic-format dates #152204

Open

bedevere-app Bot added the awaiting review label Jun 25, 2026

Tighten the _read() comment

e780e2c

tonghuaroot mentioned this pull request Jun 25, 2026

datetime.fromisoformat() (C) drops the sub-second part of a UTC offset, breaking round-trips #152079

Closed

serhiy-storchaka reviewed Aug 17, 2026

View reviewed changes

Validate with isascii() + isdigit() instead of a per-character check

46f6881

StanFromIreland reviewed Aug 19, 2026

View reviewed changes

serhiy-storchaka approved these changes Aug 19, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting review labels Aug 19, 2026

Validate the time fields too; move the digit reader to module level

a025467

Merge branch 'main' and remove duplicate test cases

aed8e2d

Restore the NEWS entry (dropped by the merge) and cover the time fields

9476ebc

Uh oh!

Uh oh!

Conversation

tonghuaroot commented Jun 25, 2026 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

tonghuaroot commented Aug 19, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tonghuaroot commented Aug 19, 2026

Uh oh!

StanFromIreland commented Aug 19, 2026

Uh oh!

tonghuaroot commented Aug 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tonghuaroot commented Jun 25, 2026 •

edited by bedevere-app Bot

Loading