Skip to content

gh-83895: Accept input larger than 2 GiB in the C implementation of ElementTree - #156746

Merged
serhiy-storchaka merged 2 commits into
python:mainfrom
serhiy-storchaka:gh-83895-large-input
Sep 1, 2026
Merged

gh-83895: Accept input larger than 2 GiB in the C implementation of ElementTree#156746
serhiy-storchaka merged 2 commits into
python:mainfrom
serhiy-storchaka:gh-83895-large-input

Conversation

@serhiy-storchaka

@serhiy-storchaka serhiy-storchaka commented Aug 31, 2026

Copy link
Copy Markdown
Member

XMLParser.feed() and ElementTree.parse() raised OverflowError: size does not fit in an int for input larger than 2 GiB, because Expat takes the length as an int. The data is now fed to Expat in chunks of 1 MiB, exactly as xml.parsers.expat has done since bpo-17089, so the pure Python implementation already accepted such input:

>>> data = b'<r>' + b'x' * ((1 << 31) + (1 << 27)) + b'</r>'
>>> len(ET.fromstring(data).text)
2281701376

Chunking is not slower for ordinary documents. Parsing 1 GiB with a million elements takes 2.37 s with chunks and 2.60 s with a single XML_Parse() call, presumably because 1 MiB chunks stay in cache.

test_length_overflow, which asserted the old error, is replaced by a bigmem test of the new behaviour, plus two cheap tests which run always, because the chunking loop is entered for any input larger than 1 MiB.

…n of ElementTree

XMLParser.feed() and ElementTree.parse() raised OverflowError, because Expat
takes the length as an int.  The data is now fed to Expat in chunks of 1 MiB,
as xml.parsers.expat does since bpo-17089, so the pure Python implementation
already accepted such input.
# Conflicts:
#	Lib/test/test_xml_etree.py
@serhiy-storchaka
serhiy-storchaka enabled auto-merge (squash) September 1, 2026 12:44
@serhiy-storchaka
serhiy-storchaka merged commit 9259e8b into python:main Sep 1, 2026
54 checks passed
@serhiy-storchaka
serhiy-storchaka deleted the gh-83895-large-input branch September 1, 2026 13:31
@serhiy-storchaka serhiy-storchaka added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels Sep 1, 2026
@miss-islington-app

Copy link
Copy Markdown

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 9259e8ba4de8e2f534d97233b8e0f9c2537691d3 3.13

@miss-islington-app

Copy link
Copy Markdown

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 9259e8ba4de8e2f534d97233b8e0f9c2537691d3 3.14

@miss-islington-app

Copy link
Copy Markdown

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.15 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 9259e8ba4de8e2f534d97233b8e0f9c2537691d3 3.15

@bedevere-app

bedevere-app Bot commented Sep 1, 2026

Copy link
Copy Markdown

GH-156771 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Sep 1, 2026
@bedevere-app

bedevere-app Bot commented Sep 1, 2026

Copy link
Copy Markdown

GH-156772 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Sep 1, 2026
@bedevere-app

bedevere-app Bot commented Sep 1, 2026

Copy link
Copy Markdown

GH-156773 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Sep 1, 2026
@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot wasm32-wasi 3.14 (tier-2) has failed when building commit 8becd16.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1745/builds/1893) and take a look at the build logs.
  4. Check if the failure is related to this commit (8becd16) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1745/builds/1893

Summary of the results of the build (if available):

Click to see traceback logs
Note: switching to '8becd160e7eb1b060506398bd97e466207cda363'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 8becd160e7eb1 [3.14] gh-83895: Accept input larger than 2 GiB in the C implementation of ElementTree (GH-156746) (GH-156772)
Switched to and reset branch '3.14'

⚠️ WARNING: This script is deprecated and slated for removal in Python 3.20; execute the `wasi/` directory instead (i.e. `python Tools/wasm/wasi`)


⚠️ WARNING: This script is deprecated and slated for removal in Python 3.20; execute the `wasi/` directory instead (i.e. `python Tools/wasm/wasi`)

rror: unable to open output file 'Modules/pyexpat.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3666: Modules/pyexpat.o] Error 1
make: *** Waiting for unfinished jobs....
rror: unable to open output file 'Programs/_bootstrap_python.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3306: Programs/_bootstrap_python.o] Error 1
rror: unable to open output file 'Modules/_elementtree.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3668: Modules/_elementtree.o] Error 1
rror: unable to open output file 'Modules/_ctypes/_ctypes_test.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3828: Modules/_ctypes/_ctypes_test.o] Error 1
Traceback (most recent call last):
  File "/buildarea/3.14.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi.py", line 12, in <module>
    runpy.run_path(pathlib.Path(__file__).parent / "wasi", run_name="__main__")
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen runpy>", line 311, in run_path
  File "<frozen runpy>", line 88, in _run_code
  File "/buildarea/3.14.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 589, in <module>
    main()
    ~~~~^^
  File "/buildarea/3.14.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 585, in main
    dispatch[context.subcommand](context)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/buildarea/3.14.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 118, in wrapper
    return func(context, working_dir)
  File "/buildarea/3.14.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 200, in make_build_python
    call(["make", "--jobs", str(cpu_count()), "all"], context=context)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildarea/3.14.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 152, in call
    subprocess.check_call(command, **kwargs, stdout=stdout, stderr=stderr)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.14/subprocess.py", line 420, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '--jobs', '4', 'all']' returned non-zero exit status 2.

⚠️ WARNING: This script is deprecated and slated for removal in Python 3.20; execute the `wasi/` directory instead (i.e. `python Tools/wasm/wasi`)

Traceback (most recent call last):
  File "/buildarea/3.14.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi.py", line 12, in <module>
    runpy.run_path(pathlib.Path(__file__).parent / "wasi", run_name="__main__")
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen runpy>", line 311, in run_path
  File "<frozen runpy>", line 88, in _run_code
  File "/buildarea/3.14.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 589, in <module>
    main()
    ~~~~^^
  File "/buildarea/3.14.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 444, in main
    default_logdir = pathlib.Path(tempfile.gettempdir())
                                  ~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib64/python3.14/tempfile.py", line 312, in gettempdir
    return _os.fsdecode(_gettempdir())
                        ~~~~~~~~~~~^^
  File "/usr/lib64/python3.14/tempfile.py", line 305, in _gettempdir
    tempdir = _get_default_tempdir()
  File "/usr/lib64/python3.14/tempfile.py", line 222, in _get_default_tempdir
    raise FileNotFoundError(_errno.ENOENT,
                            "No usable temporary directory found in %s" %
                            dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/buildarea/3.14.bcannon-wasi.wasi.debug/build']

serhiy-storchaka added a commit that referenced this pull request Sep 1, 2026
…on of ElementTree (GH-156746) (GH-156772)

XMLParser.feed() and ElementTree.parse() raised OverflowError, because Expat
takes the length as an int.  The data is now fed to Expat in chunks of 1 MiB,
as xml.parsers.expat does since bpo-17089, so the pure Python implementation
already accepted such input.
(cherry picked from commit 9259e8b)
@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Stable LTO 3.14 (tier-3) has failed when building commit 8becd16.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1731/builds/1318) and take a look at the build logs.
  4. Check if the failure is related to this commit (8becd16) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1731/builds/1318

Failed tests:

  • test_urllib2net

Failed subtests:

  • test_ftp_timeout - test.test_urllib2net.TimeoutTest.test_ftp_timeout
  • test_ftp_no_leak - test.test_urllib2net.OtherNetworkTests.test_ftp_no_leak
  • test_ftp_default_timeout - test.test_urllib2net.TimeoutTest.test_ftp_default_timeout

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 173, in test_ftp_no_leak
    with check_no_resource_warning(self):
         ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/support/warnings_helper.py", line 147, in check_no_resource_warning
    with check_no_warnings(testcase, category=ResourceWarning, force_gc=True):
         ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/support/warnings_helper.py", line 130, in check_no_warnings
    testcase.assertEqual(warns, [])
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
AssertionError: Lists differ: [<WarningMessage {message : ResourceWarnin[290 chars]ne}>] != []


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 411, in test_ftp_timeout
    u = _urlopen_with_retry(self.FTP_HOST, timeout=60)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 32, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 28, in _retry_thrice
    raise last_exc
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 24, in _retry_thrice
    return func(*args, **kwargs)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 187, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 1572, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind>


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 1559, in ftp_open
    fp, retrlen = fw.retrfile(file, type)
                  ~~~~~~~~~~~^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 1849, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
                    ~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/ftplib.py", line 353, in ntransfercmd
    host, port = self.makepasv()
                 ~~~~~~~~~~~~~^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/ftplib.py", line 327, in makepasv
    untrusted_host, port = parse227(self.sendcmd('PASV'))
                                    ~~~~~~~~~~~~^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/ftplib.py", line 281, in sendcmd
    return self.getresp()
           ~~~~~~~~~~~~^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/ftplib.py", line 254, in getresp
    raise error_perm(resp)
ftplib.error_perm: 500 OOPS: vsf_sysutil_bind


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 26, in _retry_thrice
    last_exc = e
ResourceWarning: unclosed <socket.socket fd=5, family=2, type=1, proto=6, laddr=('148.100.85.118', 45642)>
Warning -- Unraisable exception
Exception ignored while finalizing socket <socket.socket fd=6, family=2, type=1, proto=6, laddr=('148.100.85.118', 58452)>:
Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 26, in _retry_thrice
    last_exc = e
ResourceWarning: unclosed <socket.socket fd=6, family=2, type=1, proto=6, laddr=('148.100.85.118', 58452)>
k


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 390, in test_ftp_default_timeout
    u = _urlopen_with_retry(self.FTP_HOST)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 32, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 28, in _retry_thrice
    raise last_exc
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 24, in _retry_thrice
    return func(*args, **kwargs)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 187, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/urllib/request.py", line 1572, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind>


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.14.cstratak-fedora-stable-s390x.lto/build/Lib/test/test_urllib2net.py", line 26, in _retry_thrice
    last_exc = e
ResourceWarning: unclosed <socket.socket fd=3, family=2, type=1, proto=6, laddr=('148.100.85.118', 54950)>
k

@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 CentOS9 LTO 3.14 (tier-2) has failed when building commit 8becd16.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1691/builds/865) and take a look at the build logs.
  4. Check if the failure is related to this commit (8becd16) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1691/builds/865

Failed tests:

  • test_urllibnet
  • test_urllib2net

Failed subtests:

  • test_ftp_no_timeout - test.test_urllib2net.TimeoutTest.test_ftp_no_timeout
  • test_ftp_no_leak - test.test_urllib2net.OtherNetworkTests.test_ftp_no_leak
  • test_getcode - test.test_urllibnet.urlopenNetworkTests.test_getcode

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/test/test_urllibnet.py", line 107, in test_getcode
    self.assertEqual(e.exception.code, 404)
                     ^^^^^^^^^^^^^^^^
AttributeError: 'URLError' object has no attribute 'code'


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/urllib/request.py", line 1559, in ftp_open
    fp, retrlen = fw.retrfile(file, type)
                  ~~~~~~~~~~~^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/urllib/request.py", line 1849, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
                    ~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/ftplib.py", line 353, in ntransfercmd
    host, port = self.makepasv()
                 ~~~~~~~~~~~~~^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/ftplib.py", line 327, in makepasv
    untrusted_host, port = parse227(self.sendcmd('PASV'))
                                    ~~~~~~~~~~~~^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/ftplib.py", line 281, in sendcmd
    return self.getresp()
           ~~~~~~~~~~~~^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/ftplib.py", line 254, in getresp
    raise error_perm(resp)
ftplib.error_perm: 500 OOPS: vsf_sysutil_bind


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/test/test_urllib2net.py", line 180, in test_ftp_no_leak
    with check_no_resource_warning(self):
         ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/test/support/warnings_helper.py", line 147, in check_no_resource_warning
    with check_no_warnings(testcase, category=ResourceWarning, force_gc=True):
         ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/test/support/warnings_helper.py", line 130, in check_no_warnings
    testcase.assertEqual(warns, [])
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
AssertionError: Lists differ: [<WarningMessage {message : ResourceWarnin[276 chars]ne}>] != []


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/test/test_urllib2net.py", line 402, in test_ftp_no_timeout
    u = _urlopen_with_retry(self.FTP_HOST, timeout=None)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/test/test_urllib2net.py", line 32, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/test/test_urllib2net.py", line 28, in _retry_thrice
    raise last_exc
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/test/test_urllib2net.py", line 24, in _retry_thrice
    return func(*args, **kwargs)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/urllib/request.py", line 187, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-aarch64/3.14.cstratak-CentOS9-aarch64.lto/build/Lib/urllib/request.py", line 1572, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind>

@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 CentOS9 LTO + PGO 3.14 (tier-1) has failed when building commit 8becd16.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1789/builds/1159) and take a look at the build logs.
  4. Check if the failure is related to this commit (8becd16) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1789/builds/1159

Failed tests:

  • test_urllib2net

Failed subtests:

  • test_ftp - test.test_urllib2net.OtherNetworkTests.test_ftp
  • test_ftp_no_leak - test.test_urllib2net.OtherNetworkTests.test_ftp_no_leak

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/test_urllib2net.py", line 173, in test_ftp_no_leak
    with check_no_resource_warning(self):
         ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/support/warnings_helper.py", line 147, in check_no_resource_warning
    with check_no_warnings(testcase, category=ResourceWarning, force_gc=True):
         ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/support/warnings_helper.py", line 130, in check_no_warnings
    testcase.assertEqual(warns, [])
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
AssertionError: Lists differ: [<WarningMessage {message : ResourceWarnin[611 chars]ne}>] != []


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/urllib/request.py", line 1559, in ftp_open
    fp, retrlen = fw.retrfile(file, type)
                  ~~~~~~~~~~~^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/urllib/request.py", line 1832, in retrfile
    raise URLError(f'ftp error: {reason}') from reason
urllib.error.URLError: <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind>


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/test_urllib2net.py", line 307, in _test_urls
    f = urlopen(url, req, support.INTERNET_TIMEOUT)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/urllib/request.py", line 1572, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind>>


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/test_urllib2net.py", line 26, in _retry_thrice
    last_exc = e
ResourceWarning: unclosed <socket.socket fd=3, family=2, type=1, proto=6, laddr=('10.16.107.46', 50326)>
Warning -- Unraisable exception
Exception ignored while finalizing socket <socket.socket fd=4, family=2, type=1, proto=6, laddr=('10.16.107.46', 50336)>:
Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/test_urllib2net.py", line 26, in _retry_thrice
    last_exc = e
ResourceWarning: unclosed <socket.socket fd=4, family=2, type=1, proto=6, laddr=('10.16.107.46', 50336)>


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/test_urllib2net.py", line 307, in _test_urls
    f = urlopen(url, req, support.INTERNET_TIMEOUT)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/test_urllib2net.py", line 32, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/test_urllib2net.py", line 28, in _retry_thrice
    raise last_exc
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/test_urllib2net.py", line 24, in _retry_thrice
    return func(*args, **kwargs)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/urllib/request.py", line 1572, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind>>


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/urllib/request.py", line 1829, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
                    ~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/ftplib.py", line 353, in ntransfercmd
    host, port = self.makepasv()
                 ~~~~~~~~~~~~~^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/ftplib.py", line 327, in makepasv
    untrusted_host, port = parse227(self.sendcmd('PASV'))
                                    ~~~~~~~~~~~~^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/ftplib.py", line 281, in sendcmd
    return self.getresp()
           ~~~~~~~~~~~~^^
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/ftplib.py", line 254, in getresp
    raise error_perm(resp)
ftplib.error_perm: 500 OOPS: vsf_sysutil_bind


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/test_urllib2net.py", line 26, in _retry_thrice
    last_exc = e
ResourceWarning: unclosed <socket.socket fd=4, family=2, type=1, proto=6, laddr=('10.16.107.46', 50360)>
Warning -- Unraisable exception
Exception ignored while finalizing socket <socket.socket fd=4, family=2, type=1, proto=6, laddr=('10.16.107.46', 50378)>:
Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-CentOS9-x86_64/3.14.cstratak-CentOS9-x86_64.lto-pgo/build/Lib/test/test_urllib2net.py", line 26, in _retry_thrice
    last_exc = e
ResourceWarning: unclosed <socket.socket fd=4, family=2, type=1, proto=6, laddr=('10.16.107.46', 50378)>

@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Fedora Stable Clang Installed 3.14 (tier-2) has failed when building commit 8becd16.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1762/builds/798) and take a look at the build logs.
  4. Check if the failure is related to this commit (8becd16) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1762/builds/798

Failed tests:

  • test_urllib2net

Failed subtests:

  • test_ftp - test.test_urllib2net.OtherNetworkTests.test_ftp
  • test_ftp_no_leak - test.test_urllib2net.OtherNetworkTests.test_ftp_no_leak

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/urllib/request.py", line 1829, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
                    ~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/ftplib.py", line 353, in ntransfercmd
    host, port = self.makepasv()
                 ~~~~~~~~~~~~~^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/ftplib.py", line 327, in makepasv
    untrusted_host, port = parse227(self.sendcmd('PASV'))
                                    ~~~~~~~~~~~~^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/ftplib.py", line 281, in sendcmd
    return self.getresp()
           ~~~~~~~~~~~~^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/ftplib.py", line 254, in getresp
    raise error_perm(resp)
ftplib.error_perm: 500 OOPS: vsf_sysutil_bind


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/urllib/request.py", line 1559, in ftp_open
    fp, retrlen = fw.retrfile(file, type)
                  ~~~~~~~~~~~^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/urllib/request.py", line 1832, in retrfile
    raise URLError(f'ftp error: {reason}') from reason
urllib.error.URLError: <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind>


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/test/test_urllib2net.py", line 173, in test_ftp_no_leak
    with check_no_resource_warning(self):
         ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/test/support/warnings_helper.py", line 147, in check_no_resource_warning
    with check_no_warnings(testcase, category=ResourceWarning, force_gc=True):
         ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/test/support/warnings_helper.py", line 130, in check_no_warnings
    testcase.assertEqual(warns, [])
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
AssertionError: Lists differ: [<WarningMessage {message : ResourceWarnin[324 chars]ne}>] != []


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/test/test_urllib2net.py", line 307, in _test_urls
    f = urlopen(url, req, support.INTERNET_TIMEOUT)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/test/test_urllib2net.py", line 32, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/test/test_urllib2net.py", line 28, in _retry_thrice
    raise last_exc
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/test/test_urllib2net.py", line 24, in _retry_thrice
    return func(*args, **kwargs)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-aarch64/3.14.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.14/urllib/request.py", line 1572, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind>>

serhiy-storchaka added a commit that referenced this pull request Sep 1, 2026
…on of ElementTree (GH-156746) (GH-156773)

XMLParser.feed() and ElementTree.parse() raised OverflowError, because Expat
takes the length as an int.  The data is now fed to Expat in chunks of 1 MiB,
as xml.parsers.expat does since bpo-17089, so the pure Python implementation
already accepted such input.
(cherry picked from commit 9259e8b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants