Repository navigation
Fix Pynini compatibility on Python 3.14 - #531
Open
janbernloehr wants to merge 4 commits into
Open
janbernloehr wants to merge 4 commits into
janbernloehr wants to merge 4 commits into
Conversation
Use pynini 2.1.7 on Python >= 3.14 and preserve 2.1.6.post1 on older interpreters in both runtime and test requirements. Fixes NVIDIA#530 Signed-off-by: Jan Bernloehr <jbernloehr@nvidia.com>
janbernloehr
force-pushed
the
fix/pynini-python314
branch
from
November 12, 2026 13:19
b8bb72c to
36e2321
Compare
Give the alphanumeric fallback a slightly higher weight in deterministic mode so equal-cost paths do not change the reading of 133-ABC when upgrading Pynini. The existing serial regression test covers this case. Signed-off-by: Jan Bernloehr <jbernloehr@nvidia.com>
Signed-off-by: Jan Bernloehr <jbernloehr@nvidia.com>
Signed-off-by: Jan Bernloehr <jbernloehr@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Allow Pynini 2.1.7 on all supported Python versions by using the shared upper bound
pynini<=2.1.7in runtime requirements, test requirements, and the README Conda installation command. This fixes Linux x86_64 installation on Python 3.14.Fixes #530.
The unconditional 2.1.6.post1 pin forces Python 3.14 installations to build Pynini from source because that release has no compatible wheel. Without OpenFst development headers, the build fails with
fst/util.h: No such file or directory. Pynini 2.1.7 provides a CPython 3.14 Linux x86_64 wheel.The upgrade also changes which equally weighted English grammar paths are selected. Add deterministic priorities to preserve the existing expected readings of
133-ABCand the URL suffixprogram(which otherwise becomesproGRAMin Sparrowhawk). Existing serial and electronic test cases cover both regressions; nondeterministic grammar behavior is unchanged.Use the same upper bound in both requirements files so installing the
testorallextra does not reintroduce the incompatible pin. Quote the Conda requirement so the shell does not interpret<as redirection.Packaging verification
Manually resolved the Pynini entries extracted from each changed requirements file against public PyPI, targeting
x86_64-manylinux_2_28with source builds disabled:The unpatched
pynini==2.1.6.post1requirement fails the same Python 3.14 check with “no usable wheels.” The built package'sRequires-Distmetadata containspynini<=2.1.7for runtime dependencies and thetestandallextras, with no Python-version marker.To reproduce the resolver check from the patched checkout:
Manual grammar validation
Repeated all 22 CPU test groups on the final revision with Linux x86_64 / Python 3.13.12 / Pynini 2.1.7 after applying the common upper bound: 13,533 passed, 4 known baseline failures, no errors or skips. This verifies the newer Pynini version on Python below 3.14.
Earlier validation installed the checkout with its
testextra on Linux x86_64 / Python 3.14.3 / Pynini 2.1.7 and ran all 22 CPU test groups separately: 13,533 passed, 4 failed. The four failures are Italian electronic-address test-mode mismatches addressed separately in #533; all four also reproduce on unchanged upstreamddadfb2awith Python 3.13.12 / Pynini 2.1.6.post1 (48 passed, 4 failed).The complete English suite was repeated after both grammar corrections: 1,615 passed on each dependency version. The aggregate above combines the broad dependency-upgrade sweep with this final English rerun. Kinyarwanda passed all 100 cases without the optional cache flag; enabling caching causes the same collection error on upstream. Optional audio-based expansion and model downloads were not enabled.
English Sparrowhawk with the same grammar source: 39/39 test functions passed across cased TN and lower-cased/cased ITN, using exported Pynini 2.1.7 grammars and the repository's deployment container and shell tests. All 17 TN functions were repeated after both corrections; the unchanged ITN functions passed in the preceding run. Both affected TN groups (serial and electronic) also passed with corrected grammars exported using Pynini 2.1.6.post1. Non-English Sparrowhawk exports, Python versions after 3.14, and other platforms were not tested.
If #474 lands first, the upper bound needs to move into its
pyproject.tomlruntime dependencies andtestextra.Before your PR is "Ready for review"
Pre checks:
Checked items below have been reviewed and are either satisfied or explicitly not applicable to this compatibility fix.
git commit -sto sign. — All commits have matching authors andSigned-off-bytrailers.pytestor (if your machine does not have GPU)pytest --cpufrom the root folder (given you marked your test cases accordingly@pytest.mark.run_only_on('CPU')).bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...pytestand Sparrowhawk here. — No new feature. Existing Python and Sparrowhawk serial/electronic test cases cover both grammar regressions.__init__.pyfor every folder and subfolder, includingdatafolder which has .TSV files? — N/A: no new folders.Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.to all newly added Python files? — N/A: no Python files added.Copyright 2015 and onwards Google, Inc.. See an example here. — N/A: no copied code.try import: ... except: ...) if not already done. — No import guards added.PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Prepared with assistance from OpenAI Codex (GPT-6).