Skip to content

{AKS} Pin compatible pytest for live-test runners - #10556

Merged
JaysonTaiMicrosoft merged 2 commits into
Azure:mainfrom
FumingZhang:fumingzhang/fix-aks-live-test-pytest-forked
Nov 5, 2026
Merged

JaysonTaiMicrosoft merged 2 commits into
Azure:mainfrom
FumingZhang:fumingzhang/fix-aks-live-test-pytest-forked

Conversation

@FumingZhang

@FumingZhang FumingZhang commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

🤖 PR Validation — ️✔️ All clear

Breaking Changes Suppressions
️✔️ None ️✔️ None

This checklist is used to make sure that common guidelines for a pull request are followed.

Related command

AKS live/unit test harness for the ACS module and aks-preview; no az aks command behavior changes.

Description

Fresh live-test environments can install pytest 9.2.0 through azdev's unconstrained dependency, while installTestPackages explicitly installs pytest-forked==1.6.0. Pytest 9.2 reports contain a structured NodeId object that this plugin cannot encode with marshal.dumps. Even a passing test is consequently reported as CRASHED with signal 0 with ValueError: unmarshallable object.

Pin pytest==9.1.1 alongside the existing plugin versions in the shared environment setup script. This installation runs after CLI/extension setup, so it restores the compatible version before tests execute. Keep forked isolation, test selection, exit-code handling, and the other dependency pins unchanged. Document why the two versions must stay compatible.

Related: #10472. This complements #10534, which constrains ordinary CI setup but does not cover this AKS live-test harness. No extension release/version or generated index changes are needed.

Validation

Validated on Linux with Python 3.12.3, retaining pytest-xdist==3.8.0, pytest-json-report==1.5.0, pytest-rerunfailures==11.0, pytest-cov==4.0.0, and pytest-forked==1.6.0.

  • bash -n src/aks-preview/azcli_aks_live_test/scripts/setup_venv.sh passed.
  • Executed the edited installTestPackages function in an isolated virtual environment initially containing pytest 9.2.0. Confirmed it installs pytest 9.1.1 and retains forked 1.6.0.
  • python -m pip check passed.
  • Temporary offline probes exercised passing, assertion-failure, fixture-setup-error, and skipped tests with python -m pytest -n 1 --forked --json-report.
Dependency combination Offline probe result
pytest 9.2.0 + forked 1.6.0 All four probes misreported as serialization crashes, including the passing probe
Edited installer: pytest 9.1.1 + forked 1.6.0 Correctly reports one pass, one expected assertion failure, one expected setup error, and one skip; preserves original error details
Edited installer, passing probe only One passed; exit code 0

The negative probes are intentional and their expected outcomes were asserted. They are not extension test failures. No Azure live tests were queued or Azure resources created; the KMS scenario still needs a live rerun after this harness fix is available.

General Guidelines

  • Have you run azdev style <YOUR_EXT> locally? (pip install azdev required) — Not applicable: only a Bash dependency installation and harness documentation changed.
  • Have you run python scripts/ci/test_index.py -q locally? (pip install azdev required) — Not run: extension packages and index are unchanged.
  • My extension version conforms to the Extension version schema — Not applicable: no extension package/version change.

For new extensions:

About Extension Publish

There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify src/index.json.

Keep pytest 9.1.1 paired with pytest-forked 1.6.0 so report serialization does not turn passing live tests into subprocess crashes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@x-engineering-agent

Copy link
Copy Markdown

Hi FumingZhang,
Please write the description of changes which can be perceived by customers into HISTORY.rst.
If you want to release a new extension version, please update the version in pyproject.toml (or setup.py, if the extension has not migrated yet) as well.

@yonzhan

Copy link
Copy Markdown
Collaborator

Please fix CI issues

Update only the six vault-resource URI query versions to match azure-mgmt-keyvault 14.1.0. Preserve request/response payloads and verify the full extension scenario in local playback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@FumingZhang
FumingZhang marked this pull request as ready for review November 4, 2026 23:55
Copilot AI balanced review requested due to automatic review settings November 4, 2026 23:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the test-harness compatibility and recording updates.

No open issues 🎉

What changed in this PR

This PR fixes pytest compatibility in the shared AKS live/unit test harness, complementing the ordinary CI constraint in #10534 without changing az aks command behavior.

Changes:

  • Pins pytest==9.1.1 alongside the existing plugins to prevent forked test-report serialization failures.
  • Documents the pytest/pytest-forked compatibility requirement.
  • Updates the app-routing test recording’s Key Vault API version to address the playback mismatch noted in #10472.
File Description
src/​aks-preview/​azext_aks_preview/​tests/​latest/​recordings/​test_aks_approuting_enable_with_keyvault_secrets_provider_addon_and_keyvault_id.yaml Updates recorded Key Vault request URLs to API version 2026-05-15.
src/​aks-preview/​azcli_aks_live_test/​scripts/​setup_venv.sh Pins pytest to 9.1.1, preserving existing plugin pins.
src/​aks-preview/​azcli_aks_live_test/​README.md Explains the runner dependency pins and serialization incompatibility.

Review effort: Balanced

@JaysonTaiMicrosoft
JaysonTaiMicrosoft merged commit 86605f3 into Azure:main Nov 5, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants