Skip to content

[documentdb] Fix restore authentication and CMK support - #10537

Merged
xuming-ms merged 10 commits into
Azure:mainfrom
amatarritamicrosoft:users/amatarrita/restore-auth
Nov 17, 2026
Merged

xuming-ms merged 10 commits into
Azure:mainfrom
amatarritamicrosoft:users/amatarrita/restore-auth

Conversation

@amatarritamicrosoft

@amatarritamicrosoft amatarritamicrosoft commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

🤖 PR Validation — ⚠️ Review suggested

Breaking Changes Suppressions
⚠️ None ️✔️ None
⚠️Azure CLI Extensions Breaking Change Test
⚠️documentdb
rule cmd_name rule_message suggest_message
⚠️ 1006 - ParaAdd documentdb mongocluster restore cmd documentdb mongocluster restore added parameter auth_allowed_modes
⚠️ 1006 - ParaAdd documentdb mongocluster restore cmd documentdb mongocluster restore added parameter encryption
⚠️ 1006 - ParaAdd documentdb mongocluster restore cmd documentdb mongocluster restore added parameter mi_user_assigned
⚠️ 1006 - ParaAdd documentdb mongocluster restore cmd documentdb mongocluster restore added parameter public_network_access
⚠️ 1006 - ParaAdd documentdb mongocluster restore cmd documentdb mongocluster restore added parameter storage_type
⚠️ 1006 - ParaAdd documentdb mongocluster restore cmd documentdb mongocluster restore added parameter tags
⚠️ 1009 - ParaPropRemove documentdb mongocluster restore cmd documentdb mongocluster restore update parameter admin_password: removed property required=True
⚠️ 1009 - ParaPropRemove documentdb mongocluster restore cmd documentdb mongocluster restore update parameter admin_user: removed property required=True

Summary

  • Support authentication modes, user-assigned managed identities, and customer-managed
    key encryption in az documentdb mongocluster restore.
  • Add optional --storage-type, --public-network-access, and --tags parameters. Reuse the
    existing generated AAZ argument definitions; update only the custom restore argument
    filter and request serializer.
  • Preserve the previous request payload when the new options are omitted. Retain the
    NativeAuth credential-pair checks, Entra-only credential validation, and CMK identity
    assignment validation.
  • Update help and README to identify supported target configuration and unavailable
    overrides without claiming that all source settings are inherited.
  • Document existing core networking and REST paths for Private Link, without adding
    dedicated DocumentDB commands or treating public access settings as Private Link setup.
  • Include these changes in the existing 1.1.0b2 release entry. Extension preview
    metadata and command-level preview annotations are unchanged.

REST contract findings

The shared ARM CreateOrUpdate schema covers multiple creation modes. A writable
property in that schema does not, by itself, establish that PITR accepts or honors an
override. The original full-configuration restore request was rejected by the service.

The following observations come from controlled REST tests using
2026-06-15-preview in eastus2; they are not claims about every API version or cloud.
Successful cases required a terminal Succeeded / Ready response and the expected
property value, not just HTTP 201 admission.

Property Observed behavior and CLI scope
serverVersion, compute, sharding, highAvailability, dataApi, networkBypassMode Rejected as undefined properties in the PITR request schema, including probes using the source values. Restore does not expose these overrides.
storage.sizeGb Accepted in the tested requests but the requested resize was not applied: the target retained 128 GiB, including a request for 1024 GiB together with PremiumSSDv2. A storage-size override is not exposed.
storage.type Explicit storage type was honored. PremiumSSD to PremiumSSDv2 succeeded; the reverse direction returned an explicit unsupported-migration error. Exposed through --storage-type.
publicNetworkAccess Explicit values were honored. Omitting the property produced Enabled in the tested restore from a Disabled source. Exposed through --public-network-access.
tags The requested resource-root tag map was applied exactly, without merging source-only tags. Exposed through optional --tags; omission preserves the existing request payload.
previewFeatures (GeoReplicas) Rejected with HTTP 400 because previewFeatures is undefined in the PITR-specific request schema. Restore does not expose this override. This does not establish inheritance behavior from an already feature-enabled source.
authConfig.allowedModes NativeAuth, restores of Entra-only sources, and dual-authentication source-to-target restores were validated. The dual-auth target retained exactly NativeAuth and MicrosoftEntraID and the expected native administrator. Entra-only authentication with native administrator credentials returns HTTP 400; the CLI rejects this invalid combination before sending the request.
User-assigned identity and CMK configuration Supported combinations were validated and are already covered by this PR's identity, encryption, and authentication tests. This does not claim support for every value in the shared managed-identity schema.

Test coverage

  • Exact request serialization for individual options and their combinations.
  • Both supported storage/network values, nonempty and empty tag maps, omission behavior,
    and preservation of the NativeAuth, Entra-only, CMK, and user-assigned identity request shapes.
  • Independent restore, create, and update argument schemas; unsupported restore
    arguments remain hidden.
  • Independent CLI scenarios for storage type, public network access, and their
    combination. Each starts from a PremiumSSD / Enabled source and checks the
    restored cluster's terminal state and exact target settings before cleanup.
  • Existing NativeAuth, Entra-only, CMK, and Entra-only + CMK recording regressions.
  • Independent real CLI scenarios for tags and dual-authentication restore. The tags
    scenario checks an exact target map distinct from the source; the dual-auth scenario
    checks both modes without relying on their order and verifies the native administrator.

Live CLI results

The three storage/network scenarios below completed successfully using the modified CLI, with
assertions enabled and a separate genuine recording for each scenario.

Scenario Final storage type Final public network access Final state
--storage-type only PremiumSSDv2 Enabled Succeeded / Ready
--public-network-access only PremiumSSD Disabled Succeeded / Ready
Both options PremiumSSDv2 Disabled Succeeded / Ready

Each of these three restored clusters retained 128 GiB and NativeAuth. The recordings contain the
actual restore requests, terminal-state responses, and source/target deletions.
Independent Azure checks confirmed that all three owned resource groups were
deleted after the runs.

The review follow-up adds two more independently recorded CLI scenarios:

Scenario Verified target configuration Final state
--tags Exactly environment=restored and team=documentdb; the source-only tag is absent. NativeAuth, administrator, and inherited storage configuration are preserved. Succeeded / Ready
Dual-authentication source to dual-authentication target Exactly MicrosoftEntraID and NativeAuth, the expected administrator, and matching restore source/time and inherited configuration. Succeeded / Ready

Both follow-up scenarios passed against the service. Their genuine recordings contain
59 and 58 interactions respectively, including source/target deletion. Independent Azure
checks confirmed both resource groups were removed.

Follow-up validation

  • Full guarded playback passed 43 tests and 23 subtests, covering all 19 recorded
    scenarios
    , with assertions enabled and unchanged cassette hashes.
  • Style, command linter, and credential scans passed. Request passwords and signed
    operation values are redacted without fabricating service results or weakening matchers.
  • Six independent reviewers found no significant issues in schema/serialization,
    authentication interactions, tests, recording integrity, service claims, or documentation.
  • The documented Private Link resource/connection list commands and their explicit
    2026-06-15-preview REST alternatives were verified read-only against a test cluster.
    Endpoint creation, connection approval, and private DNS setup were not exercised here.

Authentication assertions verify management-plane configuration; these scenarios do not
validate database-document contents or successful data-plane sign-in.

Expose Entra and CMK/UAMI options while preserving legacy NativeAuth behavior; add validation and recorded coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 46bb57cf-0490-4f99-a750-27ca921e5d89
Copilot AI balanced review requested due to automatic review settings October 29, 2026 18:04
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Recordings can retain signed operation-result URL parameters, and the Entra-only, CMK-encrypted restore path lacks service coverage.

2 open issues
What changed in this PR

This PR extends DocumentDB point-in-time restore to support Microsoft Entra-only authentication, customer-managed keys, and user-assigned identities while preserving the legacy NativeAuth payload.

Changes:

  • Add restore authentication and encryption options with argument validation.
  • Expand payload tests and restore scenarios, and introduce polling-token redaction for recordings.
  • Update restore documentation and release metadata to 1.1.0b2.
File Description
src/​documentdb/​setup.py Bumps the extension version to 1.1.0b2.
src/​documentdb/​README.md Documents authentication, encryption, and restore limitations.
src/​documentdb/​HISTORY.rst Records the new restore capabilities.
src/​documentdb/​azext_documentdb/​tests/​latest/​test_restore_custom.py Adds payload, validation, and token-redaction tests.
src/​documentdb/​azext_documentdb/​tests/​latest/​test_documentdb.py Expands restore scenarios and adds recording sanitization.
src/​documentdb/​azext_documentdb/​custom.py Implements restore options, validation, and request serialization.

Review effort: Balanced

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

Re-record with Azure CLI 2.92 and Key Vault SDK 14.0.1. Redact signed operation-result URLs and cover both polling routes in sanitizer tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 46bb57cf-0490-4f99-a750-27ca921e5d89
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

Apply the operation-result URL sanitizer to legacy and Entra restore cassettes without changing recorded service behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 46bb57cf-0490-4f99-a750-27ca921e5d89
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

Record restoration from an Entra-only encrypted source without native administrator credentials or authConfig. Verify inherited authentication, CMK and identity while reusing the existing CMK scenario setup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 46bb57cf-0490-4f99-a750-27ca921e5d89
@amatarritamicrosoft

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Commenter does not have sufficient privileges for PR 10537 in repo Azure/azure-cli-extensions

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

@yonzhan

Copy link
Copy Markdown
Collaborator

Please fix CI issues

Expose supported PITR target settings without changing omitted-option payloads, with focused help and separate live CLI recordings.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 46bb57cf-0490-4f99-a750-27ca921e5d89
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

amatarritamicrosoft and others added 2 commits November 9, 2026 23:36
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 46bb57cf-0490-4f99-a750-27ca921e5d89
Re-record all three CMK scenarios with Key Vault API 2026-05-15. Preserve strict request matching and verify the full DocumentDB scenario and unit suite.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 46bb57cf-0490-4f99-a750-27ca921e5d89
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

@sachim

sachim commented Nov 13, 2026

Copy link
Copy Markdown
Member

ARM REST API coverage review

Reviewed commit d1cfe7e4f4ede45b8a14612dca4450ea2633d082 against the 2026-06-15-preview ARM specification.

Summary: This covers the principal point-in-time restore scenarios, but it should not be described as complete ARM parity. I did not identify a confirmed payload defect in the newly added restore options. The broader gaps below are predominantly pre-existing and need not all be addressed in this scoped PR.

Restore coverage

The serializer covers all request fields used by the official native-auth, Entra-auth, and CMK PITR examples:

Setting Coverage
PITR creation mode, source resource ID, and restore timestamp Implemented
Administrator username/password Implemented; preserves legacy behavior and validates credential pairs
Authentication modes NativeAuth, Entra-only, and dual-auth payloads implemented
User-assigned identities and CMK configuration Implemented, including validation that the CMK identity is assigned
Storage type Both enum values exposed; migration restrictions remain service-enforced
Public network access Enabled and Disabled exposed
Target resource tags Not exposed by restore
Preview features Not exposed by restore; PITR behavior remains unverified

The shared CreateOrUpdate schema does not fully encode mode-specific restrictions. The PR's reported REST probes show that several otherwise writable fields are rejected or ignored during PITR. I would not expose server version, compute tier, storage size, shard count, HA, Data API, or network bypass merely to match that shared schema. Those runtime observations are specific to the tested API version and region.

Coverage gaps and recommendations

  1. Entra-only cluster creation remains unavailable through create (pre-existing). The generated create schema unconditionally requires administrator username/password, even when selecting Entra-only authentication. The new scenarios seed Entra-only sources using REST to work around this. Track conditional administrator validation for create, similar to restore.

  2. CMK replica creation remains uncovered (pre-existing). ARM has an explicit CMK geo-replica creation example, but ReplicaCreate hides encryption/identity arguments and serializes neither. Restore CMK support does not complete replica CMK support; this can be a separate follow-up.

  3. Make an explicit decision about restore tags and preview features. Both are excluded by the restore argument filter and omitted from serialization, but neither appears in the documented list of unsupported overrides. Validate target tags during PITR and expose them if supported. Establish whether preview features are accepted, inherited, or rejected before adding that flag. Document intentional exclusions; their presence in the shared schema alone does not establish PITR support.

  4. Private Link operations lack dedicated DocumentDB commands (broader extension scope). ARM includes private endpoint connection list/get/create/delete and private link resource listing. The extension has no corresponding dedicated group. This is not necessarily an Azure CLI-wide gap: core networking commands or az rest may provide these workflows. Document the intended command path or track dedicated coverage.

  5. Add service coverage for dual-auth restore. A unit test verifies its request payload, but I did not find a dedicated recorded dual-auth restore scenario. Validate the final restored authentication modes and administrator configuration against the service.

Validation performed

  • Restore unit tests: 23 passed, with 18 subtests passed.
  • Isolated recorded Entra restore scenario: passed.
  • Combined playback: not fully completed. The first reproduced failure was a Key Vault client/recording API-version mismatch: the local client requested 2025-05-01, while the recording expected 2026-05-15. This is not evidence of a DocumentDB restore payload defect.
  • A subsequent long-running playback attempt was stopped; a full playback pass is not claimed.
  • No live Azure operations were run during this review.

Recommendation: Treat this as a scoped restore enhancement. Before claiming comprehensive coverage, resolve or explicitly track the gaps above and obtain a full playback pass with aligned dependencies.

@amatarritamicrosoft

amatarritamicrosoft commented Nov 13, 2026 •

Copy link
Copy Markdown
Member Author

ARM REST API coverage review

Reviewed commit d1cfe7e4f4ede45b8a14612dca4450ea2633d082 against the 2026-06-15-preview ARM specification.

Summary: This covers the principal point-in-time restore scenarios, but it should not be described as complete ARM parity. I did not identify a confirmed payload defect in the newly added restore options. The broader gaps below are predominantly pre-existing and need not all be addressed in this scoped PR.

Restore coverage

The serializer covers all request fields used by the official native-auth, Entra-auth, and CMK PITR examples:

Setting Coverage
PITR creation mode, source resource ID, and restore timestamp Implemented
Administrator username/password Implemented; preserves legacy behavior and validates credential pairs
Authentication modes NativeAuth, Entra-only, and dual-auth payloads implemented
User-assigned identities and CMK configuration Implemented, including validation that the CMK identity is assigned
Storage type Both enum values exposed; migration restrictions remain service-enforced
Public network access Enabled and Disabled exposed
Target resource tags Not exposed by restore
Preview features Not exposed by restore; PITR behavior remains unverified
The shared CreateOrUpdate schema does not fully encode mode-specific restrictions. The PR's reported REST probes show that several otherwise writable fields are rejected or ignored during PITR. I would not expose server version, compute tier, storage size, shard count, HA, Data API, or network bypass merely to match that shared schema. Those runtime observations are specific to the tested API version and region.

Coverage gaps and recommendations

  1. Entra-only cluster creation remains unavailable through create (pre-existing). The generated create schema unconditionally requires administrator username/password, even when selecting Entra-only authentication. The new scenarios seed Entra-only sources using REST to work around this. Track conditional administrator validation for create, similar to restore.
  2. CMK replica creation remains uncovered (pre-existing). ARM has an explicit CMK geo-replica creation example, but ReplicaCreate hides encryption/identity arguments and serializes neither. Restore CMK support does not complete replica CMK support; this can be a separate follow-up.
  3. Make an explicit decision about restore tags and preview features. Both are excluded by the restore argument filter and omitted from serialization, but neither appears in the documented list of unsupported overrides. Validate target tags during PITR and expose them if supported. Establish whether preview features are accepted, inherited, or rejected before adding that flag. Document intentional exclusions; their presence in the shared schema alone does not establish PITR support.
  4. Private Link operations lack dedicated DocumentDB commands (broader extension scope). ARM includes private endpoint connection list/get/create/delete and private link resource listing. The extension has no corresponding dedicated group. This is not necessarily an Azure CLI-wide gap: core networking commands or az rest may provide these workflows. Document the intended command path or track dedicated coverage.
  5. Add service coverage for dual-auth restore. A unit test verifies its request payload, but I did not find a dedicated recorded dual-auth restore scenario. Validate the final restored authentication modes and administrator configuration against the service.

Validation performed

  • Restore unit tests: 23 passed, with 18 subtests passed.
  • Isolated recorded Entra restore scenario: passed.
  • Combined playback: not fully completed. The first reproduced failure was a Key Vault client/recording API-version mismatch: the local client requested 2025-05-01, while the recording expected 2026-05-15. This is not evidence of a DocumentDB restore payload defect.
  • A subsequent long-running playback attempt was stopped; a full playback pass is not claimed.
  • No live Azure operations were run during this review.

Recommendation: Treat this as a scoped restore enhancement. Before claiming comprehensive coverage, resolve or explicitly track the gaps above and obtain a full playback pass with aligned dependencies.

About "* Combined playback: not fully completed. The first reproduced failure was a Key Vault client/recording API-version mismatch: the local client requested 2025-05-01, while the recording expected 2026-05-15. This is not evidence of a DocumentDB restore payload defect."

Thanks for checking. The CMK recordings were generated with  azure-mgmt-keyvault==14.1.0 , which uses API version  2026-05-15 . Full playback passed on the reviewed commit,  d1cfe7e4f , in CI build 373066: 40 tests and 18 subtests passed on each Python version from 3.10 through 3.14, including all three CMK scenarios. This supports your conclusion that the local failure is a dependency mismatch rather than a restore payload defect.

"CMK replica creation remains uncovered (pre-existing). ARM has an explicit CMK geo-replica creation example, but ReplicaCreate hides encryption/identity arguments and serializes neither. Restore CMK support does not complete replica CMK support; this can be a separate follow-up."

I will take in a different PR as this is for another command, out of scope for this PR

"Entra-only cluster creation remains unavailable through create (pre-existing). The generated create schema unconditionally requires administrator username/password, even when selecting Entra-only authentication. The new scenarios seed Entra-only sources using REST to work around this. Track conditional administrator validation for create, similar to restore."

I will take in a different PR as well

amatarritamicrosoft and others added 2 commits November 13, 2026 13:52
Expose optional resource-root tags after service validation, preserve omitted-option behavior, document rejected preview overrides and existing Private Link paths, and add independent real tags and dual-auth recordings.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 46bb57cf-0490-4f99-a750-27ca921e5d89
Keep the target branch index and unrelated upstream changes synchronized without rewriting the reviewed restore commits.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 46bb57cf-0490-4f99-a750-27ca921e5d89
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

@amatarritamicrosoft

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Commenter does not have sufficient privileges for PR 10537 in repo Azure/azure-cli-extensions

Configure Entra authentication and users after restore. Preserve create/update
and CMK coverage, retire explicit Entra restore scenarios, and document the flow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 46bb57cf-0490-4f99-a750-27ca921e5d89
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

@xuming-ms
xuming-ms merged commit 722ea99 into Azure:main Nov 17, 2026
29 of 30 checks passed
@azclibot

Copy link
Copy Markdown
Collaborator

[Release] Update index.json for extension [ documentdb-1.1.0b2 ] : https://dev.azure.com/msazure/One/_build/results?buildId=191624581&view=results

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants