Skip to content

[aks-preview] Add direct Agent Sandbox extension commands - #10517

Open
Alima Azamat (alimaazamat) wants to merge 7 commits into
Azure:mainfrom
alimaazamat:alimaazamat/aks-agent-sandbox-cli
Open

Alima Azamat (alimaazamat) wants to merge 7 commits into
Azure:mainfrom
alimaazamat:alimaazamat/aks-agent-sandbox-cli

Conversation

@alimaazamat

@alimaazamat Alima Azamat (alimaazamat) commented Oct 23, 2026 •

Copy link
Copy Markdown

🤖 PR Validation — ❌ Action needed

Breaking Changes Suppressions
❌ 1 ⚠️ 1 added
❌Azure CLI Extensions Breaking Change Test
❌managednetworkfabric
rule cmd_name rule_message suggest_message
❌ 1002 - CmdRemove networkfabric bootstrapdevice reboot cmd networkfabric bootstrapdevice reboot removed please confirm cmd networkfabric bootstrapdevice reboot removed
❌ 1002 - CmdRemove networkfabric bootstrapdevice refresh-config cmd networkfabric bootstrapdevice refresh-config removed please confirm cmd networkfabric bootstrapdevice refresh-config removed
❌ 1002 - CmdRemove networkfabric bootstrapdevice resync-password cmd networkfabric bootstrapdevice resync-password removed please confirm cmd networkfabric bootstrapdevice resync-password removed
❌ 1002 - CmdRemove networkfabric bootstrapdevice update-admin-state cmd networkfabric bootstrapdevice update-admin-state removed please confirm cmd networkfabric bootstrapdevice update-admin-state removed
❌ 1002 - CmdRemove networkfabric bootstrapinterface update-admin-state cmd networkfabric bootstrapinterface update-admin-state removed please confirm cmd networkfabric bootstrapinterface update-admin-state removed

This PR adds warning or check suppressions. Fix the underlying warning where possible and justify each remaining suppression in a code comment or the PR description.

1 added suppression
File Line Kind Code
src/aks-preview/azext_aks_preview/tests/latest/test_agent_sandbox_extension_scenario.py 18 skipped test self.skipTest(

Related commands

  • az aks extension create --extension-type Microsoft.AgentSandbox
  • az aks extension update/show/list/delete
  • az aks extension type discovery for Agent Sandbox

Changes

Replaces this PR's earlier managed-profile flags with direct extension installation through the existing aks extension wrapper. Add microsoft.agentsandbox to its allowlist; reuse the installed k8s-extension implementation and extension client with cluster_type="managedClusters".

The request goes to Microsoft.KubernetesConfiguration/extensions (ClusterConfig), then Extension Manager and Helm. Configuration settings, protected settings, and configuration-file arguments are forwarded by the existing wrapper. No agentSandboxProfile writes or profile enable/disable flags remain.

Add create/update examples, lifecycle and configuration documentation, Pending release notes, focused delegation/discovery/error/argument-alias tests, and a live-only fresh-install lifecycle scenario. Re-raise logged create/update/delete/show/list failures so authorization or installation failures do not appear successful; this error-propagation correction also applies to existing supported extension types.

Fresh-create testing exposed a reserved-name restriction: names containing aks-managed are restricted to authorized service clients. The direct-install examples and tests now use the customer-managed name agent-sandbox.

Prerequisites and ownership

  • Install the k8s-extension CLI dependency.
  • The shared delete-lookup fix, its tests, and the k8s-extension version/history changes have moved to #10528 - [k8s-extension] Propagate non-404 delete lookup failures. This PR changes only aks-preview.
  • Normal successful lifecycle operations do not depend on that fix; the fresh lifecycle below passed with installed k8s-extension 1.8.0. Correct error propagation when deletion's initial lookup fails with a non-404 error requires the corrected package to be separately published and installed (1.9.5 proposed in [k8s-extension] Propagate non-404 delete lookup failures #10528). Updating aks-preview alone does not update the dependency; no minimum-version gate is added here.
  • ClusterConfig registration must explicitly permit direct customer lifecycle operations. This CLI change does not alter registration permissions, preview eligibility, or Azure RBAC.
  • This path does not invoke the AKS profile's Microsoft.ContainerService/AgentSandboxPreview AFEC validation. Direct-install eligibility remains a service registration concern.
  • Do not operate directly on an instance owned by agentSandboxProfile. Disabling the profile requests deletion, not an ownership handoff; existing installations need a separately coordinated migration.
  • The base chart has defaults. Configuration overrides can be passed with --config or --config-file / --config-settings-file. Kata node pools and PSR are not provisioned by this wrapper.

Validation

Live lifecycle run - October 27, 2026

Passed: fresh create -> show/list -> configuration update -> delete, using the actual installed delegated implementation and Azure service, not mocks.

  • Tested source: d8d8f0381.
  • Live-only scenario, with reproduction instructions.
  • CLI 2.89.1; aks-preview 22.0.0b15, loaded from the tested repository source; installed k8s-extension 1.8.0; service-selected Agent Sandbox chart 1.0.4.
  • Region westcentralus; fresh, dedicated development AKS cluster; Kubernetes 1.36; one Standard_D4s_v6 node; managed identity and OIDC enabled.
  • Prerequisites: direct-customer CRUD allowed by service registration, caller authorized for the target, neither AKS profile enabled, and no pre-existing Agent Sandbox extension/Helm release. No registration or authorization changes were made during this run.
Step Observed/asserted outcome
Fresh az aks extension create --extension-type Microsoft.AgentSandbox --config replicaCount=1 Succeeded
show and list Instance present; configuration replicaCount=1
update --config replicaCount=2 --yes, then show Succeeded; configuration persisted as 2
delete --yes, then list Instance absent
Final AKS inspection Both agentSandboxProfile and podSnapshotProfile remained null
Independent post-run inspection Generic extension list contained no Agent Sandbox instance; no Helm releases, deployments, or pods in agent-sandbox-system

Sanitized runner log excerpt (local live execution, not an ADO pipeline run; machine paths and unrelated Python deprecation warnings omitted):

test_agent_sandbox_extension_lifecycle (azext_aks_preview.tests.latest.test_agent_sandbox_extension_scenario.AgentSandboxExtensionScenarioTest.test_agent_sandbox_extension_lifecycle)
Run against an eligible, dedicated cluster with no Agent Sandbox installation.
...
ok

----------------------------------------------------------------------
Ran 1 test in 465.650s

OK

The registered cleanup repeats deletion after the scenario's explicit delete/list assertion, exercising idempotent deletion of an already-absent instance. Its "nothing to delete" warning is expected; it is not the evidence for successful primary deletion.

The first attempt, using aks-managed-agent-sandbox, was blocked, not validated:

(ValidationFailed) Creating extensions with 'aks-managed' in the name is restricted to authorized clients only.

The corrected customer-managed name passed the full lifecycle above. This validates installation/configuration/removal, not Sandbox workload execution, snapshot/restore E2E, protected-settings updates, upgrades, or long-term configuration durability.

Local checks for the lifecycle changes

  • 7 Agent Sandbox wrapper tests passed.
  • Live-only scenario skips when live execution is disabled.
  • azdev style aks-preview and repo-aware azdev linter aks-preview --min-severity medium passed.
  • Repository index checks passed (9 tests, 2 skipped). No generated-index edits.

Branch synchronization and CI

  • Rebased onto current main; no generated-index edits.
  • ADO build 366841 tested the refreshed merge containing the latest PR head and passed. Style and linter both executed and passed, confirmed from their logs.

Earlier direct-update validation - October 26, 2026

  • This PR's isolated CLI build successfully updated an existing directly managed extension in westcentralus using --config replicaCount=1 --yes.
  • ClusterConfig reported Succeeded; configuration persisted in the resource and Helm values; Helm revision advanced from 4 to 5.
  • Chart 1.0.4, extension identity, and namespace were preserved; the controller remained healthy at 1/1; both AKS profiles remained unset.
  • That earlier run exercised only an existing-instance update. Fresh creation and deletion are evidenced by the October 27 lifecycle above, not inferred from the earlier update.

General guidelines

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@microsoft-github-policy-service microsoft-github-policy-service Bot added the customer-reported Issues that are reported by GitHub users external to the Azure organization. label Oct 23, 2026
@microsoft-github-policy-service

Copy link
Copy Markdown
Contributor

Thank you for your contribution Alima Azamat (@alimaazamat)! We will review the pull request and get back to you soon.

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

@x-engineering-agent

Copy link
Copy Markdown

Hi Alima Azamat (@alimaazamat),
Please write the description of changes which can be perceived by customers into HISTORY.rst.
If you want to release a new extension version, please update the version in pyproject.toml (or setup.py, if the extension has not migrated yet) as well.

@alimaazamat
Alima Azamat (alimaazamat) marked this pull request as ready for review October 23, 2026 23:17
Copilot AI balanced review requested due to automatic review settings October 23, 2026 23:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The additive flags follow the existing decorator flow, preserve profile state, and include focused coverage with no identified blocking issues.

No open issues 🎉

What changed in this PR

Adds Agent Sandbox controls to aks-preview, using the existing managed-cluster create/update decorators and vendored SDK.

Changes:

  • Adds preview flags to enable Agent Sandbox during cluster creation and enable or disable it during updates.
  • Rejects conflicting flags and preserves existing profile fields and state when appropriate.
  • Adds focused tests, command help, usage guidance, and Pending release notes.
File Description
src/​aks-preview/​README.rst Documents usage, prerequisites, and service eligibility.
src/​aks-preview/​HISTORY.rst Adds Pending release notes for the new flags.
src/​aks-preview/​azext_aks_preview/​tests/​latest/​test_agent_sandbox_profile.py Tests registration, forwarding, serialization, profile preservation, and decorator pipelines.
src/​aks-preview/​azext_aks_preview/​managed_cluster_decorator.py Validates flags and applies the requested profile state.
src/​aks-preview/​azext_aks_preview/​custom.py Adds Agent Sandbox parameters to command handlers.
src/​aks-preview/​azext_aks_preview/​_params.py Registers the preview CLI flags.
src/​aks-preview/​azext_aks_preview/​_help.py Describes the flags, prerequisites, and mutual exclusion.

Review effort: Balanced

@alimaazamat
Alima Azamat (alimaazamat) marked this pull request as draft October 24, 2026 18:14
@alimaazamat Alima Azamat (alimaazamat) changed the title [aks-preview] Add Agent Sandbox managed profile commands [aks-preview] Add direct Agent Sandbox extension commands Oct 24, 2026
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

@alimaazamat
Alima Azamat (alimaazamat) marked this pull request as ready for review October 26, 2026 22:18
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

@FumingZhang FumingZhang left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you split the shared k8s-extension delete-error fix, its tests, and version/history changes into a separate PR for review by the corresponding owners?

For this AKS review, please keep this PR focused on the aks-preview Agent Sandbox integration and its wrapper changes. If the AKS behavior depends on the separately published fix, please document that dependency explicitly.

By the way, the ADO style/linter failures in build 366323 occur before either checker runs: the generated-index guard detects a change that is not present in the PR's merge-base diff. The stale synthetic merge is being compared against an advanced main containing an automated index update.

Please sync the branch with the latest main and trigger a new build against the refreshed merge commit, rather than only retrying the old build's failed jobs. Please do not edit the generated index to work around this failure.

Once rerun, please confirm that both ADO tasks actually execute and pass. If the comparison issue recurs, it should be addressed separately by the CI owners.

from azext_aks_preview._helpers import raise_validation_error_if_extension_type_not_in_allow_list


class AgentSandboxExtensionTestCase(unittest.TestCase):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These unit tests provide useful wrapper coverage, but the delegated extension implementation is mocked, so they do not exercise the actual command-to-service lifecycle.

Could you add or extend a scenario covering az aks extension create → show/list → configuration update → delete for Microsoft.AgentSandbox on an eligible cluster?

The existing-instance live update described in the PR is useful evidence, but fresh-create authorization and deletion remain unverified. Please provide a live run/log link with the tested SHA, CLI/extension versions, region, and relevant prerequisites. If service registration still blocks a path, please record it explicitly as blocked rather than validated.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

@alimaazamat

Alima Azamat (alimaazamat) commented Oct 27, 2026 •

Copy link
Copy Markdown
Author

Split out fix: #10528
This pr is fine to be merged before the fix.

Expose managed profile enablement on create/update and disablement on update, preserving existing profiles when flags are omitted.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3e9999ee-7df3-4e69-9868-14375740b6b3
Replace profile flags with the existing ClusterConfig extension wrapper, document configuration and ownership prerequisites, and propagate lifecycle errors.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3e9999ee-7df3-4e69-9868-14375740b6b3
Keep deletion idempotent for missing extensions while surfacing authorization, throttling, and service failures. Cover the real delete delegate with lookup-failure regression tests and prepare the 1.9.5 dependency release.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3e9999ee-7df3-4e69-9868-14375740b6b3
Move k8s-extension code, regression tests, and release changes to a dedicated PR for module-owner review.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3e9999ee-7df3-4e69-9868-14375740b6b3
@alimaazamat
Alima Azamat (alimaazamat) force-pushed the alimaazamat/aks-agent-sandbox-cli branch from b38c120 to 18edde4 Compare October 27, 2026 17:42
Exercise the actual direct extension client on an explicitly selected eligible cluster, including fresh create, show/list, replica update, delete, and cleanup. Fail service errors rather than substituting mock validation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3e9999ee-7df3-4e69-9868-14375740b6b3
Fresh lifecycle validation demonstrated that ClusterConfig rejects aks-managed names for direct customer creation. Use agent-sandbox in the scenario and command examples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3e9999ee-7df3-4e69-9868-14375740b6b3
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

@alimaazamat

Copy link
Copy Markdown
Author

FumingZhang I don't know how I have been retrying the old build's failed jobs, are you talking about the "/azp run" triggering the builds? I haven't retried anything but if there is a new /azp run will it trigger the updated refreshed commit?

@FumingZhang

Copy link
Copy Markdown
Member

FumingZhang I don't know how I have been retrying the old build's failed jobs, are you talking about the "/azp run" triggering the builds? I haven't retried anything but if there is a new /azp run will it trigger the updated refreshed commit?

You're right—pushes or an authorized /azp run trigger a new build. Sorry for implying you had retried old jobs.

CI is green, and the review feedback is addressed. LGTM from the AKS/aks-preview perspective.

FumingZhang
FumingZhang previously approved these changes Oct 28, 2026
@alimaazamat

Copy link
Copy Markdown
Author

@Azure/act-observability-squad Can you PTAL, this is Ignite critical

Keep feature notes under Pending for the December 1 release target.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3e9999ee-7df3-4e69-9868-14375740b6b3
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

@xuming-ms

Copy link
Copy Markdown

LGTM - just two small things before I approve and merge

  1. Please fix the  azdev style lint failure
  2. Please rebase/merge main once more, your branch picked up a point on main right before an unrelated networkfabric PR (Network Fabric CLI command and example updates - v10.0.0b4 #10538) landed, so the breaking-change bot is flagging  networkfabric bootstrapdevice commands as "removed" even though this PR never touches that extension.
    A fresh rebase with main should clear that stale note from the PR description.
    Once both are done I'll re-review and merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

act-observability-squad AKS Auto-Assign Auto assign by bot customer-reported Issues that are reported by GitHub users external to the Azure organization.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants