Skip to content

feat(copilot): SDK session options, skill tracking and image selection - #210

Merged
Kavya Sree Kaitepalli (KavyaSree2610) merged 4 commits into
mainfrom
bala/copilot-skills
Nov 2, 2026
Merged

Kavya Sree Kaitepalli (KavyaSree2610) merged 4 commits into
mainfrom
bala/copilot-skills

Conversation

@0xba1a

@0xba1a Bala (0xba1a) commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Summary

Let CopilotBot use a repository's agent skills and run on any sandbox image. The motivating case is the Azure Linux repository: it ships AGENTS.md and azldev-* skills under .agents/skills, and with this PR CopilotBot can follow them to upgrade a component end to end.

Changes

  • session_options: extra keyword arguments forwarded to the Copilot SDK's CopilotClient.create_session, for example skill_directories, disabled_skills, custom_agents, mcp_servers or working_directory.
    • Keys are checked against the installed SDK's create_session signature; unknown keys raise ValueError.
    • model, provider, streaming, hooks and system_message stay managed by CopilotBot and are rejected.
    • run(session_options=...) overrides them for one run without changing the bot's defaults.
    • Only option names are logged, never values, because values such as MCP server environment variables can hold secrets.
  • invoked_skills: after a run, lists the skills the agent used, taken from the SDK's skill.invoked events. Skill load and invoke events are also logged.
  • image: the container image of the sandbox CopilotBot creates, for example AZURELINUX_IMAGE. Passing both image and environment raises ValueError.
  • copilot-cli as a tool definition: the CLI and its Node.js 22 runtime are installed from the new tool_definitions/copilot-cli.yaml, using the normal distro-aware tool model (feat(tools): distro-aware tool commands #205) instead of install code inside CopilotBot.
    • Debian: NodeSource, as before.
    • Azure Linux: its repos only package Node 20, so it uses the official nodejs.org build, checked against the release's SHASUMS256.txt.
    • Pinned version: @github/copilot@1.0.105, verified with copilot --version. It must match the CLI that the pinned github-copilot-sdk targets; see below.
    • Distro handling: an undetectable distro falls back to the first (Debian) entry; a detected but unsupported distro raises UnsupportedDistroError before anything runs.
    • Infrastructure only: CopilotBot installs this tool itself. It is never an additional_tool and adds nothing to the model's prompt.
    • Unchanged: starting the headless server, passing the GitHub token and the readiness check stay in CopilotBot. The host still gets the SDK from the microbots[ghcp] pip extra.
  • Copilot SDK 1.0.28 and copilot-cli 1.0.105: upgraded from 0.3.0 and 1.0.39, the latest compatible pair.
    • Why both: each SDK 1.x release pins the CLI it targets (CLI_VERSION in copilot/_cli_version.py). The old and new halves don't mix: SDK 0.3.0 can't parse CLI 1.0.105 responses, and CLI 1.0.39 lacks the connect RPC that SDK 1.x calls.
    • Code: connects with CopilotClient(connection=UriRuntimeConnection(...)), which replaces ExternalServerConfig, and imports SessionEventType from the public copilot.session_events. An SDK older than 1.x now fails with an explicit upgrade message, and the copilot.types fallback for SDKs before 0.3.0 is gone.
    • Pins: pyproject.toml (ghcp extra) and copilot-cli.yaml each carry a comment to update the other. The ghcp CI job installs the ghcp extra instead of repeating the version.
    • Check: test/bot/test_copilot_version_pins.py verifies both pins are exact. Under the ghcp marker, it also verifies the installed SDK matches the pin and targets the pinned CLI version.
  • model_id for BYOK deployments with custom names: new parameter and COPILOT_BYOK_MODEL_ID env var, mapped to the SDK's provider model_id; the deployment name stays the wire model.
    • Why: copilot-cli 1.0.105 picks request settings such as temperature and reasoning_effort from the model name. For CI's Azure deployment mini-swe-agent-gpt5, it fell back to generic defaults and sent temperature: 0, which GPT-5 rejects.
    • CI: the ghcp job now uses the Responses wire API and COPILOT_BYOK_MODEL_ID=gpt-5.
    • Verified: reproduced the failure locally against the same deployment. wire_api="responses" alone fixes it, and so does model_id alone. With CI's settings, all ghcp tests pass.
  • Docs: CopilotBot guide sections on SDK session options and on using a repository's agent skills, with an Azure Linux example.

Testing

  • Unit:
    • Session options: validation (reserved keys, unknown keys, non-mapping input), forwarding, per-run overrides, and option values kept out of the logs.
    • Skills: invoked_skills tracking, reset on each run.
    • Image and copilot-cli: image forwarding and rejection together with environment; copilot-cli.yaml resolution per distro (Node install, pinned CLI, no usage instructions); _install_copilot_cli driving the tool, and rejecting unsupported distros.
    • SDK 1.x: import and connection tests (UriRuntimeConnection to the container IP and CLI port; a pre-1.x SDK is rejected), a full run() through the 1.x session API, and the version-pin tests.
    • model_id: provider config in each BYOK path (explicit credentials, env vars, token provider) and through the constructor; unset by default.
    • Results: no new failures compared to main, including under -n auto as CI runs them. The session-options test that failed occasionally before was depending on COPILOT_BYOK_MODEL in the environment; it no longer does.
  • Integration: CI's integration selection, run locally: 25 passed, with no new failures compared to main.
  • Live: installing through copilot-cli.yaml works on both stock images (DEBIAN_IMAGE, AZURELINUX_IMAGE) with Node 22. With SDK 1.0.28 and CLI 1.0.105, the live ghcp CopilotBot task on Debian passes.
  • End to end: the new test/bot/test_copilot_bot_azurelinux_skills.py (marked integration, slow, ghcp).
    • Setup: it mounts an Azure Linux checkout READ_ONLY on AZURELINUX_IMAGE with sudo_access=True, installs rpm/mock/azldev (feat(tools): rpm, mock and azldev tool definitions #209), and points the session at the checkout's .agents/skills.
    • Task: it asks the agent to upgrade util-linux.
    • Result with BYOK gpt-5.6-sol: passed with SDK 1.0.28 and CLI 1.0.105 in about 12 minutes (and before the upgrade, with 0.3.0 and 1.0.39). The agent used the azldev, azldev-comp-toml and azldev-update-component skills, upgraded util-linux from 2.41.5 to 2.42.4 inside the sandbox, never ran sudo, and left the host checkout unchanged.
    • Wire API: with CLI 1.0.105, gpt-5.6-sol needs COPILOT_BYOK_WIRE_API=responses. Over the default completions API, the CLI now sends reasoning_effort together with function tools, and the endpoint rejects that combination.
    • Keeping changes: set AZURELINUX_KEEP_CHANGES=1 to mount the checkout READ_WRITE and keep the agent's changes.

Stack

  1. refactor(env): run stock MCR images, prepare the sandbox on the fly #202 (merged): stock MCR images prepared on the fly.
  2. feat(tools): distro-aware tool commands #205 (merged): distro-aware tool commands.
  3. feat(env): Azure Linux 3 sandbox image #207 (merged): Azure Linux 3 sandbox and image on the bots.
  4. feat(env): optional passwordless sudo for the sandbox agent #208 (merged): optional passwordless sudo for the sandbox agent.
  5. feat(tools): rpm, mock and azldev tool definitions #209 (merged): rpm / mock / azldev tool definitions.
  6. This PR: CopilotBot session options, skill tracking and image selection.

Base automatically changed from bala/azl-build-tools to main November 2, 2026 09:49
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Nov 2, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.48%. Comparing base (3f05bc3) to head (f6b8a99).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #210      +/-   ##
==========================================
+ Coverage   77.22%   77.48%   +0.25%     
==========================================
  Files          47       47              
  Lines        3219     3256      +37     
==========================================
+ Hits         2486     2523      +37     
  Misses        733      733              
Flag Coverage Δ
ghcp 33.53% <68.42%> (+1.66%) ⬆️
integration 42.22% <15.78%> (-0.27%) ⬇️
ollama_local 34.36% <15.78%> (-0.18%) ⬇️
slow-browser 18.36% <15.78%> (+<0.01%) ⬆️
slow-other 40.17% <15.78%> (-0.03%) ⬇️
unit 75.12% <100.00%> (+0.28%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/microbots/bot/CopilotBot.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Bala (0xba1a) and others added 2 commits November 2, 2026 11:01
Let CopilotBot use a repository's agent skills and run on any sandbox
image:

- session_options: extra kwargs forwarded to CopilotClient.create_session
  (skill_directories, custom_agents, mcp_servers, working_directory, ...).
  Keys are validated against the installed SDK; model, provider,
  streaming, hooks and system_message stay managed by CopilotBot. run()
  accepts per-run overrides. Only option names are logged.
- invoked_skills: names of skills used in the last run (skill.invoked
  events); skill load/invoke events are logged.
- image: container image of the auto-created environment.
  copilot-cli's Node.js 22 is installed for the distro read from
  /etc/os-release (NodeSource on Debian, checksum-verified nodejs.org
  build on Azure Linux, which only packages Node 20; Debian's steps when
  the distro is unknown; unsupported distros fail before running).
- Integration test: the agent upgrades util-linux in an Azure Linux
  checkout using its AGENTS.md and azldev-* skills (READ_ONLY by default,
  AZURELINUX_KEEP_CHANGES=1 to keep the changes).
- Docs: session options and Azure Linux skills example.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

# Conflicts:
#	docs/advanced/bots/copilot-bot.md
#	src/microbots/bot/CopilotBot.py
#	test/bot/test_copilot_bot.py
Review follow-up: CopilotBot had its own per-distro Node.js/copilot-cli
install table and matching logic, duplicating the distro-aware tool
command resolution (#205). Move it into the normal tool model:

- New tool_definitions/copilot-cli.yaml: Node.js 22 per distro
  (NodeSource on Debian; checksum-verified nodejs.org build on Azure
  Linux, which only packages Node 20) and the pinned
  @github/copilot@1.0.39 CLI, verified with `copilot --version`. It has no
  usage instructions, since it is infrastructure rather than a tool for
  the model.
- CopilotBot._install_copilot_cli() installs and verifies that tool;
  _NODE_INSTALL_COMMANDS, _COPILOT_CLI_PACKAGE and _node_install_commands
  are removed. Unsupported distros now raise UnsupportedDistroError, and
  an undetectable distro falls back to the first (Debian) entry.
- The headless server start, token injection and readiness check stay
  in CopilotBot; the host-side SDK stays the `ghcp` pip extra.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Bala (0xba1a) and others added 2 commits November 2, 2026 11:20
…to 1.0.105

github-copilot-sdk 1.x pins the copilot-cli release it targets
(CLI_VERSION in copilot/_cli_version.py); 1.0.28 targets 1.0.105. The
0.3.0 / 1.0.39 pair is incompatible with either newer half (0.3.0 cannot
parse 1.0.105 responses; 1.0.39 lacks the 1.x `connect` RPC).

- Connect with CopilotClient(connection=UriRuntimeConnection(...)), which
  replaces ExternalServerConfig.
- Import PermissionHandler from copilot.session only and SessionEventType
  from the public copilot.session_events module; a pre-1.x SDK now fails
  with an explicit upgrade message.
- Cross-reference the two pins in pyproject.toml and copilot-cli.yaml, let
  the ghcp CI job install the `ghcp` extra instead of repeating the pin, and
  add test_copilot_version_pins.py to check the pins (and, under `ghcp`,
  that the installed SDK targets the pinned CLI).
- Replace the pre-0.3.0 SDK-layout tests with 1.x import/connection tests.
- Make the session-options test independent of COPILOT_BYOK_MODEL.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
copilot-cli 1.0.105 picks request settings (temperature, reasoning_effort,
prompts, token limits) from the model name. For a BYOK deployment with a
custom name, such as CI's Azure `mini-swe-agent-gpt5`, it falls back to
generic defaults and sends temperature 0, which GPT-5 rejects.

- New `model_id` parameter and COPILOT_BYOK_MODEL_ID env var, mapped to the
  SDK's provider `model_id`; the deployment name is kept as `wire_model`.
- ghcp CI job: use the Responses wire API and COPILOT_BYOK_MODEL_ID=gpt-5.
- Docs and unit tests for the new option.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@KavyaSree2610
Kavya Sree Kaitepalli (KavyaSree2610) merged commit ed44547 into main Nov 2, 2026
10 checks passed
@KavyaSree2610
Kavya Sree Kaitepalli (KavyaSree2610) deleted the bala/copilot-skills branch November 2, 2026 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants