Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions docs/advanced/sandbox-images.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,33 @@ env = LocalDockerEnvironment(port=8080, image="mcr.microsoft.com/devcontainers/b

You never declare the sandbox's distribution. Tools read it from the running container's `/etc/os-release` and pick their [distro-specific commands](tools/tesseract_ocr_tool_use.md#distro-specific-commands) from it. A tool that only defines `debian` commands, such as `cscope` or `browser-use`, fails to install on the Azure Linux image with `UnsupportedDistroError`.

## Azure Linux build tools

Microbots ships tool definitions for Azure Linux and RPM work:

| Tool | Provides | Distros |
|------|----------|---------|
| `rpm.yaml` | `rpm`, `rpmbuild`, `rpmspec`, `rpm2cpio`, `createrepo_c` | Azure Linux, Debian |
| `mock.yaml` | `mock` with the `rpmautospec` plugin. Needs `sudo_access=True`. | Azure Linux |
| `azldev.yaml` | [Azure Linux Dev Tools](https://github.com/microsoft/azure-linux-dev-tools). Uses `mock` for render and build. | Azure Linux, Debian |

```python
from microbots import WritingBot
from microbots.environment.local_docker.images import AZURELINUX_IMAGE
from microbots.tools.tool_yaml_parser import parse_tool_definition

bot = WritingBot(
model="azure-openai/your-deployment",
folder_to_mount="/path/to/azurelinux",
image=AZURELINUX_IMAGE,
sudo_access=True,
additional_tools=[parse_tool_definition(n) for n in ("rpm.yaml", "mock.yaml", "azldev.yaml")],
)
```

- `mock.yaml` adds the agent to the `mock` group, so the agent runs `mock` (and `azldev`, which drives it) as itself. `mock`'s setuid launcher does the privileged work, which is why it needs `sudo_access=True`. Files that `azldev` and `mock` write into the repository stay owned by the host user.
- `azldev.yaml` installs the `azldev` version pinned in the mounted repository's `.azldev-version` file, or the latest release if there is no pin.

## Sudo access

By default the sandbox shell drops to an unprivileged `agent` account that has your host user's uid, and the container blocks privilege escalation (`no-new-privileges`). Some tools, such as `mock`, need root for part of their work. Pass `sudo_access=True` to any bot or to `LocalDockerEnvironment` to give the agent passwordless `sudo`:
Expand Down
51 changes: 51 additions & 0 deletions src/microbots/tools/tool_definitions/azldev.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: azldev
tool_type: internal
description: Azure Linux Dev Tools (azldev) — manages Azure Linux components, locks, rendered specs and mock builds.

# azldev drives mock for render/build, so also install the `mock` and `rpm` tool
# definitions and create the bot with image=AZURELINUX_IMAGE (images module) and sudo_access=True.
usage_instructions_to_llm: |
# `azldev` (Azure Linux Dev Tools) is installed, together with mock and rpm.
# Run it from the Azure Linux repo root (where azldev.toml lives) or pass
# `-C <repo-root>`. Follow the repository's AGENTS.md and azldev-* skills;
# they describe the supported workflows. Run azldev as yourself, never with sudo.
#
# Agent-friendly global flags: -q (quiet) and -O json (machine-readable output).
# Common commands:
# azldev comp list -p <name> -q -O json # inspect a component's config
# azldev comp update -p <name> # refresh locks/<name>.lock
# azldev comp render -p <name> # re-render specs/<c>/<name>/
# azldev comp build -p <name> # build RPMs with mock (slow)
# azldev adv mock shell -p <rpm> <<'CMDS' # smoke-test inside a mock chroot
# ...commands...
# CMDS
# Use `azldev <command> --help` for current syntax; the tool evolves quickly.
# render/build start mock chroots and download from the network; they may take minutes.
# End of instructions.

install_commands:
azurelinux:
- tdnf install -y golang git ca-certificates
- &install_azldev |
set -e
# Prefer the azldev version pinned by a mounted Azure Linux repo.
version=""
for pin in /workdir/*/.azldev-version; do
if [ -f "$pin" ]; then version="$(tr -d '[:space:]' < "$pin")"; break; fi
done
# MS_GOTOOLCHAIN_ALLOW_NON_LOCAL lets Microsoft Go (Azure Linux) fetch a newer
# toolchain like upstream Go does, if azldev ever needs one.
export GOTOOLCHAIN=auto MS_GOTOOLCHAIN_ALLOW_NON_LOCAL=1
export GOPATH=/opt/go GOBIN=/usr/local/bin GOFLAGS=-buildvcs=false
go install "github.com/microsoft/azure-linux-dev-tools/cmd/azldev@${version:-latest}"
debian:
- rm -f /etc/apt/sources.list.d/yarn.list
- apt-get update -qq
- DEBIAN_FRONTEND=noninteractive apt-get install -y -qq golang-go git ca-certificates > /dev/null
- *install_azldev

verify_commands:
- azldev --version

uninstall_commands:
- rm -f /usr/local/bin/azldev
47 changes: 47 additions & 0 deletions src/microbots/tools/tool_definitions/mock.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: mock
tool_type: internal
description: Fedora's mock — builds RPMs and runs commands inside clean, distro-specific chroots.

# mock's launcher (consolehelper/userhelper) is setuid, and the agent runs mock as
# itself through its `mock` group membership, so the sandbox must allow privilege
# escalation: create the bot/environment with image=AZURELINUX_IMAGE (images module)
# and sudo_access=True.
usage_instructions_to_llm: |
# `mock` is installed. It builds SRPMs/RPMs and runs commands inside clean chroots
# described by a mock config (.cfg). Run it as yourself (you are in the `mock`
# group); do not use sudo, or the files it creates will be owned by root.
#
# Always pass the project's config explicitly:
# mock -r <path/to/config.cfg> --configdir <dir-of-config> <action>
# Common actions:
# --init create/refresh the chroot
# --rebuild <file.src.rpm> build binary RPMs (add --resultdir <dir>)
# --install <pkg|file.rpm> ... install packages into the chroot
# --copyin <src> <dest> copy a file into the chroot
# --chroot '<command>' run a command inside the chroot (non-interactive)
# --scrub=chroot | --scrub=all reset the chroot (all also drops the bootstrap)
# Add --enable-network when the chroot needs network access.
#
# Never start an interactive `mock --shell` without a heredoc; there is no TTY.
# Use: mock -r <cfg> --shell <<'CMDS' ... CMDS
# Builds can take many minutes; run them in the foreground and wait for them.
# End of instructions.

# Only Azure Linux packages mock; other distros fail to install this tool.
install_commands:
azurelinux:
- tdnf install -y mock mock-rpmautospec
# mock and azldev check the group database, so this applies to the running shell too.
- if id agent > /dev/null 2>&1; then usermod -aG mock agent; fi

verify_commands:
# `false` rather than `exit`: verify commands share the bot's persistent shell.
- >-
if grep -q '^NoNewPrivs:[[:space:]]*1' /proc/self/status; then
echo "mock needs sudo_access=True (its setuid launcher is blocked by no-new-privileges)" >&2;
false; fi
- mock --version

uninstall_commands:
azurelinux:
- tdnf remove -y mock mock-rpmautospec
41 changes: 41 additions & 0 deletions src/microbots/tools/tool_definitions/rpm.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: rpm
tool_type: internal
description: RPM package tooling (rpm, rpmbuild, rpmspec, rpm2cpio, createrepo_c) for inspecting and building RPM packages.

usage_instructions_to_llm: |
# RPM tooling is installed: rpm, rpmbuild, rpmspec, rpm2cpio, cpio and createrepo_c.
# Use them to inspect packages and spec files. Build and smoke-test packages
# inside a mock chroot instead of on the sandbox itself.
#
# Examples:
# rpm -qpi <file.rpm> # package metadata
# rpm -qlp <file.rpm> # files in a package
# rpm -qp --requires <file.rpm> # runtime dependencies
# rpm2cpio <file.rpm> | cpio -idmv # extract a package into the current directory
# rpmspec -q --srpm --qf '%{VERSION}-%{RELEASE}\n' <file.spec>
# createrepo_c <dir-with-rpms> # build a local repo
#
# Never install a built RPM on the sandbox itself with `rpm -i`.
# End of instructions.

install_commands:
azurelinux:
- tdnf install -y rpm rpm-build createrepo_c cpio
debian:
# Stale third-party apt sources with expired keys would break apt-get update.
- rm -f /etc/apt/sources.list.d/yarn.list
- apt-get update -qq
- DEBIAN_FRONTEND=noninteractive apt-get install -y -qq rpm rpm2cpio cpio python3-rpm createrepo-c > /dev/null

verify_commands:
- rpm --version
- rpmbuild --version
- rpmspec --version
- createrepo_c --version

uninstall_commands:
azurelinux:
- tdnf remove -y rpm-build createrepo_c
debian:
- apt-get remove -y rpm rpm2cpio python3-rpm createrepo-c
- apt-get autoremove -y
36 changes: 36 additions & 0 deletions test/tools/test_azurelinux_build_tools.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
"""Integration test: the Azure Linux build tools install in an Azure Linux sandbox with sudo access.

Requires Docker and network access (the stock AZURELINUX_IMAGE is prepared on
the fly, then the tools download packages and build azldev), so it is marked
slow.
"""

import os
import sys

import pytest

sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../src")))

from microbots.environment.local_docker.LocalDockerEnvironment import LocalDockerEnvironment
from microbots.environment.local_docker.images import AZURELINUX_IMAGE
from microbots.tools.tool_yaml_parser import parse_tool_definition
from microbots.utils.network import get_free_port


@pytest.mark.integration
@pytest.mark.docker
@pytest.mark.slow
def test_rpm_mock_and_azldev_install_on_azurelinux():
env = LocalDockerEnvironment(port=get_free_port(), image=AZURELINUX_IMAGE, sudo_access=True)
try:
for name in ("rpm.yaml", "mock.yaml", "azldev.yaml"):
tool = parse_tool_definition(name)
tool.install_tool(env)
tool.verify_tool_installation(env)

ret = env.execute("azldev --version && mock --version && rpmbuild --version")
assert ret.return_code == 0, ret.stderr
assert "azldev version" in ret.stdout
finally:
env.stop()
44 changes: 44 additions & 0 deletions test/tools/test_tool_yaml_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,50 @@ def test_parse_browser_use_yaml(self):
assert tool.name == "browser-use"
assert tool.tool_type == TOOLTYPE.INTERNAL

@pytest.mark.parametrize(
"yaml_name, verify_binary",
[("rpm.yaml", "rpm"), ("mock.yaml", "mock"), ("azldev.yaml", "azldev")],
)
def test_parse_azure_linux_build_tool_yamls(self, yaml_name, verify_binary):
"""The Azure Linux build tools parse and support Azure Linux."""
tool = parse_tool_definition(yaml_name)
assert isinstance(tool, Tool)
assert tool.name == yaml_name.removesuffix(".yaml")
assert tool.usage_instructions_to_llm.strip()
install = tool.resolve_commands("install", ["azurelinux"])
verify = tool.resolve_commands("verify", ["azurelinux"])
assert install and all(isinstance(c, str) and c.strip() for c in install + verify)
assert install[0].startswith("tdnf install -y")
assert any(c.startswith(f"{verify_binary} --version") for c in verify)

def test_mock_yaml_is_azure_linux_only_and_runs_through_mock_group(self):
from microbots.tools.tool import UnsupportedDistroError

tool = parse_tool_definition("mock.yaml")
with pytest.raises(UnsupportedDistroError):
tool.resolve_commands("install", ["debian"])
install = tool.resolve_commands("install", ["azurelinux"])
assert install[0] == "tdnf install -y mock mock-rpmautospec"
assert "usermod -aG mock agent" in install[1]
# No root wrapper: the agent runs mock itself through the mock group.
assert not any("/usr/bin/mock" in c for c in install)
verify = tool.resolve_commands("verify", ["azurelinux"])
assert "NoNewPrivs" in verify[0] and "sudo_access=True" in verify[0]
# Verify commands share the bot's persistent shell, so they must never `exit`.
assert "exit" not in verify[0]

def test_azldev_yaml_shares_install_steps_across_distros(self):
tool = parse_tool_definition("azldev.yaml")
azl = tool.resolve_commands("install", ["azurelinux"])
deb = tool.resolve_commands("install", ["ubuntu", "debian"])
assert azl[-1] == deb[-1]
install = azl[-1]
assert "/workdir/*/.azldev-version" in install
assert "azldev@${version:-latest}" in install
assert "GOBIN=/usr/local/bin" in install
# azldev runs as the agent, so it needs no root override.
assert not any("AZLDEV_ALLOW_ROOT" in c for c in azl + deb)

def test_parse_absolute_path(self, tmp_path):
"""Test parsing with absolute path works correctly."""
yaml_content = """
Expand Down
Loading