Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .github/assets/logo/logo-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/assets/logo/logo-light.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 0 additions & 18 deletions .github/brand/brand.lock.json

This file was deleted.

Binary file removed .github/brand/social.png
Binary file not shown.
19 changes: 12 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
name: CI
on:
pull_request:
types: [opened, synchronize, reopened]
push:
branches: [master]
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ci-${{ github.ref }}
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
package:
Expand All @@ -22,8 +23,8 @@ jobs:
python-version: '3.13'
- name: Verify deterministic declaration package
run: |
python3 -m unittest discover -s Tests -p 'test_*.py'
python3 Scripts/build_package.py dist > package-receipt.json
python3 -m unittest discover -s tests -p 'test_*.py'
python3 scripts/build_package.py dist > package-receipt.json
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: computer-use-plugin-${{ github.sha }}
Expand All @@ -44,7 +45,11 @@ jobs:
python-version: '3.11'
- name: Verify the oldest supported build interpreter
run: |
python3 -m unittest discover -s Tests -p 'test_*.py'
python3 Scripts/build_package.py dist
brand:
uses: computer-mcp/.github/.github/workflows/brand-check.yml@f402911abcdfb87ef0c41a2759e8c51387b18935 # master
python3 -m unittest discover -s tests -p 'test_*.py'
python3 scripts/build_package.py dist
validate:
if: always()
needs: [package, minimum-python]
uses: computer-mcp/github-workflows/.github/workflows/result.yml@d65bc20504f39c3a30232d67043abeca43d2c7b1
with:
needs-json: ${{ toJSON(needs) }}
2 changes: 1 addition & 1 deletion .github/workflows/notify-catalog.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
steps:
- name: Request complete catalog reconciliation
id: catalog
uses: computer-mcp/computer-mcp.github.io/.github/actions/notify-catalog@cda692839487f03a2b4740d945f9a8bd818554e5
uses: computer-mcp/github-workflows/actions/notify-catalog@d65bc20504f39c3a30232d67043abeca43d2c7b1
with:
client-id: ${{ vars.CATALOG_APP_CLIENT_ID }}
private-key: ${{ secrets.CATALOG_APP_PRIVATE_KEY }}
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,11 @@ jobs:
permissions:
contents: write
actions: read
uses: computer-mcp/.github/.github/workflows/plugin-release.yml@35766b350427ed900b456aa3c39098e97d9d3aa1 # master
uses: computer-mcp/github-workflows/.github/workflows/plugin-release.yml@d65bc20504f39c3a30232d67043abeca43d2c7b1
with:
ref: ${{ inputs.ref }}
publish: ${{ inputs.publish }}
build: Scripts/build-package.sh
build: scripts/build-package.sh
ci_artifact: computer-use-plugin-${commit}
notify-catalog:
needs: release
Expand Down
17 changes: 17 additions & 0 deletions .github/workflows/source-policy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Source policy checks
on:
pull_request:
types: [opened, synchronize, reopened, edited]
push:
branches: [master]
merge_group:
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
policy:
uses: computer-mcp/github-workflows/.github/workflows/policy.yml@d65bc20504f39c3a30232d67043abeca43d2c7b1
with:
declaration-revision: 5c54a40cb7953db7472ebbefd3db608036b7b3ab
38 changes: 14 additions & 24 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,17 +70,17 @@ Use this check:

- Before changing versions, dependencies, packaging or release workflows, read
the [organization VERSIONING](https://github.com/computer-mcp/.github/blob/master/VERSIONING.md)
for version rules and `Documentation/Reference/Release.md` for release facts
for version rules and `docs/Reference/Release.md` for release facts
and check entry points.

- For repository-native documentation placement, read `Documentation/README.md`
- For repository-native documentation placement, read `docs/README.md`
before editing.
- For current canonical structure, read
`Documentation/Architecture/README.md` and the relevant architecture files.
- For design-in-progress, use `Documentation/Proposals/*` when that subtree is
`docs/Architecture/README.md` and the relevant architecture files.
- For design-in-progress, use `docs/Proposals/*` when that subtree is
present.
- For change history, consult `Documentation/Decisions/*`,
`Documentation/Migrations/*`, and `Documentation/Archive/*` when those
- For change history, consult `docs/Decisions/*`,
`docs/Migrations/*`, and `docs/Archive/*` when those
subtrees are present.
- For GitHub-facing collaboration files, use `.github/` and root governance
files.
Expand All @@ -92,10 +92,10 @@ Use this check:
- `AGENTS.md` is the agent guide: first-principles guardrails, task
route, authority boundaries, and boundary guardrails.
- `README`-class files index scope and placement.
- `Documentation/Architecture/*` is current truth.
- `Documentation/Proposals/*` is proposal space when that subtree is present.
- `Documentation/Decisions/*`, `Documentation/Migrations/*`, and
`Documentation/Archive/*` are history when those subtrees are present.
- `docs/Architecture/*` is current truth.
- `docs/Proposals/*` is proposal space when that subtree is present.
- `docs/Decisions/*`, `docs/Migrations/*`, and
`docs/Archive/*` are history when those subtrees are present.
- `.github/*` is GitHub-facing governance.

## Boundary Guardrails
Expand All @@ -113,8 +113,8 @@ and operating guidance. Keep temporary implementation notes, local evidence,
local paths, run-specific artifacts, and historical comparison notes out of
README files, Reference docs, API docs, and bundled user-facing skills. Promote
only accepted decision records into
`Documentation/Decisions/*` and durable transition or cutover records into
`Documentation/Migrations/*`.
`docs/Decisions/*` and durable transition or cutover records into
`docs/Migrations/*`.

Use this decision test:

Expand Down Expand Up @@ -147,7 +147,7 @@ source of truth instead of a literal value.
- Keep Agent Guide and Index separate.
- Keep current truth out of proposal and history subtrees when they are
present.
- Keep GitHub collaboration configuration out of `Documentation/`.
- Keep GitHub collaboration configuration out of `docs/`.

## Repository Guardrails

Expand All @@ -161,19 +161,9 @@ Read `README.md` and `computer-mcp-plugin.toml` before changing this package.
defaults and skill text cannot grant access.
- Keep `skills/observe-act-verify/SKILL.md` portable and independent of a
particular user's paths, targets, credentials or tool catalog snapshot.
- Validate the package with `Scripts/build-package.sh OUTPUT_DIRECTORY`, then
- Validate the package with `scripts/build-package.sh OUTPUT_DIRECTORY`, then
inspect it using the host's package validation and installation tests.
- Keep test evidence outside the repository; do not publish repositories or
artifacts without explicit authorization.
- Preserve native AX fallback and never replay an uncertain write through a
second backend. Real GUI verification requires a disposable, authorized target.

## Brand delivery

The organization `.github` repository owns the design system and renders every
image; the main Computer MCP repository owns ProductIdentity. Imported
`Documentation/Brand/header-light.png` and `header-dark.png` travel with the
packaged manual; `.github/brand/social.png` supplies the repository preview.
`.github/brand/brand.lock.json` binds them to the organization exports, and CI
verifies it with the shared brand-check workflow. Update imports with the
organization repository's `python3 Brand/brand.py sync`.
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ external dependencies and credentials.

## Validation

CI runs `python3 -m unittest discover -s Tests -p 'test_*.py'` and `python3 Scripts/build_package.py OUTPUT_DIRECTORY` on Python 3.13 and 3.11, and the organization brand check; run the first two before opening a pull request. Build an archive with `Scripts/build-package.sh OUTPUT_DIRECTORY` and inspect its file list. GUI checks use disposable, explicitly authorized targets.
CI runs `python3 -m unittest discover -s tests -p 'test_*.py'` and `python3 scripts/build_package.py OUTPUT_DIRECTORY` on Python 3.13 and 3.11; run both before opening a pull request. Build an archive with `scripts/build-package.sh OUTPUT_DIRECTORY` and inspect its file list. GUI checks use disposable, explicitly authorized targets.

## Documentation

The [documentation index](Documentation/README.md) links current architecture
The [documentation index](docs/README.md) links current architecture
and reference material. Agent work routes live in AGENTS.md; public usage lives
in the root README. GitHub collaboration files belong in .github/ and contributor
policy belongs in root governance files.
Expand Down
Binary file removed Documentation/Brand/header-dark.png
Binary file not shown.
Binary file removed Documentation/Brand/header-light.png
Binary file not shown.
150 changes: 92 additions & 58 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,68 +1,102 @@
<picture>
<source media="(prefers-color-scheme: dark)" srcset="Documentation/Brand/header-dark.png">
<img alt="Computer MCP — Computer Use" src="Documentation/Brand/header-light.png">
</picture>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset=".github/assets/logo/logo-dark.png">
<img src=".github/assets/logo/logo-light.png" width="128" height="128" alt="">
</picture>
</p>

# Computer MCP — Computer Use
<h1 align="center">Computer Use</h1>

Part of the [Computer MCP](https://computer-mcp.github.io/) family.
**Wherever you chat, your computer is there.**
<p align="center">Computer MCP family plugin for native desktop observation and actions.</p>

A declarative Computer MCP package that connects to the vendor's native
Computer Use MCP server and supplies observation/action verification guidance.
It contains no proxy, vendor executable, installer or running skill service.
<p align="center">
<a href="https://github.com/computer-mcp/plugin-computer-use/actions/workflows/ci.yml"><img alt="CI" src="https://img.shields.io/github/actions/workflow/status/computer-mcp/plugin-computer-use/ci.yml?branch=master&label=CI&style=flat"></a>
<a href=".github/workflows/ci.yml"><img alt="Python 3.11 or newer to build" src="https://img.shields.io/badge/build-Python%203.11%2B-blue?style=flat"></a>
<a href="LICENSE"><img alt="License: FSL-1.1-ALv2" src="https://img.shields.io/badge/license-FSL--1.1--ALv2-blue?style=flat"></a>
</p>

<p align="center">
<a href="#install">Install</a> ·
<a href="#usage">Usage</a> ·
<a href="skills/observe-act-verify/SKILL.md">Skill</a> ·
<a href="docs/README.md">Documentation</a> ·
<a href="CONTRIBUTING.md">Contributing</a>
</p>

## Overview

A declarative [Computer MCP](https://computer-mcp.github.io/) package that connects the host to
the vendor's native Computer Use MCP server and adds guidance for verifying every observation and
action. It contains no proxy, vendor executable, installer or running skill service. The vendor
owns installation, updates and caller compatibility.

## Quick look

The plugin contributes:

| Contribution | What it adds |
| --- | --- |
| `native` MCP server | The vendor client's own tools, started as `SkyComputerUseClient mcp` and exposed under the `computer-use` prefix |
| `observe-act-verify` skill | How to complete an authorized GUI task from current observations, scoped actions and post-action checks, and how to recover from stale references or uncertain results |

Install the archive in the host, review the `native` contribution, choose which tools to expose,
and enable it.

## Install

1. Build the archive: `scripts/build-package.sh OUTPUT_DIRECTORY` writes `computer-use.zip`.
2. Install it through the host's Plugins page or management CLI with its declared ID and
version and the SHA-256 the build prints. A local development registration can point at this
checkout instead.
3. The host starts new plugins disabled. Review the `native` MCP contribution, choose its tool
whitelist or all-tools exposure explicitly, and keep the appropriate caller and profile
permissions. Enabling the package does not override those permissions.

The host finds the client on its search path, then by the installed application's bundle
identifier, `com.openai.sky.CUAService`. When neither finds it, bind the actual executable in the
plugin's dependency settings; an explicit binding takes precedence.

Host state lives outside this package. Uninstalling a version does not delete the vendor client,
this development checkout or saved host grants.

## Usage

A successful MCP handshake or tools listing does not prove that observation or actions work.
Before relying on the plugin, choose a disposable, authorized target: read its state, perform one
harmless action, then read and verify the result. Diagnose permission or caller-trust failures
through the vendor's and the system's supported flows.

Native accessibility (AX) remains available for diagnostics and fallback where appropriate. An
uncertain write is observed before any further action; it is never replayed automatically
through AX.

## Requirements

Use a Computer MCP host with plugin support and a vendor-supported Computer Use
client. The client must provide its `mcp` entry point. The host checks its search
path, then locates the declared installed application by bundle identifier.
Supply the actual executable through the host's dependency binding when the
vendor installation is not discoverable by either method. An explicit binding
takes precedence. The vendor owns installation, updates and caller compatibility.

The host starts new plugins disabled. Review the `native` MCP contribution,
explicitly choose its tool whitelist or all-tools exposure, and retain the
appropriate caller/profile permissions. Enabling the package does not override
those permissions. A successful MCP handshake or tools listing does not prove
that observation or actions work.

For first-use verification, choose a disposable, authorized target: read its
state, perform one harmless action, then read and verify the result. Diagnose
permission or caller-trust failures through supported vendor/system flows.
Native AX remains available for diagnostics and appropriate fallback. An
uncertain write must be observed before any further action, never automatically
replayed through AX.

## Package

Run `Scripts/build-package.sh OUTPUT_DIRECTORY` to produce `computer-use.zip`.
The archive contains this manual, contributor and architecture documentation,
the license and third-party notices, the manifest and the skill directory. Build
output excludes Git metadata, local test evidence and external executables.
The build requires Python 3.11 or later's standard library; it installs nothing and
produces deterministic archive bytes from the same inputs.

`.github/workflows/ci.yml` runs the package tests and retains the ZIP and
SHA-256 receipt on pull requests, `master` pushes and manual runs, and repeats
the tests and build on Python 3.11. This declaration-only
job runs independently of macOS and never starts the vendor client. Its workflow
artifacts are not a public plugin release or proof of GUI compatibility.

Install the archive through the host's Plugins page or management CLI using its
declared ID/version and the SHA-256 emitted by the build. A local development
registration can point at this checkout instead. Local ownership or a bundled
source label is not a verified GitHub publisher or artifact signature.

Host state belongs outside this package. Uninstalling an installed version must
not delete the vendor client, this development checkout, or saved host grants.

The [repository documentation](Documentation/README.md) describes the package
architecture; the [contributor guide](CONTRIBUTING.md) covers validation.
- A Computer MCP host with plugin support, on macOS.
- A vendor-supported Computer Use client that provides its `mcp` entry point.
- Python 3.11 or newer (standard library only) to build the archive.

## Documentation

The archive contains this manual, the contributor and architecture documentation, the license
and third-party notices, the manifest and the skill directory; it excludes Git metadata, local
test evidence and external executables, and the same inputs produce the same bytes. The
[documentation index](docs/README.md) covers the package architecture and the
[release process](docs/Reference/Release.md).

CI runs the package tests and keeps the archive and its SHA-256 receipt on pull requests, `master`
pushes and manual runs, and repeats the tests and build on Python 3.11. It runs without macOS and
never starts the vendor client, so its artifacts are neither a public plugin release nor proof of
GUI compatibility. Local ownership or a bundled source label is not a verified GitHub publisher or
artifact signature.

## Contributing

See the [contributor guide](CONTRIBUTING.md).

## License

Computer MCP-owned code and resources use the
[Functional Source License 1.1, Apache 2.0 Future License](LICENSE)
(FSL-1.1-ALv2). See [third-party notices](THIRD_PARTY_NOTICES.md) for the
external vendor client's separate ownership and terms.
[Functional Source License 1.1, Apache 2.0 Future License](LICENSE) (FSL-1.1-ALv2). See the
[third-party notices](THIRD_PARTY_NOTICES.md) for the vendor client's separate ownership and
terms.
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Documentation structure

The root README is the public setup and usage manual. Documentation/README.md
The root README is the public setup and usage manual. docs/README.md
indexes the architecture and reference material. AGENTS.md routes agent work;
CONTRIBUTING.md describes contributor checks. Current package facts belong in
Documentation/Architecture/Package.md; commands, formats and operating detail
belong in Documentation/Reference/.
docs/Architecture/Package.md; commands, formats and operating detail
belong in docs/Reference/.

Version rules belong in the
[organization VERSIONING](https://github.com/computer-mcp/.github/blob/master/VERSIONING.md).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

The manifest contributes the native Computer Use MCP entry and a portable
observe–act–verify skill. The vendor owns the executable and caller compatibility;
the host owns registration, workspace/profile grants and dependency bindings.
the host owns registration, project and profile grants and dependency bindings.
Skills are guidance resources and do not run during installation or reading.

The dependency declares a command name and an application bundle locator. The
Expand All @@ -17,8 +17,9 @@ directory. Discovery does not establish signing identity or GUI caller trust.

The package builder validates regular files and bounded input, rejects links and
special files, and emits deterministic ZIP bytes. The shipped contents are the
manifest, manuals, license and notices, architecture documentation and skill tree. Agent execution
state and source-level test helpers stay with the checkout.
manifest, manuals, license and notices, architecture documentation, skill tree
and the README logo images in `.github/assets/logo/`. Agent execution state
and source-level test helpers stay with the checkout.

The MCP connection goes directly to the vendor. Native AX remains a diagnostic
and fallback facility; uncertain actions require observation before another
Expand Down
File renamed without changes.
File renamed without changes.
Loading
Loading