Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ python -m skillopt_sleep <action> [options]
```

Actions are `run`, `dry-run`, `status`, `adopt`, `harvest`, `schedule`,
`unschedule`, and `evalkit`. `evalkit` is also available as
`unschedule`, `export-rules`, `import-rules`, and `evalkit`. `evalkit` is also available as
`python -m skillopt_sleep.evalkit` and compares two conditions on one fixed
task manifest (McNemar + bootstrap CI). Exactly one of its `--b` comparison
input or `--aa` identity-check flag is required. See `docs/sleep/evalkit.md`.
Expand Down Expand Up @@ -168,6 +168,13 @@ roots. Use `--skill-root` for another integration-specific location. Configure
the canonical `multi_skill_fanout` key to enable proposal fan-out;
`multi_skill_report` remains a compatibility alias.

`export-rules` converts accepted skill additions from one staging report into a
versioned, transcript-free manifest. `import-rules` first displays the complete
manifest for review; after `--reviewed` is supplied, it evaluates every rule on
the importer's task file `val` split and stages only strict, no-regression improvements.
Publisher-reported effects do not affect the gate. See
[community rule exchange](../sleep/community-rules.md).

The `mock` and `handoff` backends make no network calls. A real backend sends
mining, replay, judging, and reflection prompts derived from harvested
transcripts and tasks to its selected provider. Review that provider's
Expand Down
7 changes: 7 additions & 0 deletions docs/sleep/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,16 @@ skillopt-sleep status # show state + the latest staged proposal
skillopt-sleep adopt --legacy # apply a reviewed managed proposal
skillopt-sleep adopt --skill NAME # adopt one staged skill (repeatable)
skillopt-sleep adopt --all-skills # adopt every still-pending fan-out skill
skillopt-sleep export-rules ... # export accepted rules without transcripts
skillopt-sleep import-rules ... # review + locally gate community rules
skillopt-sleep schedule # install a nightly cron entry for this project
```

Community manifests contain distilled rules and aggregate effect metadata, not
session transcripts. Imports require an explicit review acknowledgement and put
every rule through a strict local no-regression gate before staging. See
[community rule exchange](community-rules.md).

> **Version note.** This page tracks `main`. PyPI 0.2.0 provides the base
> commands above. Cursor source/backend/plugin support, VS Code Copilot
> transcript harvesting, Pi source/backend support, Sleep handoff, non-Azure
Expand Down
99 changes: 99 additions & 0 deletions docs/sleep/community-rules.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Community rule exchange

SkillOpt-Sleep can exchange distilled skill rules without exchanging session
transcripts. Imported rules are untrusted candidates: each rule must be reviewed
and must strictly improve the importer's own task set without regressing any task
before it is staged for adoption.

This is a local workflow. It does not upload manifests, operate a registry, or
automatically adopt imported rules.

An empirical gate is not a sandbox or a security review. Imported text changes
agent behavior during validation, so inspect the rule and license before passing
`--reviewed`, and use the same provider and execution-boundary precautions as an
ordinary Sleep replay.

## Export accepted rules

Export reads an accepted staging report and includes only accepted `skill/add`
edits. Memory edits, tasks, responses, session identifiers, and evidence logs are
not copied.

```bash
skillopt-sleep export-rules \
--staging .skillopt-sleep/staging/20260722-031700 \
--output community-rules.json \
--category coding \
--license MIT
```

The observed effect belongs to the complete candidate set evaluated by that
staging run, not to an individual rule in isolation. The exporter labels this as
`"scope": "candidate_set"`. It also refuses secret-shaped rule or rationale text,
but that is not an anonymization guarantee. Inspect the output before publishing
it to a public Git repository.

## Review and import

The first invocation prints the complete manifest and exits without running it:

```bash
skillopt-sleep import-rules --manifest community-rules.json
```

After reviewing every rule and the manifest license, run the local gate with a
reviewed task file and an explicit target skill:

```bash
skillopt-sleep import-rules \
--manifest community-rules.json \
--reviewed \
--project /path/to/project \
--target-skill-path .agents/skills/my-skill/SKILL.md \
--tasks-file reviewed-tasks.json \
--backend codex
```

The importer evaluates rules sequentially using only the task file's `val`
split; `train` and `test` remain outside the import decision. For each rule it
replays the same validation tasks against the current skill and the candidate
skill. A rule enters the staged proposal only when its configured gate score
strictly increases and no task score decreases. Publisher-reported effects are
informational and never participate in the local decision.

Review an accepted `proposed_SKILL.md`, then use the existing explicit adoption
step:

```bash
skillopt-sleep adopt --legacy
```

## Manifest v1

```json
{
"schema": "skillopt.community-rules",
"schema_version": 1,
"license": "MIT",
"rules": [
{
"id": "rule-63e143cbd8ab167d",
"category": "coding",
"rule": "Run focused tests before reporting a change as complete.",
"rationale": "Prevents false completion reports.",
"observed_effect": {
"metric": "local_gate_score",
"baseline": 0.5,
"candidate": 0.75,
"delta": 0.25,
"sample_size": 20,
"scope": "candidate_set"
}
}
]
}
```

The v1 parser rejects unknown fields. This keeps the public artifact bounded to
the rule, a provenance-free rationale, aggregate effect metadata, category, and
license; raw trajectories have no field in the format.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ nav:
- Deep Learning Analogy: guide/dl-analogy.md
- SkillOpt-Sleep:
- Overview: sleep/README.md
- Community Rule Exchange: sleep/community-rules.md
- Paired A/B Evalkit: sleep/evalkit.md
- Multi-skill Staging: sleep/multi-skill-staging.md
- OpenAI-compatible Endpoints: sleep/openai-compatible-endpoints.md
Expand Down
Loading