MASkills: Continual Skills Optimization for Multi-Agent LLM Systems

Homepage · Quick Start · Data Preparation · Training and Evaluation · Key Components · Benchmarks · Skill Format · Citation · FAQ

An LLM agent that learns from experience usually keeps what it has learned in one system prompt. Every update rewrites that prompt as a whole, so an edit aimed at one failure mode silently perturbs unrelated competence, and no single piece of knowledge can be credited, revised, or retired on its own. MASkills makes that knowledge discrete and persistent -- a library of individually addressable sub-skills -- and evolves it with typed operators under per-skill credit assignment, so an edit is aimed at the one skill the evidence implicates and is rolled back if it does not hold up.

---

News

[Aug 2026] MASkills v1.0.0 released -- three benchmarks (GAIA, HotpotQA, LOCOMO) behind six entry points, each with a settable agent topology and skill-library initialization.

[Aug 2026] Paper accepted to EMNLP 2026 Findings.

---

Overview

One MASkills iteration: agents execute their skills, an LLM critic assigns credit per agent and per invoked skill, the credit is aggregated and momentum-smoothed into edit directions, and typed operators update the library behind a validation gate.

MASkills replaces the monolithic prompt with a skill library: a directory of SKILL.md files in the Anthropic Agent Skills format, each an independently addressable unit of task knowledge that can be read and edited by hand. Training acts on that structure rather than on one string.

	Monolithic prompt optimization	MASkills
Knowledge	One opaque prompt string	Library K = {k_1, ..., k_m} of readable skills with provenance and measured utility
Update	Rewrites the whole prompt, which only grows	Typed operators on one skill at a time: induct a skill from a recurring failure, refine one (k <- k + dk), consolidate overlaps, prune low-utility skills
Credit	Lands on the agent	Lands on the skills the trajectory actually invoked
Bad updates	Discovered later, if at all	A held-out validation gate rolls the candidate library back

The release covers three benchmarks (GAIA tool use, HotpotQA multi-hop QA, LOCOMO long-term conversational memory) under decentralized, centralized and hybrid topologies, with single or paired agents that each co-evolve their own library (retriever + reasoner, Researcher + Solver). The role prompt can optionally evolve alongside the skills (it stays fixed by default), and evaluation is resumable: an interrupted sweep re-runs only the missing task ids.

---

Quick Start

Installation

git clone https://github.com/DaRL-GenAI/MASkills.git
cd MASkills

# Create and activate a Python 3.10+ environment, e.g. with uv (https://docs.astral.sh/uv/)
uv venv --python 3.11 && source .venv/bin/activate

uv pip install -e .                  # core library
uv pip install -e ".[language]"      # + language tasks data fetch, metrics and tools (enough for the demo)
uv pip install -e ".[gaia]"          # + GAIA attachment parsing (xlsx / pdf / docx / pptx)
uv pip install -e ".[all]"           # everything

Without uv, use any Python 3.10+ environment (venv, conda) and drop the uv prefix: pip install -e ".[language]".

Set your API key -- never hardcode it in a script:

export OPENAI_API_KEY="sk-..."
export OPENAI_BASE_URL="..."      # optional; defaults to https://api.openai.com/v1
export TAVILY_API_KEY="..."       # only for GAIA's SEARCH and BROWSE tools

Demonstration

One command trains a two-agent HotpotQA skill library for ten iterations (20 train / 50 val / 100 test questions). It downloads HotpotQA on the first run (~45 MB):

python scripts/demo.py

The trained libraries land under experiments/runs/<run id>/skills_autogen/iter_<i>/<agent>/<skill_id>/SKILL.md, next to the per-iteration metrics. Pass --iters, --n-train, --n-val, --n-test to change the run size, and --model / --critic-model to change the agents' and the critic's LLM.

How the demo works

scripts/demo.py is a short use of the library API. Its four steps map onto one training loop.

1. Configure the task. skill_evolution=True turns on the discrete skill library and the four operators. paradigm="central_credit" makes the critic assign credit to each agent and to each skill it invoked, rather than one score for the whole team. n_val sets the held-out split the validation gate uses. As in the paper, the agents run on gpt-4o-mini while the critic and the operators run on gpt-5.1.

import maskills

config = maskills.LanguageTaskConfig(
    task_type="qa",
    paradigm="central_credit",
    benchmark_path="env/lang_benchmark/HotPotQA",
    n_train=20, n_val=50, n_test=100,
    num_iterations=10,
    eval_test_every_iter=True,
    skill_evolution=True,
    llm=maskills.LLMConfig.from_preset("gpt-4o-mini"),          # the agents
    critic_llm=maskills.LLMConfig.from_preset("gpt-5.1"),       # credit assignment
    optimizer_llm=maskills.LLMConfig.from_preset("gpt-5.1"),    # the four operators
)

2. Build the environment. The language environment loads HotpotQA and runs two agents in sequence (agent_1 -> agent_2, sharing a message pool), the default decentralized topology. Each agent starts with an empty skill library.

env = maskills.make_env("language", config)

3. Assemble the trainer. SkillCreditCritic reads each trajectory together with the skills it invoked and attributes the outcome per skill. SkillEvolutionOptimizer turns that credit into operator calls on individual skills. SkillEvolutionTrainer runs the loop and the validation gate.

trainer = maskills.SkillEvolutionTrainer(
    config=config,
    env=env,
    critic=maskills.SkillCreditCritic(config),
    optimizer=maskills.SkillEvolutionOptimizer(config.get_optimizer_llm()),
)

4. Train. Iteration 0 scores the empty libraries on the test split as a baseline. Each later iteration rolls out on the training split, assigns credit, applies the operators due that iteration (refine every iteration, induct every 2nd, consolidate and prune every 3rd by default), and commits the candidate library only if its validation reward stays within skill_eval_delta (0.02) of the current one. With eval_test_every_iter=True, every iteration is also scored on the test split, giving a per-iteration test curve in metrics/training_curve.jsonl.

trainer.train()

The same run can also be driven by a config file:

maskills.train("configs/language_task/qa_hotpot_decentralized.json")

---

Data Preparation

Benchmark data is not redistributed here (data/ and env/ are git-ignored). Fetch each dataset you need into the path its loader expects. Running a script without the data fails fast with a FileNotFoundError naming the missing path.

HotpotQA

Used by scripts/train_hotpotqa.py, scripts/eval_hotpotqa.py.

One command fetches the dev-distractor split from the Hugging Face mirror (hotpotqa/hotpot_qa, distractor / validation), rewrites it to the original hotpot_dev_distractor_v1.json format, and clones the hotpotqa/hotpot repo for the official EM / F1 scorer that the metrics import:

pip install pyarrow
python scripts/fetch_hotpotqa.py

Expected layout:

env/lang_benchmark/HotPotQA/
├── hotpot_dev_distractor_v1.json     # 7,405 questions (~45 MB)
└── hotpot/hotpot_evaluate_v1.py      # official EM / F1 scorer

--n-train / --n-val / --n-test slice the file sequentially from the top. The original download at http://curtis.ml.cmu.edu/datasets/hotpot/hotpot_dev_distractor_v1.json is the same file and works if you already have it; the CMU host is often unreachable, which is why the script uses the mirror. The loader also accepts a tasks.jsonl in the same directory with question, context and ground_truth keys.

LOCOMO

Used by scripts/train_locomo.py, scripts/eval_locomo.py.

Clone the upstream repo; the loader reads env/locomo/data/locomo10.json and the metrics import the official F1 from env/locomo/task_eval/evaluation.py:

git clone https://github.com/snap-research/LoCoMo.git env/locomo

The 2/2/6 conversation split (train/val/test, split_seed=42) is documented in configs/locomo/SPLIT_STATS.md.

GAIA

Used by scripts/train_gaia.py, scripts/eval_gaia.py.

GAIA is gated on Hugging Face: accept the terms at gaia-benchmark/GAIA, then download the validation split and its attachments:

pip install -U "huggingface_hub[cli]"
huggingface-cli login
huggingface-cli download gaia-benchmark/GAIA --repo-type dataset \
  --include "2023/validation/*" --local-dir data/gaia/raw

The scripts take any JSONL of GAIA items (one JSON object per line, with at least task_id, Question, Final answer, and file_name / file_path for attached files). Build your own splits from the raw metadata, for example:

python - <<'PY'
import json, random, shutil
from pathlib import Path
raw = Path("data/gaia/raw/2023/validation")
out = Path("data/gaia"); out.mkdir(exist_ok=True)
items = [json.loads(l) for l in (raw / "metadata.jsonl").read_text().splitlines() if l.strip()]
for it in items:                      # copy attachments next to the JSONL
    if it.get("file_name"):
        shutil.copy(raw / it["file_name"], out / it["file_name"])
        it["file_path"] = it["file_name"]
random.Random(42).shuffle(items)
for name, chunk in [("train100", items[:100]), ("test65", items[100:165])]:
    (out / f"{name}.jsonl").write_text("".join(json.dumps(i) + "\n" for i in chunk))
PY

Attachments are resolved relative to data/gaia/ via each item's file_path. The defaults are data/gaia/train100.jsonl for training and data/gaia/test65.jsonl for evaluation; override with --train / --input.

---

Training and Evaluation

Six entry points: train and evaluate, for each benchmark. Every one takes --topology and a skill-library flag, so a library can be trained under one topology and scored under another.

	GAIA	HotpotQA	LOCOMO
train	scripts/train_gaia.py	scripts/train_hotpotqa.py	scripts/train_locomo.py
evaluate	scripts/eval_gaia.py	scripts/eval_hotpotqa.py	scripts/eval_locomo.py

# HotpotQA and LOCOMO: one invocation runs every iteration.
python scripts/train_hotpotqa.py --n-train 100 --n-val 50 --n-test 100 --iters 5
python scripts/train_locomo.py --category 1 --iters 5   # 1=multihop 2=temporal 3=opendomain 4=singlehop

# GAIA keeps its library on disk, so one invocation is one iteration.
python scripts/train_gaia.py --topology decentralized --iter 1 \
  --init-skills <K_i dir> --out-skills <K_i+1 dir>

# Evaluate. --skills empty is the no-skills floor each benchmark reports.
python scripts/eval_hotpotqa.py --topology hybrid --skills <library dir>
python scripts/eval_locomo.py --category 1 --skills empty
python scripts/eval_gaia.py --skills empty --limit 20

Every trainer also accepts --init-skills empty to start from no skills at all, and --ablation to disable one mechanism at a time.

---

Key Components

MASkills adds four components on top of a language-space CTDE loop:

1. Skill Library -- Each agent holds a set of discrete skills K_i = {k_1, ..., k_m}, each with a description used for routing, a body the agent reads, and a running utility estimate.

2. Skill-Conditioned Credit -- The critic sees the trajectory and the skills it invoked, and attributes the outcome per skill, plus a residual for what no skill covers.

3. Typed Operators -- Induct, refine, consolidate and prune, each on its own cadence. Refinement is cheap and runs often; induction, consolidation and pruning act on longer horizons, once evidence has accumulated.

4. Validation Gate -- The candidate library is rolled out on a held-out split and committed only if its reward stays within a tolerance of the current one. Otherwise the current library is restored, so a bad iteration costs time rather than competence.

One iteration:

Rollout under K_i -> per-skill credit + residual -> aggregate to a gradient per skill
  -> apply the scheduled operators -> candidate K_i+1 -> validation gate -> commit or roll back

Library API

Class	Description
maskills.SkillLibrary / maskills.Skill	The discrete library and its units
maskills.SkillCreditCritic	Critic performing per-skill causal attribution
maskills.SkillEvolutionOptimizer	The four operators over a library
maskills.SkillEvolutionTrainer	Training loop with the operator schedule and the validation gate
maskills.skill_lib	On-disk SKILL.md library I/O (load_lib, apply_ops, snapshot_lib)
maskills.LLMConfig	Unified LLM backend configuration
maskills.make_env / maskills.register_env	Environment registry

---

Benchmarks

Benchmark	Agents	Tools	What the skills have to learn
GAIA	1, or Researcher + Solver	SEARCH / BROWSE (Tavily), COMPUTE (sandboxed Python), attachment parsing	Tool-call syntax, evidence discipline, the exact-match answer format
HotpotQA	2, sequential	Wiki search	Multi-hop decomposition, keeping the final answer minimal
LOCOMO	Retriever + Reasoner	grep over the conversation	A grep plan per hop, carrying session dates through the handoff, the grader's answer shape

LOCOMO is additionally trained per question category, because a good skill for a temporal question looks nothing like one for a multi-hop question.

Test score per training iteration on each benchmark. Markers show the held-out validation score of each candidate library: green candidates were accepted, red ones were rolled back.

Topologies

Topology	Shape
decentralized	Sequential: agent_1 -> agent_2, sharing a message pool
centralized	A main agent delegates via <retrieve>; each sub-call runs in a fresh context
hybrid	Same delegation, but the sub-agent accumulates state across calls

Any library can be scored under any topology, which is what makes the comparison controlled:

for topo in decentralized centralized hybrid; do
  python scripts/eval_hotpotqa.py --topology $topo --skills <library dir>
done

GAIA is the exception, because it has two rollout implementations. --engine library (the default) is the on-disk pipeline the trainer uses, so its score is comparable to training, and it implements centralized and decentralized. --engine env drives the agents through GaiaEnv, the path the other benchmarks use, and adds hybrid -- so a topology comparison on GAIA needs that engine for all three.

---

Skill Format

A skill library is a directory of SKILL.md files in the Anthropic Agent Skills format:

<lib_dir>/
    SKILL.md            # root identity / always-on instructions
    <skill_name>/
        SKILL.md        # one sub-skill, loaded and edited independently
    ...

Each SKILL.md is YAML frontmatter followed by a markdown body:

---
name: cross-verify-fact
description: Re-derive a numeric or factual answer via a second independent method before finalizing.
---

Before emitting FINAL ANSWER, recompute the result using a different approach
(unit conversion, alternate source, or a sanity-check calculation). If the two
disagree, investigate before answering.

description is what the agent sees when deciding whether to load a skill, so it carries the routing signal; the body is what it reads once it does.

Two paths consume a library. On disk (GAIA), config.agent_skills_dirs points an agent at a directory and maskills.skill_lib applies the optimizer's operations back to it. In memory (HotpotQA, LOCOMO), load_skills_dir reads it into a SkillLibrary that lives inside the policy and is evolved by SkillEvolutionTrainer.

Setting skills_dir on a config instead installs a fixed library: rendered into every agent's prompt, but the optimizer never sees it and no operator touches it.

Evolving the role as well as the skills

An agent's prompt has two parts: its role -- where it sits in the team and how it must communicate -- and its skill library. By default training moves only the library, because the role carries the collaboration protocol the environment parses (the handoff format, the tags the agent emits, which agent speaks last), and an edit there can invalidate every trajectory at once in a way no skill can repair.

--evolve-role on any trainer turns it on. What drives an edit is the residual: the failures credit assignment could not attribute to any skill, which is exactly the part of the prompt the library does not cover. Edits are anchor-based rather than a rewrite, and the proposed role goes through the same validation gate as the library.

python scripts/train_hotpotqa.py --evolve-role --iters 5

On GAIA the role is the library's root SKILL.md, edited by a refine operation on the _root_ slug.

---

Extending to a New Benchmark

Subclass maskills.BaseEnvironment, implement reset, step, sample_tasks and collect_trajectory, and register it:

from maskills.envs import register_env

@register_env("my_task")
class MyEnv(maskills.BaseEnvironment):
    ...

To make it skill-aware, render each agent's library into its system prompt with render_skill_library, and record which skills each turn invoked with build_skill_trace -- that trace is what lets the critic attribute credit per skill rather than per agent.

---

Project Structure

maskills/                  # The library (pip-installable)
├── core/                  # Skill library, per-skill credit, the four operators,
│                          #   base classes, critic, policy, trajectory
├── config/                # Unified configuration system
├── trainer/               # SkillEvolutionTrainer, MonteCarloTrainer, callbacks
├── experiments/           # Per-benchmark training and evaluation logic
├── envs/                  # Environment registry and the three benchmarks
│   ├── gaia/              #   Tool stack, rollouts, task loader
│   ├── language/          #   HotpotQA / MATH / HumanEval, search tools
│   └── locomo/            #   Long-term conversational memory, grep tool
├── llm/                   # LLM client and token tracking
├── store/                 # Checkpointing, trajectory storage, logging
└── cli/                   # `maskills train` entry point

scripts/                   # demo.py, fetch_hotpotqa.py, and train / evaluate x three benchmarks
configs/                   # Experiment configs, one per benchmark × paradigm
tests/                     # Test suite (pytest)

---

FAQ

How do I configure the API key?

Option 1: Set environment variables:

export OPENAI_API_KEY="sk-..."
export TAVILY_API_KEY="tvly-..."   # GAIA only

Option 2: Point api_key_env_var inside the config's llm object at a different environment variable.

Never put a literal key in a tracked file.

Which LLM backends work?

Any OpenAI-compatible endpoint. The default is the OpenAI API (https://api.openai.com/v1) with OpenAI model names (gpt-4o, gpt-5.1). For other models, such as qwen/qwen-2.5-7b-instruct for the GAIA actor, set OPENAI_BASE_URL to OpenRouter or another gateway and use that gateway's model names. GAIA's SEARCH tool uses Perplexity Sonar when the endpoint is OpenRouter and Tavily otherwise; GAIA_SEARCH_BACKEND=tavily|sonar overrides the choice. maskills.list_available_models() lists the presets.

What does --skills empty do?

It evaluates the agents with no task knowledge -- the floor each benchmark's table reports alongside the trained libraries. The same idea on a trainer (--init-skills empty) starts training from nothing, which tests whether the operators can build a useful library from scratch. It is also the default everywhere, since no library ships with the repository.

On HotpotQA and LOCOMO the library is literally empty. On GAIA it is protocol only: the tool syntax (SEARCH:, BROWSE:, COMPUTE:) and the output contract (HANDOFF_TO_SOLVER, FINAL ANSWER:), with no task knowledge of any kind. A GAIA agent given a blank prompt would fail on the wire format rather than on the task, which would measure the wrong thing; see maskills/envs/gaia/protocol.py.

How do I resume an interrupted evaluation?

Re-run the same command. Results append to <tag>_rows.jsonl keyed by task id, and a re-run skips the tasks already there. --report-only re-summarizes what is on disk without running anything.

Why does GAIA have two evaluation engines?

GAIA has two rollout implementations: the on-disk SKILL.md pipeline the trainer uses, and GaiaEnv, which drives the agents the way the other benchmarks do and is the only one implementing hybrid. Their numbers are not interchangeable, so --engine makes the choice explicit rather than mixing them silently.

What does each ablation disable?

• credit -- the optimizer is shown every rollout rather than only the failures, removing the per-task credit signal.
• rollback -- the validation gate never rejects, so every candidate library is committed.
• consolprune -- only induct and refine run, so the library can grow but never contract.
• momentum (LOCOMO) -- skill utility is overwritten with the batch mean instead of an EMA.

---

Citation

If MASkills is useful in your research, please cite:

@inproceedings{yao2026maskills,
  title     = {MASkills: Continual Skills Optimization for Multi-Agent LLM Systems},
  author    = {Yao, Huaiyuan and Liu, Xiaoou and Fleming, Charles and Chen, Tianlong and Wei, Hua},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2026},
  year      = {2026}
}

---

Acknowledgements

MASkills builds on LangMARL for its language-space CTDE substrate, and on the GAIA, HotpotQA and LOCOMO benchmarks. Each keeps its own license.

---

License

Released under the MIT License.