SkillOpt: Executive Strategy for Self-Evolving Agent Skills

Train agent skills like you train neural networks — with epochs, (mini-)batchsize, learning rates, and validation gates — but without touching model weights.

• [2026-06-02] 🎉 SkillOpt v0.1.0 is now available on PyPI! Install with pip install skillopt. This initial release includes the full training loop (rollout → reflect → aggregate → select → update → evaluate), multi-backend support (OpenAI / Azure / Claude / Qwen / MiniMax), six built-in benchmarks, and WebUI dashboard.

Modern agent skills are usually hand-crafted, generated one-shot by a strong LLM, or evolved through loosely controlled self-revision — none of which behaves like a deep-learning optimizer for the skill itself, and none of which reliably improves over its starting point under feedback.

SkillOpt treats the skill document as the trainable state of a frozen agent, and trains it with the discipline that makes weight-space optimization reproducible. A separate optimizer model turns scored rollouts into bounded add / delete / replace edits on a single skill document; a candidate edit is accepted only when it strictly improves a held-out validation score. A textual learning-rate budget, a rejected-edit buffer, and an epoch-wise slow / meta update make skill training stable while adding zero inference-time model calls at deployment.

The deployed artifact is a compact best_skill.md (typically 300–2,000 tokens) that runs against the unchanged target model. Across six benchmarks, seven target models, and three execution harnesses (direct chat, Codex CLI, Claude Code CLI), SkillOpt is best or tied-best on all 52 evaluated (model, benchmark, harness) cells and on GPT-5.5 lifts the average no-skill accuracy by +23.5 points in direct chat, +24.8 inside the Codex agentic loop, and +19.1 inside Claude Code. Optimized skill artifacts transfer across model scales, between Codex and Claude Code harnesses, and to nearby benchmarks without further optimization.

For the full method, ablations, and per-cell results see the paper; for a visual walkthrough of the loop see the project page; for deeper API / backend / benchmark docs see docs/.

▶ Watch the full demo on YouTube

• Python 3.10+
• uv

pip install skillopt

# With optional extras:
pip install skillopt[alfworld]    # ALFWorld benchmark
pip install skillopt[webui]       # Gradio monitoring dashboard
pip install skillopt[claude]      # Claude model backend

git clone https://github.com/microsoft/SkillOpt.git
cd SkillOpt

uv venv --python 3.12
source .venv/bin/activate
uv pip install -e .

# For the ALFWorld benchmark (optional):
uv pip install -e ".[alfworld]"
alfworld-download

.env.example ships preconfigured for Claude via the Claude CLI (OPTIMIZER_BACKEND / TARGET_BACKEND = claude_chat, model claude-sonnet-4-6). For that path you don't need to edit anything — just copy and source it, then log in to the CLI (see below):

cp .env.example .env
source .env

To use a different backend (Azure OpenAI, Qwen, MiniMax), edit .env: set OPTIMIZER_BACKEND / TARGET_BACKEND and that backend's credentials — see the sections below. You can also override the backend per run with --backend on any script.

The claude_chat backend doesn't call the Anthropic API directly — it shells out to the claude CLI (claude -p). Authentication therefore piggybacks on the CLI's own login, so there is no API key to manage and nothing secret in .env. The backend itself is already selected by .env.example (see Configure API Credentials above) — all that's left is the CLI login.

Step 1 — Install the CLI and log in once (skip if you already have it):

claude auth login          # or run `claude` and use /login
claude --version           # sanity check: should print a version

Step 2 — Verify the wiring with a one-line smoke test:

OPTIMIZER_BACKEND=claude_chat TARGET_BACKEND=claude_chat python -c "
from skillopt.model import set_target_backend, chat_target
set_target_backend('claude_chat')
text, usage = chat_target('You are a calculator. Reply with only the number.', 'What is 21 + 21?')
print('reply =', text.strip(), '| usage =', usage)
"
# Expected: reply = 42 | usage = {...}

If you see reply = 42, the CLI, login, and backend are all wired correctly.

Note: Do not set ANTHROPIC_API_KEY when relying on the CLI subscription login — if it is present in the environment, the CLI bills against the API key instead of your login. Set it only if you specifically want API-key billing.

export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
# Option 1: API key auth
export AZURE_OPENAI_API_KEY="your-key"
# Option 2: Azure CLI auth (no API key needed)
export AZURE_OPENAI_AUTH_MODE="azure_cli"

Note: AZURE_OPENAI_ENDPOINT is required for all three modes (api_key, azure_cli, openai_compatible). Without it, all LLM calls will fail.

export AZURE_OPENAI_ENDPOINT="https://api.openai.com/v1"
export AZURE_OPENAI_API_KEY="sk-..."
export AZURE_OPENAI_AUTH_MODE="openai_compatible"

This routes all calls through the plain OpenAI Python client (no Azure auth, no api-version header).

Note: SkillOpt reuses the AZURE_OPENAI_* env var names even in this mode — there is no separate OPENAI_API_KEY knob.

export QWEN_CHAT_BASE_URL="http://localhost:8000/v1"
export QWEN_CHAT_MODEL="Qwen/Qwen3.5-4B"

qwen_chat can also be used as the optimizer backend. When optimizer and target should point to different local vLLM services, use the role-specific settings:

python scripts/train.py \
    --config configs/searchqa/default.yaml \
    --optimizer_backend qwen_chat \
    --target_backend qwen_chat \
    --optimizer_model Qwen/Qwen3.5-4B \
    --target_model Qwen/Qwen3.5-4B \
    --optimizer_qwen_chat_base_url http://localhost:8001/v1 \
    --target_qwen_chat_base_url http://localhost:8000/v1

export MINIMAX_BASE_URL="https://api.minimax.io/v1"
export MINIMAX_API_KEY="..."
export MINIMAX_MODEL="MiniMax-M2.7"

What is SearchQA? A context-grounded question-answering benchmark used as the running example below. The agent receives the skill document, a question, and context passages, then must return its answer inside <answer>...</answer> tags (single turn). SkillOpt doesn't fine-tune the model — it iteratively rewrites the skill so the agent answers more accurately. See Supported Benchmarks for the full list.

⚠️ Materialize the data first. The repo ships only split manifests (data/*_id_split/ — lists of example IDs, no questions/contexts/answers). Pointing --split_dir at a *_id_split/ manifest will run but score 0.0 everywhere, since the examples have no content. Materialize the runnable split into the split_dir each config expects first:

uv pip install -e ".[data]"                 # one-time: pulls in datasets + huggingface_hub
python scripts/materialize_searchqa.py      # → data/searchqa_split/
python scripts/materialize_livemath.py      # → data/livemathematicianbench_split/

Claude CLI way — no endpoint, no API key, no --*_model flags (defaults to claude-sonnet-4-6). Just add --backend claude:

# Train on SearchQA with Claude (after materializing data/searchqa_split):
python scripts/train.py \
    --config configs/searchqa/default.yaml \
    --split_dir data/searchqa_split \
    --backend claude

# Other benchmarks (Claude) — each split_dir is also the config default, so it's optional:
python scripts/train.py --config configs/livemathematicianbench/default.yaml --split_dir data/livemathematicianbench_split --backend claude
python scripts/train.py --config configs/alfworld/default.yaml --split_dir data/alfworld_path_split --backend claude

Each --split_dir above is already the config default, so you can omit it. The same pattern works for any benchmark — swap the --config/--split_dir (after materializing that benchmark's data per the table above). Using Azure OpenAI instead, drop --backend claude and pass the endpoint + models:

# Train on SearchQA with Azure OpenAI:
python scripts/train.py \
    --config configs/searchqa/default.yaml \
    --split_dir data/searchqa_split \
    --azure_openai_endpoint https://your-resource.openai.azure.com/ \
    --optimizer_model gpt-5.5 \
    --target_model gpt-5.5

# Other benchmarks (Azure):
python scripts/train.py --config configs/livemathematicianbench/default.yaml --split_dir data/livemathematicianbench_split --azure_openai_endpoint https://your-resource.openai.azure.com/ --optimizer_model gpt-5.5 --target_model gpt-5.5
python scripts/train.py --config configs/alfworld/default.yaml --split_dir data/alfworld_path_split --azure_openai_endpoint https://your-resource.openai.azure.com/ --optimizer_model gpt-5.5 --target_model gpt-5.5

Key CLI arguments:

Evaluate a trained skill on specific data splits without training.

Claude CLI way — just add --backend claude (no endpoint, no models):

# Evaluate the packaged SearchQA skill on one split with Claude:
python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
  --skill ckpt/searchqa/gpt5.5_skill.md \
  --split valid_unseen \
  --split_dir data/searchqa_split \
  --backend claude

# Evaluate on all splits (train + val + test):
python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
  --skill ckpt/searchqa/gpt5.5_skill.md \
  --split all \
  --split_dir data/searchqa_split \
  --backend claude

Using Azure OpenAI instead, drop --backend claude and pass --azure_openai_endpoint https://your-resource.openai.azure.com/.

To evaluate a skill produced by your own training run, replace --skill with that run's best-skill path, for example outputs/my_run/best_skill.md.

Each training run writes to a structured output directory:

outputs/<run_name>/
├── config.json              # Flattened runtime config
├── history.json             # Per-step training history
├── runtime_state.json       # Resume checkpoint
├── best_skill.md            # Best validated skill document
├── skills/skill_vXXXX.md   # Skill snapshot per step
├── steps/step_XXXX/        # Per-step artifacts (patches, evals)
├── slow_update/epoch_XX/   # Slow update logs
└── meta_skill/epoch_XX/    # Meta skill logs

Re-running the same command auto-resumes from the last completed step.

We provide a subset of the paper's main Table 1 GPT-5.5 optimized skills in ckpt/ as reference artifacts. Use them with scripts/eval_only.py to evaluate the provided skills on a matching data split without re-running training. See ckpt/README.md for the full per-benchmark command. This is the first artifact batch; we plan to continue uploading the remaining optimized skills and benchmark split manifests as they are cleaned and verified.

SkillOpt expects data in a split directory with train/, val/, test/ subdirectories, each containing a JSON file (e.g., items.json):

data/my_split/
├── train/items.json
├── val/items.json
└── test/items.json

Each JSON file is an array of task items. The required fields depend on the benchmark. For example, SearchQA items look like:

[
  {
    "id": "unique_item_id",
    "question": "Who wrote the novel ...",
    "context": "[DOC] relevant passage text ...",
    "answers": ["expected answer"]
  }
]

See skillopt/envs/<benchmark>/dataloader.py for the exact format each benchmark expects.

Note: Most benchmark datasets are not included in this repository — the data/*_id_split/ directories are ID manifests only (stable example IDs per split, no payload). To get a runnable split you materialize the full examples from the upstream source listed in data/README.md into the split_dir path each config expects (e.g. data/searchqa_split/). For SearchQA this is automated: python scripts/materialize_searchqa.py joins data/searchqa_id_split/ (400 / 200 / 1400) against lucadiliello/searchqa and writes data/searchqa_split/. For other benchmarks, follow the lookup keys in data/README.md.

configs/_base_/default.yaml is the single source of truth for SkillOpt's runtime knobs. Out of the box, every included benchmark config inherits from it and keeps the paper protocol visible: 4 epochs, rollout batch 40, reflection minibatch 8, textual learning rate 4 with cosine decay, strict hard validation gating, and slow-update + meta-skill enabled. One detail to watch is slow-update acceptance: the current main default is the newer post-submission force-accept mode, while the paper protocol and the paper-aligned skills under ckpt/ use the gated semantics described in paper Section 3.6.

The epoch-boundary slow / meta update can be applied two ways, controlled by optimizer.slow_update_gate_with_selection:

optimizer:
  slow_update_gate_with_selection: false   # current main default

• false (current main default): force-accept. The slow-update guidance is injected into both current_skill and best_skill unconditionally at the epoch boundary. This is the newer post-submission behavior on main.
• true (paper / ckpt-skill reproduction): gated, matching paper Section 3.6 verbatim. The slow-update candidate is evaluated on the selection split and accepted only if it passes the same validation gate as a step-level edit. Use this setting when re-running optimization to match the paper protocol and the provenance of the provided ckpt/ skills.

The trainer prints which mode is active at startup ([slow update] acceptance=...). See issue #22 for the discussion that led to the flag.

The validation gate compares candidate vs. current skills on the selection split using gate_metric:

• hard (default, paper): exact-match accuracy, strictly greater than the current score is required.
• soft: per-item soft / partial-credit score. Useful when the selection split is small (e.g. ≤10 items) and the reward is continuous, where the discrete hard gate often rejects every candidate.
• mixed: weighted average, (1 - w) * hard + w * soft, with w set by gate_mixed_weight (default 0.5).

Default is hard. Use the optional feature config below to switch.

These are not default SkillOpt settings — they are optional feature configs contributed by users for specific scenarios. The paper-reported numbers were obtained with the default settings, not these.

• configs/features/soft_gate.yaml (PR #25, contributed by @lvbaocheng) — switches gate_metric to soft (or mixed). See the comment at the top of the file for when to use and when not to.

A backend = a chat / exec target (e.g. openai_chat, claude_chat, qwen_chat, minimax_chat, codex_exec, claude_code_exec). See docs/guide/new-backend.md for the full contract; in short you add a skillopt/model/<name>_backend.py module, register it in skillopt/model/common.py + backend_config.py, and wire it through the router in skillopt/model/__init__.py. qwen_backend.py and minimax_backend.py are good templates.

A benchmark = a skillopt/envs/<name>/ package with a dataloader.py, a rollout.py, and an initial.md seed skill. See docs/guide/new-benchmark.md for the full contract; the simplest reference is skillopt/envs/searchqa/.

Launch the monitoring dashboard (optional):

pip install -e ".[webui]"
python -m skillopt_webui.app

@misc{yang2026skilloptexecutivestrategyselfevolving,
      title={SkillOpt: Executive Strategy for Self-Evolving Agent Skills}, 
      author={Yifan Yang and Ziyang Gong and Weiquan Huang and Qihao Yang and Ziwei Zhou and Zisu Huang and Yan Li and Xuemei Gao and Qi Dai and Bei Liu and Kai Qiu and Yuqing Yang and Dongdong Chen and Xue Yang and Chong Luo},
      year={2026},
      eprint={2605.23904},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2605.23904}
}