SEAD (Self-Evolving Agent for Service Dialogue) - Co-evolutionary Training Loop Framework. The controller samples initial states (Phase 1), which initialize dialogues producing trajectories (Phase 2), used to train the agent with rewards (Phase 3) and compute completion rates (Phase 4), which feed back to adjust sampling distributions, closing the co-evolutionary loop.

✨If you like this project, please give it a star🌟—it's the best encouragement for us🥺!✨

[2026-05-13] — New 2-instance User Simulator training mode is now available! This mode runs 2 independent user simulator instances (1 GPU each) instead of 1 instance (2 GPUs), significantly reducing the risk of user simulator crashes and improving training stability. 🚀

[2026-04-07] — SEAD has been accepted to ACL 2026 Findings!

[2026-02-04] — We've open-sourced our full research stack!

• 📄 Paper (arXiv): arXiv:2602.03548
• 💻 Code (GitHub): Complete training, inference, and evaluation pipelines
• 🤗 Model (Hugging Face): dayll/SEAD-14B
• 📊 Benchmark: Benchmark and evaluation code is available

What's included:

• ✅ End-to-end training, inference, and evaluation pipelines
• ✅ Reproducible configs and scripts
• ✅ Pretrained checkpoints (14B parameters)
• ✅ Comprehensive evaluation suite
• ✅ Clear documentation and examples

• ✨ Highlights
• ⚙️ Features
• 🤖 Model
• 🏆 Performance
• ⬇️ Installation
• 🚀 Quick Start
  • Training
  • Evaluation
• 🙏 Acknowledgements
• 📖 Citation

🎯 Zero Training Data Required: Our co-evolutionary framework eliminates the need for manually collected dialogue data

🚀 State-of-the-Art Performance: Achieves 52.0% completion rate, outperforming GPT-4o (44.2%) with only 14B parameters

💰 Cost-Effective: Zero inference cost compared to commercial APIs (GPT-4o: ¥727.28 for 1000 samples)

🔄 Self-Evolving: Automatic curriculum learning through adaptive state sampling

⚡ Efficient Training: Supports distributed training on 8 GPUs with vLLM acceleration

• ✅ Co-evolutionary Framework: Adaptive curriculum learning via state controller
• ✅ Distributed Training: Multi-GPU support with efficient parallelization
• ✅ Checkpoint Management: Automatic saving and resuming
• ✅ User Response Cache: Accelerates training by caching User Simulator responses, reducing redundant vLLM calls

To accelerate training, we introduce a UserResponseCache (implemented in SEAD/llm_agent/generation.py) that caches User Simulator responses. Instead of calling the vLLM-served user model for every single dialogue turn, the cache reuses previously generated responses at configurable hit rates.

How it works:

• Responses are cached per dialogue stage (0–3), each with a stage-specific hit rate:
  • Stage 0 (Opening phase): 90% — highly homogeneous, safe to reuse
  • Stage 1 (Objection handling): 50% — diverse user behaviors, moderate reuse
  • Stage 2 (Cooperation phase): 20% — needs personalization, low reuse
  • Stage 3 (Confirmation step): 0% — always calls vLLM for fresh responses
• To prevent stale conversations, the cache enforces a rule: no two consecutive turns for the same sample can both use cached responses (at least one real vLLM call in between).
• Cache is persisted to disk in JSONL format and auto-loaded on restart, with automatic cleanup when exceeding size limits.

To disable the cache (e.g., for debugging or if you want fully fresh responses every turn), set all stage hit rates to 0 in generation.py:

self.stage_hit_rates = {
    '0': 0.0,
    '1': 0.0,
    '2': 0.0,
    '3': 0.0,
}

SEAD is now available on huggingface-hub:

Metrics:

• Params: Model parameters (B=billion, "--" indicates undisclosed or not applicable)
• CR: Completion Rate (%)
• ATT: Average Turns to Target (lower is better ↓)
• UPA: User Portrait Accuracy
• EI: Emotion Improvement
• TI: Trust Improvement
• CI: Cooperation Improvement
• Total Cost: Total inference cost for 1000 multi-turn samples (CNY)

Note: Bold indicates best results. Underlined indicates second-best results. Standard deviations are shown where available.

With the advancement of training, the model's metrics steadily improve, highlighting the effectiveness of RL. The hard business metric, Task Completion, achieves a significant boost, showing that the model has learned better strategies through free exploration. The increase in User Profile Accuracy demonstrates that the model understands users better, while the steady rise in the Trust Variation Mean indicates that the model can more easily gain user trust through conversation.

conda create -n SEAD python=3.10
conda activate SEAD
# install torch [or you can skip this step and let vllm to install the correct version for you]
pip install torch==2.4.0 --index-url https://download.pytorch.org/whl/cu121
# install vllm
pip3 install vllm==0.6.3 # or you can install 0.5.4, 0.4.2 and 0.3.1

# verl
pip install -e .

# flash attention 2
pip3 install flash-attn --no-build-isolation
pip install wandb

Alternatively, you can configure the environment according to requirements.txt:

pip install -r requirements.txt

The User Role-play Model operates within an isolated environment.

conda create -n vllm python=3.10
pip install -r requirements_vllm.txt

To modify prompts (such as user profiles and SOP), edit the files in: ./verl/trainer/config/format_prompt/.

Common user behaviors can be modified by editing ./assets/client_action.jsonl. These behaviors are randomly sampled and incorporated into user prompts to ensure dialogue diversity."

Our model requires no additional training data. Simply load the base model to start training.

We provide two training modes. 2-instance mode is recommended for better stability.

Runs 2 independent user simulator instances, each on 1 GPU (GPU 6, 7), listening on ports 5000 and 5001 respectively. This avoids single-instance overload and greatly reduces crash risk.

How it works: In generation.py, the chatbot URL is configured as a comma-separated list (http://localhost:5000,http://localhost:5001). During each training step, the batch of dialogue requests is evenly split and dispatched to both instances in parallel via multi-threading, then results are merged back. Since each instance runs independently on a single GPU, this eliminates NCCL communication timeouts that can occur when a single instance spans multiple GPUs under heavy load.

conda activate SEAD
bash ./scripts/2ins_main.sh

GPU allocation: Chatbot training (GPU 0-5, 6 cards) + User Sim 1 (GPU 6) + User Sim 2 (GPU 7)

Runs 1 user simulator instance on 2 GPUs (GPU 6, 7) with a single chatbot (1 user + 1 chatbot). Simpler setup with fewer moving parts — if you want to modify or extend the codebase, this mode is easier to start with. However, the single instance may crash under heavy load due to NCCL communication timeouts.

conda activate SEAD
bash ./scripts/main.sh

GPU allocation: Chatbot training (GPU 0-5, 6 cards) + User Sim (GPU 6-7, 2 cards)

Edit ./scripts/main.sh or ./scripts/2ins_main.sh to customize:

• Base model path

Edit ./scripts/train_chatbot.sh or ./scripts/2ins_train_chatbot.sh to customize:

• Batch size and learning rate
• Checkpoint save frequency

To stop a running training process:

pkill -9 -f "main.sh"
pkill -9 -f "train_chatbot.sh"
pkill -9 -f "generate_profiles_parallel.py"
ray stop --force
ps aux | grep python | grep -v grep | awk '{print $2}' | xargs kill -9

To visualize dynamic curves locally, run the following command:

python for_evaluation/metrics_vis.py

The generated plots will be saved in ./outputs/evaluation/report.

Test any local model or your custom-trained model:

# Create Evaluation Set
python utils/create_prompt_data.py \
    --train_samples 0 \
    --test_samples 1000 \
    --behavior_library ./assets/client_action.jsonl \
    --out_dir ./outputs/evaluation/test_set/ \
    --temp_dir ./outputs/evaluation/test_set/user_param/
# Run the evaluation following the instructions in the log
bash ./for_evaluation/vllm_test_suite.sh

Modify ./for_evaluation/vllm_test_suite.sh to set:

• Model checkpoint path

The SEAD framework draws inspiration from pioneering projects such as Search-R1, is built upon veRL and RAGEN. We would like to sincerely thank the teams behind these projects for their invaluable contributions to open-source research and development.

@article{SEADv1,
  title={SEAD: Self-Evolving Agent for Multi-Turn Service Dialogue},
  author={Yuqin Dai, Ning Gao, Wei Zhang, Jie Wang, Zichen Luo, Jinpeng Wang, Yujie Wang, Ruiyuan Wu, Chaozheng Wang},
  journal={arXiv preprint arXiv:2602.03548},
  year={2026}
}