Official EAU project website: eau-meta-agent.github.io · alphaXiv (arXiv:2610.16523).
Environment–Agent–User Co-evolution
for Training User-Centric Agents
EAU is a meta-agent framework that evolves the environment, the agent, and the simulated user together while training user-centric agents.
01 / The idea
EAU co-evolves the environment, agent, and simulated user to train user-centric LLM agents.
For multi-turn reinforcement learning with users and tools, its meta-agent reads trajectories, tool traces, and training statistics. It proposes and validates changes to the hybrid user simulator, environment feedback, reward design, and reinforcement-learning configuration.
Within a run, supported updates take effect while the agent policy learns. Across runs, an experience pool carries findings into later training attempts. Final-test feedback stays outside the adaptation loop.
EAU is evaluated on UserGym, ColBench, and τ²-bench with Qwen3-4B and Qwen2.5-7B-Instruct. The reported results describe these benchmark settings, not a guarantee for every agent environment.
See how co-evolution worksRelative gain in ColBench task successQwen3-4B · vs. InfoPO
Relative gain in overall scoreQwen3-4B · UserGym + ColBench
Interactive benchmarks / policy backbonesQwen3-4B · Qwen2.5-7B-Instruct
02 / The framework
The policy learns through reinforcement learning.
The meta-agent adapts the experience it learns from.
Combine explicit symbolic rules with natural LLM responses. The meta-agent can revise disclosure rules, persona instructions, generation settings, and the mixture between the two simulators.
Training feedback guides adaptation.
Final-test feedback stays outside the loop.
Revise intermediate rewards when task success is too sparse to guide exploration. The meta-agent can reward meaningful progress, correct unhelpful reward designs, and adjust supported interaction limits.
The neural policy learns through PPO with an action-token critic. The meta-agent monitors optimization logs and trajectories, adjusting configurable dynamics such as entropy and KL coefficients while keeping the training algorithm fixed.
Hot-swap supported user rules, reward designs, and training settings as the policy changes.
Consolidate findings in an experience pool. Restart when needed and build on earlier discoveries.
03 / The evidence
Evaluated on UserGym, ColBench, and τ²-bench.
Explore the results across two policy backbones.
Average score
Successful task completion
UserGym + ColBench
Higher is better. Relative gains compare eau with InfoPO. Overall scores aggregate UserGym and ColBench under the paper’s evaluation protocol.
| Method | UserGym avg. | ColBench pass | ColBench success | Overall |
|---|---|---|---|---|
| Backbone | 0.426 | 0.272 | 0.153 | 0.378 |
| ReAct | 0.457 | 0.269 | 0.145 | 0.401 |
| Reflexion | 0.455 | 0.303 | 0.184 | 0.408 |
| RAGEN | 0.468 | 0.479 | 0.361 | 0.457 |
| Search-R1 | 0.472 | 0.467 | 0.355 | 0.459 |
| UserRL | 0.509 | 0.468 | 0.342 | 0.486 |
| InfoPO | 0.559 | 0.553 | 0.439 | 0.545 |
| PPO | 0.522 | 0.284 | 0.200 | 0.462 |
| eau | 0.623 | 0.685 | 0.600 | 0.627 |
On Qwen3-4B, eau reaches an aggregate score of 21.5, compared with 18.5 for PPO. It leads on Airline and Telecom; InfoPO leads on Retail.
04 / A closer look
After diagnosing severe issues and restarting early runs, eau explores interventions within a run and carries the findings forward. The resulting frontier improves beyond the generic meta-agent in this experiment.
Starting from the same first four runs of experience, adding within-run intervention reaches 0.623 on UserGym, compared with 0.577 for between-run adaptation alone.
These curves show progression by run count, rather than a comparison under matched total compute. Evaluation conditions remain fixed while the training system evolves.
05 / Continue exploring
Environment–Agent–User Co-evolution
for Training User-Centric Agents
@misc{cai2026eauenvironmentagentusercoevolutiontraining,
title = {EAU: Environment-Agent-User
Co-evolution for Training
User-Centric Agents},
author = {Min Cai and Dan Zhang and
Gao Xing and Libin Ma and
Zhaohui Xu and Xi Ye},
year = {2026},
eprint = {2610.16523},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2610.16523}
}