Know What You Don't Know: Agentic Data Synthesis by Evolving Generation Space for Closed-Loop Learning
The continued improvement of large language models increasingly depends on synthetic experience, yet effective data synthesis for interactive agents requires not only executable and verifiable trajectories, but also interactions that provide learning value for the current task agent.
This paper introduces EvoSyn, a framework for improving synthetic data generation in AI agents by addressing feedback-to-decision gaps through adaptive synthesis spaces.
What happened
Researchers developed EvoSyn, a closed-loop learning framework that uses learner feedback to evolve synthetic data generation. The system addresses limitations in current methods by creating a revisable synthesis space that adapts to weak regions and incorporates new factors through specialized agents.
Why it matters
Improved synthetic data generation enhances AI agent training efficiency, leading to better performance in complex tasks. The framework's ability to adapt to feedback directly addresses a critical bottleneck in interactive AI systems.
- Feedback integration is critical for effective synthetic data generation
- Dynamic synthesis spaces enable better adaptation to task requirements
- Specialized agent roles improve system flexibility and performance
- Quantifiable metrics (13.3-point improvement) validate the approach
- Discovery of new controllable factors enhances model training
The bigger picture
The framework demonstrates measurable improvements in task success rates while generating higher-quality synthetic data, potentially influencing future research in AI training methodologies and synthetic data pipelines.
EvoSyn presents a novel approach to synthetic data generation by creating an adaptive feedback loop that improves AI agent training through dynamic synthesis space evolution.
Technical details
- Multi-agent system with specialized roles (Designer, Planner)
- Revisable state representation for synthesis space
- Feedback-driven adaptation mechanism
- Execution of synthetic trajectories with verifiable outcomes
- CAR-Bench and τ²-Bench evaluation benchmarks
FAQ
What problem does EvoSyn solve?
EvoSyn addresses the feedback-to-decision gap in synthetic data generation, where learner feedback isn't effectively translated into actionable synthesis changes.
How does EvoSyn differ from existing methods?
Unlike traditional approaches, EvoSyn uses a multi-agent system with a revisable synthesis space that adapts to feedback, discovers new controllable factors, and dynamically allocates synthesis effort.
What are the key results of the study?
EvoSyn achieved a 13.3-point improvement in average task success over existing baselines while generating higher-quality synthetic data and discovering previously missed controllable factors.
This summary and analysis were generated with AI from the original article and may contain errors. Read the original for the full story.
Read the full story
Originally published by arXiv - cs.AI on Oct 12, 2026.
Comments
Sign in to join the discussion.