面向智能体强化学习的动态技能生命周期管理
研究团队提出SLIM框架,用于动态管理大型语言模型智能体在强化学习中使用的外部技能。该框架将活跃技能集视为与策略学习协同优化的变量,通过留一验证评估技能边际贡献,并执行三项操作:保留高价值技能、淘汰贡献可忽略的旧技能、在持续失败时扩展技能库。在ALFWorld和SearchQA基准测试中,SLIM平均超越最佳基线方法7.1个百分点。实验表明,策略学习与外部技能保留可共存:部分技能被策略内化,另一些则持续提供外部价值,验证了动态技能管理的普适性与优越性。
Large language model agents increasingly rely on external skills to solve complex tasks, where skills act as modular units that extend their capabilities beyond what parametric memory alone supports. Existing methods assume external skills either accumulate as persistent guidance or internalized into the policy, eventually leading to zero-skill inference. We argue this assumption is overly restrictive, since with limited parametric capacity and uneven marginal contribution across skills, the optimal active skill set is non-monotonic, task- and stage-dependent. In this work, we propose SLIM, a framework of dynamic Skill LIfecycle Management for agentic reinforcement learning (RL), which treats the active external skill set as a dynamic optimization variable jointly updated with policy learning. Specifically, SLIM estimates each active skill's marginal external contribution through leave-one-skill-out validation, then applies three lifecycle operations: retaining high-value skills, retiring skills whose contribution becomes negligible after sufficient exposure, and expanding the skill bank when persistent failures reveal missing capability coverage. Experiments show that SLIM outperforms the best baselines by an average of 7.1% points across ALFWorld and SearchQA. Results further indicate that policy learning and external skill retention are not mutually exclusive: some skills are absorbed into the policy, while others continue to provide external value, supporting SLIM as a more general paradigm for skill-based agentic RL.