CapVector:在参数空间中为视觉-语言-动作模型学习可迁移的能力向量
本文提出CapVector方法,以解决预训练视觉-语言-动作模型在标准微调中性能提升有限的问题。该方法将辅助目标微调的两个核心目标——增强通用能力与拟合任务特定分布——在参数空间进行解耦。仅需在小规模任务集上使用两种策略训练至收敛,所得两模型间的参数差值即构成“能力向量”。该向量与预训练参数合并后,能形成能力增强的元模型。实验表明,结合轻量正交正则化的标准微调,能以更低计算成本达到与辅助微调基线相当的性能,且所得向量在不同模型与新环境中均表现出有效性和泛化能力。
This paper proposes a novel approach to address the challenge that pretrained VLA models often fail to effectively improve performance and reduce adaptation costs during standard supervised finetuning (SFT). Some advanced finetuning methods with auxiliary training objectives can improve performance and reduce the number of convergence steps. However, they typically incur significant computational overhead due to the additional losses from auxiliary objectives. To simultaneously achieve the enhanced capabilities of auxiliary training with the simplicity of standard SFT, we decouple the two objectives of auxiliary-objective SFT within the parameter space, namely, enhancing general capabilities and fitting task-specific action distributions. To deliver the goal, we only need to train the model to converge on a small-scale task set using two distinct training strategies, resulting in two finetuned models. The parameters' difference between the two models can then be interpreted as capability vectors provided by auxiliary objectives. These vectors are then merged with pretrained parameters to form a capability-enhanced meta model. Moreover, when standard SFT is augmented with a lightweight orthogonal regularization loss, the merged model attains performance comparable to auxiliary finetuned baselines with reduced computational overhead. Internal and external experiments demonstrate that our capability vectors (1) are effective and versatile across diverse models, (2) can generalize to novel environments and embodiments out of the box.