原文 · 未翻译
Trajectory’s concurrent multi-LoRA stack reports a 2.81× experiment-throughput gain over single-tenant RL, with all code in the NovaSky-AI/SkyRL GitHub repository.
Most language models improve in discontinuous jumps. A team collects data, trains, and ships a new version. This takes months and produces remarkable or catastrophic behavior for users. Trajectory wants to replace that cycle with continual learning.
The Trajectory team published a field report describing how. It built a concurrent, multi-LoRA training platform for continuously learning workloads. The work was done with UC Berkeley Sky Lab and Anyscale. All training code is open-sourced in the NovaSky-AI/SkyRL repository.
The result is a 2.81× end-to-end experiment-throughput improvement. The comparison is against a single-tenant training framework. Trajectory reports no regression on any training rewards.
What Multi-LoRA Training Actually Is
Continual learning requires models to update from live feedback and production interactions. A coding agent could learn engineering patterns as developers correct its work. A support agent could resolve hard tickets as operators intervene on difficult cases.
Most training infrastructure still assumes a linear lifecycle. Teams allocate GPUs, initialize the model, run a job, then spin down. Continual learning revises that relationship. When production interactions become training inputs, training becomes part of a live system.
Modern RL training reduces to three core primitives. The Sampler generates trajectories from the current policy model. The Trainer computes gradients and updates the policy weights. Parameter synchronization broadcasts updated weights back to inference workers.
Trajectory calls its approach Continuous Multi-LoRA Training, or C-LoRA. Each experiment maps to a dedicated LoRA adapter on a warm, multi-tenant engine.
The Problems It Targets
The Trajectory team identifies four inefficiencies in traditional stacks:
(1) Cold starts are slow: Every serial job reloads checkpoints, initializes the distributed runtime, and warms inference engines. For large models, this step alone can exceed 30 minutes per run.
(2) RL is memory intensive: Frontier models often exceed 100B parameters. Qwen3.5-397B can require up to eight H200 nodes to fit into memory. LoRA cuts memory usage by an order of magnitude. It freezes the base model and trains only small adapter weights.
(3) Traditional stacks are single-tenant: They run one experiment at a time. Multi-LoRA maps each experiment to one adapter, multiplexing throughput by a factor of N.
(4) Job utilization is low: Trainers and inference engines stall while waiting for each other. Multi-LoRA load balances across jobs to fill idle capacity.
Inside the Architecture
Most throughput wins come from inference. In vLLM, all adapters are hot-loaded in GPU memory. Decode steps can then mix tokens from different adapters in the same batch. The key enabler is the SGMV decode kernel. It fuses per-adapter matrix-vector work into one GPU launch per decode step.
After each optimization step, updated LoRA weights load in-place into the inference engine. The scheduler does not freeze, so other tenants keep decoding.
Training works differently. One active LoRA adapter trains on the GPU. The rest sit in pinned CPU memory. Each tenant’s state lives in an AdapterStore. It holds LoRA parameters, FP32 master weights, optimizer moments, and gradient buffers.
The engine swaps one tenant’s state onto the GPU, runs a single forward_backward pass, then swaps it back. This training path is still single-adapter. The inference concurrency gains do not yet apply to training.
The Numbers
Trajectory tested on a single H200 node with Qwen3-4B-Instruct-2507. It ran sync RL on GSM8K in an agentic setting. The Trajectory team reframed GSM8K as a tool use learning task. The model decides when to call a Calculator and a Final Answer tool. Reward is 1.0 only when Final Answer is called with the correct answer.
The policy starts near 40% accuracy at step 0. With the right learning algorithm, it climbs past 90% by step 9.
The Trajectory team scaled to eight concurrent multi-LoRA runs. Final Experiment Time hit 5433s at N=8, a 2.81× speedup. Eight concurrent experiments finished before three serial runs back-to-back. Mean Experiment Time also improved, peaking at N=4 with a 1.88× speedup. Every concurrency level reached reward_accuracy above 90% by step 9.
The Tradeoffs
Higher throughput costs per-step latency. As N grows, First Experiment Time and Step Time degrade. At N=8, the first serial experiment finishes 1.97× faster. Mean step time rises from 191s to 500s, only 2.62× slower.
Most of that increase is rollout time. Rollout grows from 162s to 401s, roughly 77% of the increase. At N=2, doubling the load adds only 15% rollout time. That is the ideal case for multi-LoRA.
The pattern held on a harder workload. On τ-bench retail with the NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 MoE model, N=2 finished 10 steps 1.28× faster. Per-tenant step time rose 1.57×.
Strengths and Weaknesses
Strengths:
2.81× end-to-end experiment-throughput gain at eight concurrent runs
No accuracy regression; runs tracked the serial baseline within ±1σ in the final steps
LoRA cuts memory by an order of magnitude versus full fine-tuning
Fully open-sourced in NovaSky-AI/SkyRL for the community to build on
Weaknesses:
Per-step latency and First Experiment Time degrade as N grows
Training remains serialized across tenants; only inference is multiplexed
Tested mainly on mid-sized models, not frontier-scale parameters
Setup requires an 8× H100/H200 node and a Megatron build
Key Takeaways
Trajectory built a concurrent, multi-LoRA RL training stack for continual learning, open-sourced in NovaSky-AI/SkyRL.
It reports a 2.81× end-to-end experiment-throughput gain over a single-tenant baseline, with no reward regression.
Each experiment maps to a dedicated LoRA adapter on an always-hot engine, multiplexing throughput by N.
Most gains come from vLLM multi-LoRA inference via the SGMV decode kernel; training stays single-adapter.
The tradeoff is per-step latency: at N=8, step time rises from 191s to 500s.
Marktechpost’s Visual Explainer
Field Report · May 27, 2026
Continuous Multi-LoRA Training for Continual Learning
Trajectory, built with UC Berkeley Sky Lab and Anyscale.
2.81× end-to-end experiment-throughput gain
Training code open-sourced in the NovaSky-AI/SkyRL repository.
01 — What it is
One always-hot engine, many adapters
Continual learning updates models from live feedback and production interactions.
Trajectory calls its approach Continuous Multi-LoRA Training (C-LoRA). Each experiment maps to a dedicated LoRA adapter on a warm, multi-tenant engine.
Sampler
Generates trajectories from the current policy model.
Trainer
Computes gradients and updates the policy weights.
Parameter sync
Broadcasts updated weights back to inference workers.
The shift
Training becomes part of a live, distributed service.
02 — The problems it targets
Four inefficiencies in serial RL stacks
Slow cold starts
Each job reloads checkpoints and warms engines. This can exceed 30 minutes per run.
Memory-intensive RL
Qwen3.5-397B can need up to eight H200 nodes. LoRA cuts memory by an order of magnitude.
Single-tenant
One experiment runs at a time. Multi-LoRA multiplexes throughput by a factor of N.
Low utilization
Trainer and inference engine stall waiting for each other. Multi-LoRA fills idle capacity.
03 — Inside the architecture
Where the throughput comes from
Inference. In vLLM, all adapters are hot-loaded in GPU memory. The SGMV decode kernel fuses per-adapter work into one GPU launch per decode step.
Weight sync. Updated LoRA weights load in-place. The scheduler does not freeze, so other tenants keep decoding.
Training. One active adapter trains on the GPU; the rest sit in pinned CPU memory.
AdapterStore
Each tenant’s state holds LoRA parameters, FP32 master weights, optimizer moments, and gradient buffers. This path is still single-adapter.
04 — The setup
GSM8K, reframed as a tool-use task
Tested on a single H200 node with Qwen3-4B-Instruct-2507, running sync RL on GSM8K in an agentic setting.
The model decides when to call a Calculator and a Final Answer tool.
Reward is 1.0 only when Final Answer is called with the correct answer.
The policy starts near 40% accuracy and climbs past 90% by step 9.
05 — The numbers
2.81× throughput, no reward regression
2.81×
Final Experiment Time at N=8 (5433s)
1.88×
Mean Experiment Time, peaking at N=4
>90%
reward_accuracy at every level by step 9
Eight concurrent experiments finished before three serial runs back-to-back. Runs tracked the serial baseline within ±1σ in the final steps.
06 — The tradeoffs
Throughput up, per-step latency up
At N=8, mean step time rises from 191s to 500s, 2.62× slower.
Rollout grows from 162s to 401s, roughly 77% of the increase.
At N=2, doubling the load adds only 15% rollout time — the ideal case.
Harder workload check
On τ-bench retail with the NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 MoE model, N=2 finished 10 steps 1.28× faster; per-tenant step time rose 1.57×.
07 — Takeaways
What to remember
Concurrent multi-LoRA RL training for continual learning, open-sourced in NovaSky-AI/SkyRL.
2.81× end-to-end experiment-throughput gain over a single-tenant baseline.
Most gains come from vLLM multi-LoRA inference; training stays single-adapter.
SkyRL implements the Tinker API; reproduce on 8× H100/H200 with the Tinker cookbook.
‹
›
Where (Inferences) to Run
Run it / Access the model
Inference & compute providers
Where to access the Qwen3-4B-Instruct-2507 base model, the SkyRL training stack, and the NVIDIA GPUs used in the experiments.
Featured Nebius Token Factory Managed inference and LoRA fine-tuning on NVIDIA GPUs, plus GPU-on-demand to reproduce the 8×H100/H200 setup.
Explore Nebius →
Hugging FaceOpen weights
Download the Qwen3-4B-Instruct-2507 base model used across the runs. huggingface.co →
NovaSky-AI / SkyRLOpen source
The full multi-LoRA RL training stack, open-sourced for reproduction. github.com →
Fireworks AIServerless API
Hosted Qwen3-4B-Instruct-2507 endpoint with a playground to test it. fireworks.ai →
OpenRouterUnified API
Route to Qwen3 models through one OpenAI-compatible endpoint. openrouter.ai →
LambdaGPU cloud
On-demand NVIDIA H100 and H200 instances to run the stack yourself. lambda.ai →
Destinations verified May 30, 2026. Sources: Nebius Token Factory, Hugging Face, SkyRL, Fireworks AI, OpenRouter, Lambda. Availability and pricing may change.
Check out the Repo and Technical Details. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
The post Trajectory Releases a Concurrent Multi-LoRA Training Stack for Continual Learning, Reporting a 2.81× Experiment-Throughput Gain appeared first on MarkTechPost.