The paper argues that sparse autoencoders may not be bad steering tools after all， and much of the earlier failure may have come from choosing and naming the wrong features.

The problem is that earlier work made sparse autoencoders look weak because their features were labelled in a way that may not match what those features actually cause inside the model.

A sparse autoencoder is a small helper model that breaks an LLM's hidden activity into many possible "features，" such as a topic， style， or concept.

So a sparse autoencoder finds directions inside a model， but an unnamed direction is not yet a usable control knob.

The authors replace vague or inherited labels with a supervised pipeline that asks whether one feature's activity reliably tracks a real label in data.

As to the mechanism， if a feature fires on "alcohol，" and forcing that feature upward makes the model talk about alcohol， the label is no longer just descriptive； it has causal weight.

The paper also finds that very high sparsity may not be necessary， meaning the feature does not need to be extremely rare to be useful for steering.

Also to note here， both prompting and feature steering are ways to push an LLM toward a desired behavior.

Prompting remains stronger because the model was trained to obey prompts， while feature steering is more like pressing directly on the machinery and hoping the rest stays intact. Prompting says "write about alcohol" in the input； feature steering instead turns up the model's internal "alcohol-related" feature and sees whether the output changes in that direction.

----

Link - arxiv. org/abs/2605.31183

Title： "Steering LLMs？ Actually， Sparse Autoencoders can outperform simple baselines"