The Steerability Challenge


The steerability challenge is an open competition in steering large language models toward decreased dishonesty while maintaining general capabilities. Competition entries take the form of steering interventions, which in general can be any combination of control over the model's input/prompt, structure/weights, state/activations, and output/decoding procedure. Participants will build and evaluate their interventions with the steerability toolkit.

Register competition guide
close
Nov 21, 2026 AoE
platform
Steerability + Hugging Face
awards
cash prizes (for 🥇🥈🥉 place)
workshop
NeurIPS 2026 (Paris)

The task

Dishonesty, i.e., a model's tendency to make assertions that diverge from what it internally represents as true, persists even in state-of-the-art models. Steering a model to be more honest is challenging due to the tight coupling of dishonesty with other (desirable) behavioral traits. The task of the competition is to design a steering pipeline (an ordered composition of steering interventions) to maximally suppress a model's dishonest behavior while maintaining its general capabilities.

Submissions will be scored based on a composite score of dishonesty reduction relative to the unsteered model minus the mean regression across side-effects (as measured by general capability benchmarks). Only regressions are penalized; incidental capability gains are permitted but will not be rewarded. The specific measures of dishonesty and general capabilities will be kept private to prevent gaming.

A collection of baseline steering pipelines will be provided at competition launch. A starter kit to help participants familiarize themselves with the code will also be made available. Winning teams will be awarded with a cash prize and will be invited to present at the competition workshop at NeurIPS 2026. Select teams will also be given the opportunity for their pipelines to be incorporated into the toolkit as community-contributed examples for dishonesty reduction.

Rules

  1. Teams can consist of anywhere from single individuals up to five people. No individual can be on more than one team.
  2. Steering pipelines must be submitted in the standardized .spipe format to enable uniform evaluation.
  3. External datasets and pretrained artifacts (steering vectors, adapters) are welcome if they are participant-created or freely accessible, with documentation and provenance disclosed upon request.
  4. Prize recipients must be able to supply full reproduction materials upon request. Additionally, at least one member of each winning team must be able to present (either in-person or virtually) at the competition workshop at NeurIPS 2026 in Paris.
  5. Participants who wish to have their submission added to the core toolkit repo must agree to license their work under Apache 2.0.

Questions

Who can participate?
We welcome participation from students, researchers, industry teams and hobbyists anywhere in the world. Anyone may enter and appear in the results. Cash prizes can only be paid where this is permitted under EU and US sanctions law and where we are able to make a bank transfer to the winner; if a winner cannot be paid, the prize passes to the next eligible entry.
Is there a registration deadline?
No, we welcome participation throughout the competition window. The only deadline is the final submission date of Nov 21 AoE.
Which model will I be steering?
Given that we are scoring the performance of the steering pipeline, rather than the performance of a steered model, we will be running steering on a small set of models announced at competition launch.
Am I allowed to use coding agents?
We allow for the use of coding agents and have included an AGENTS.md file in the toolkit to help with AI-assisted development. That said, participants must be able to fully describe their solutions in order be eligible for prizes.
How are leaderboard scores computed?
The public leaderboard runs on a downsampled subset of the evaluation; final rankings use the full private suite, including held-out steering tasks. Winners are re-run end to end and code-reviewed.
What happens to submissions afterwards?
You retain ownership of your submission. Apache 2.0 licensing is only required if you want your work added to the core toolkit repo. We strongly encourage public release either way, and the full evaluation suite will be published post-competition for reproducibility.

Updates

Sign up to our (low-volume) mailing list to learn more about the launch, office hours, the midpoint checkpoint, and results.

Organizers