The Steerability Challenge


The steerability challenge is an open competition in steering large language models toward a specific target behavior while minimizing off-target effects. Specifically, the focus is to reduce model sycophancy while maintaining the model's general capabilities. Competition entries take the form of steering interventions, which in general can be any combination of control over the model's input/prompt, structure/weights, state/activations, and output/decoding procedure. Participants will build and evaluate their interventions with the steerability toolkit.

Register starter pack
open
Aug 15, 2026
close
Nov 15, 2026 AoE
platform
Steerability + CodaBench
format
steering pipeline (.spipe file)
awards
cash prizes (for 🥇🥈🥉 place)

The task

Sycophancy, i.e., a model's tendency to align with what the user appears to want to hear rather than with what is accurate, persists even in state-of-the-art models. Steering a model to be less sycophantic is challenging due to its tight coupling with other (desirable) behavioral traits. The task of the competition is to design a steering pipeline (an ordered composition of steering interventions) to maximally suppress a model's sycophantic behavior while maintaining its general capabilities. Visual interfaces (which we term workbenches) to assist in the construction of steering pipelines will be made available to participants in an effort to lower the barrier of entry.

Submissions will be scored based on a composite score of sycophancy reduction relative to the unsteered model minus the mean regression across side-effects (as measured by general capability benchmarks). Only regressions are penalized; incidental capability gains are permitted but will not be rewarded. The specific measures of sycophancy and general capabilities will be kept private to prevent gaming. Additionally, the splits used to maintain the leaderboard throughout the competition will be an intentionally small portion of the final evaluation split.

Upon competition launch, a collection of baseline steering pipelines will be provided. A starter kit to help participants get started will also be made available. Winning teams will be awarded with a cash prize and will be invited to present at the competition workshop at NeurIPS 2026. Select teams will also be given the opportunity for their pipelines to be incorporated into the toolkit as community-contributed examples for sycophancy reduction.

Rules

  1. One account per person (verified via institutional email if available + ORCID ID). Teams can consist of anywhere from single individuals up to five people.
  2. Each team is limited to one scored submission in any rolling 24-hour window.
  3. Each team designates exactly one entry for final evaluation.
  4. All steering policies must be submitted in the toolkit's standardized .spipe format to enable uniform evaluation.
  5. External datasets and pretrained artifacts (steering vectors, adapters) are welcome if they are participant-created or freely accessible, with documentation and provenance disclosed upon request.
  6. Prize recipients must be able to supply full reproduction materials within 14 days of final submission. Additionally, at least one member of each winning team must be able to physically attend and present at the competition workshop.
  7. Participants who wish to have their submission added to the core toolkit repo must agree to license their work under Apache 2.0.

Questions

Who can participate?
We welcome participation from students, researchers, industry teams, and hobbyists. Prize eligibility is subject to local laws and geographic restrictions.
Is there a registration deadline?
No, we welcome participation throughout the competition window. The only deadline is the final submission date of Nov 15 AoE.
Is there an entry fee?
No, entry is free-of-charge.
Which model will I be steering?
Given that we are scoring the performance of the steering pipeline, rather than the performance of a steered model, we will be running steering on a small set of models announced at competition launch.
Do I need to know how to code?
No, visual interfaces (workbenches) will be made available to facilitate easier construction of steering policies.
How is the leaderboard kept honest?
The public leaderboard runs on a downsampled subset of the evaluation; final rankings use the full private suite, including held-out steering tasks. Winners are re-run end to end and code-reviewed.
What happens to submissions afterwards?
You retain ownership of your submission. Apache 2.0 licensing is only required if you want your work added to the core toolkit repo. We strongly encourage public release either way, and the full evaluation suite will be published post-competition for reproducibility.

Updates

Sign up to our (low-volume) mailing list to learn more about the launch, office hours, the midpoint checkpoint, and results.

Organizers