skip to content
NeurIPS 2026 Competition:
The Steerability Challenge Competition window: late July – mid-October 2026 Workshop: Dec 11–12, 2026; NeurIPS, Sydney

Welcome!

The competition has not yet launched. Registration, the submission portal, the designated base model, and the prize breakdown will be announced on this page when the competition opens (expected late July 2026).

The Steerability Challenge is a NeurIPS 2026 competition on steering large language models: making a targeted, deliberate change to a model's behavior while leaving everything else intact. Participants design steering policies, compositions of interventions spanning prompting, weight modification (e.g., LoRA adapters), activation steering, and decoding-time controls, that are evaluated against a common base model.

This year's target behavior is sycophancy: the tendency of a model to respond with what the user appears to want to hear rather than what is accurate. Sycophancy remains a persistent failure mode even in state-of-the-art systems and can cause real harm in advisory settings (e.g., medicine, law, finance). Your goal is to reduce it as much as possible, without degrading the model's general capabilities.

That second clause is the heart of the challenge. Steering a model toward a target behavior is well known to cause degradation along other dimensions, and the side-effects of composed interventions (how steering is typically deployed in practice) remain largely unstudied. By scoring submissions on both steering effectiveness and capability retention, the competition aims to surface which interventions, and which combinations of interventions, control model behavior reliably.

All submissions are built with AI Steerability 360, an open-source toolkit that organizes steering methods across four control surfaces (input, structural, state, and output) and supports their composition into steering policies. A starter kit, including baseline policies, workbench interfaces for constructing steering artifacts, and a local evaluation harness, will be released ahead of launch.

The competition will be hosted on CodaBench and will offer both monetary incentives (compute credits and financial awards, subject to sponsorship) and non-monetary incentives (co-authorship on the competition summary paper for top-performing teams). Questions can be raised on the competition forum, or via the Discord channel and contact email announced at launch; bug reports and feature requests for the toolkit go through GitHub issues.

Task and evaluation

What you submit. A submission consists of a steering policy and documentation describing its design. In the toolkit, a steering policy is a structured piece of code that references steering artifacts across the four control categories, applied to the model in the order they appear: input controls (e.g., system prompts, few-shot examples), structural controls (e.g., LoRA adapters, weight editing), state controls (e.g., activation steering vectors, attention manipulation), and output controls (e.g., logit manipulation, constrained decoding, custom sampling). A policy may use a single control or compose several. Organizers execute all submitted policies against the designated base model on identical infrastructure.

How submissions are scored. Each policy receives a composite score balancing two criteria: steering effectiveness, the reduction in sycophantic behavior relative to the unsteered base model, and capability retention, the degree to which general capabilities are preserved, measured as performance change across a suite of side-effect benchmarks. Only regressions on side-effect benchmarks are penalized; incidental improvements are permitted but not rewarded, which incentivizes targeted steering rather than general capability boosting. To prevent overfitting, the precise scoring formula and aggregation weights will remain private.

What is evaluated. Submissions are assessed on sycophancy reduction (the target behavior) and on capability retention across five dimensions: general knowledge and reasoning, instruction following, multilingual understanding, tool calling, and safety. Public steering tasks and development datasets are provided for local iteration; final evaluation additionally includes held-out private tasks, benchmarks, and configurations that are not disclosed until the competition concludes. Competitive solutions therefore need to generalize rather than overfit to any particular benchmark.

Leaderboard and ranking. A public leaderboard, computed on a subset of the evaluation, will be updated periodically throughout the competition. Final rankings are determined by performance on the full private evaluation. Ties are broken first by lower intervention complexity (fewer composed controls), then by lower computational overhead at inference time; cost–capability trade-offs may additionally be taken into account (e.g., via a separate "efficient steering" leaderboard).

Baselines and starter kit. The starter kit contains the toolkit with documentation and installation scripts, workbench interfaces for constructing steering artifacts, a set of baseline steering policies, and a development evaluation harness that runs locally against public sycophancy datasets. Baselines span each control surface plus a composition: (1) a system prompt instructing the model to maintain its position under user pressure; (2) a LoRA adapter fine-tuned on non-sycophantic response pairs; (3) an activation steering vector extracted from contrastive sycophantic/non-sycophantic completions; (4) a logit penalty on agreement tokens following user disagreement; and (5) the system prompt combined with the activation steering vector. Each baseline ships with its composite score on a public development split, giving a concrete target to improve upon.

Rules

The complete rules will be provided upon registration and maintained on this website; submitting an entry indicates acceptance of the rules in full. A summary of the main points:

Eligibility. Participation is open to individuals and teams of up to five members, with no institutional or geographic restrictions beyond standard export-control requirements. Each person may belong to at most one team and must register with a single account; registering multiple accounts or assuming multiple identities results in disqualification. Teams may merge prior to the team-merger deadline, after which the combined team inherits a single submission quota. Prize eligibility additionally requires that every team member is 18 or older (and has reached the legal age of majority where they reside) and has no role in organizing the competition, nor is a close relation of an organizer.

Submissions. All entries must be implemented as steering policies using the AI Steerability 360 toolkit; submissions that bypass or modify the toolkit's evaluation harness will be disqualified. A single base model, common to all participants, will be announced on this website before the competition opens; it may not be substituted, replaced, or ensembled with other models. Policies may combine any interventions across the four control surfaces, provided they are implemented through the toolkit's control interfaces. Any fine-tuning or weight modification must use only publicly available or participant-created datasets, with provenance documented and disclosed upon request. Teams may make at most 5 submissions in any rolling 24-hour window (submissions are evaluated with the periodic leaderboard updates rather than immediately), and each team selects exactly one submission as its entry for final evaluation.

Originality. Entries that merely replicate or make superficial changes to the provided baselines are not eligible for prizes. Prize-qualifying entries should introduce novel steering strategies, substantive methodological improvements, or independently developed implementations that go beyond simple parameter adjustments; the organizing committee may request additional documentation or code access to assess this. Exploring and building on the baselines for learning purposes is, of course, encouraged.

External data and tools. External datasets, pretrained steering artifacts (e.g., steering vectors, LoRA adapters), and open-source libraries may be used as long as they are freely or cheaply accessible to all participants and are fully documented in the submission. Resources that are proprietary or access-restricted in a way that confers an unfair advantage are not permitted.

Winner obligations and licensing. Within 14 days of notification, prize recipients must provide all materials needed to reproduce their results (code, trained adapters or learned steering vectors, configuration files, and written documentation). All submissions, including source code and documentation, are licensed under the Apache License 2.0, consistent with the toolkit from which they derive; public release of policies is strongly encouraged. Receiving a prize is contingent on presenting the winning approach at the NeurIPS competition workshop (in person or virtually) and contributing to the post-competition retrospective publication, on which winning teams receive authorship.

Conduct and organizer authority. All participants must adhere to the NeurIPS Code of Conduct and Code of Ethics. The organizers may revise the rules, timeline, evaluation protocol, or prize structure during the competition, with material changes communicated through this website and by email to registered participants. Participants may be removed at any stage for cheating, plagiarism, collusion, or other misconduct, and organizer decisions on rule interpretation, scoring, and eligibility are final.

Schedule

Organizers