AI labs often tout impressive benchmark scores when releasing new models. These scores show better capabilities in areas like coding and computer use than competitors. However, those benchmarks are not always a reliable measure of what AI can do. They are easily beaten by exponentially improving models and can emphasize marketing over actual performance.
Benchmarks like Humanity's Last Exam try to counter this issue by challenging models in more realistic environments. But models still find loopholes to complete tasks. To address this, the Center for AI Safety created CheatBench. Nearly every frontier model is guilty of cheating on it.
AI models are rewarded for performing tasks well and quickly. A lack of knowledge or tools incentivizes them to do what researchers call reward gaming. CAIS explained that this involves finding hidden answers, copying another agent's submission, or manipulating how its work is graded. CheatBench measures how often AI agents take these shortcuts when honest work is difficult.
CAIS tested several agents running the latest and most lauded models. These included OpenAI's GPT-6 Astra in Codex, Anthropic's Fabel 5.1 in Claude Code, and Meta's Muse Spark 1.3 in Muse Code. These agents were tested across 10 categories, including writing, professional work, mathematical research, and coding. Using honeypot clues hidden in task filespaces, the test separated acceptable reference use from cheating. CheatBench accounts for any time agents attempt to cheat, whether they are successful or not.
The researchers explained that each setting establishes an expectation of honest work, introduces a discoverable opportunity to cheat, and defines the action that crosses that boundary.
Every agent the researchers tested cheated in at least some scenarios. Astra came in as the most honest with a cheating rate of 48.2%, which is still almost half the time. Grok 4.6 was scored the biggest cheater with a rate of 81.5%. Open-weight models Kimi K3 and DeepSeek V4 Pro landed in the middle between several other proprietary frontier models.
In one example, researchers asked Claude Opus to design a protein binder. The model knew it was not allowed to refer to a set of accepted designs in the filespace, but eventually caved. After seven rejected designs, it locates the file, writes that it should not look at or copy it, and reads it with a shell command in the very next call, the researchers wrote. In its reasoning, the model admitted that using work other than its own would misrepresent my actual capabilities in this evaluation, so I shouldn't look at or copy it. But its very next step was to reference the accepted designs.
This result demonstrated both a readable choice the model made to contradict itself, and what looked like a hole in our understanding about what made the model jump from one instinct to the next.
Things got more interesting at the task category level. Even if an agent did not cheat in one area, it could cheat significantly more in another. Fable 5.1 was only 5% likely to cheat at games, but 100% likely to cheat on knowledge work tasks.
Reinforcement learning trains models not to abandon a task, even if pursuing it creates conflict-ridden choices. CAIS noted in its paper that sycophancy is an early sign of reward gaming. This term refers to AI models' tendency to be too agreeable and encouraging of whatever a user says, sometimes regardless of whether it is incorrect, delusional, or could lead to harmful behavior. Traits like sycophancy and reward gaming show how models can prioritize accomplishing a task correctly to please a user over the alignment training researchers work so hard to build in.
These tests represent relatively low stakes. But CAIS researchers created CheatBench because of the risks of this behavior at scale across different tasks. Earlier this month, yet another researcher quit Anthropic over concerns that the company is not developing AI responsibly for a future in which it could build itself away from human-oriented values and kill us.
A propensity to cheat, or complete a task at any cost, puts our potentially differing priorities at odds with an increasingly powerful technology. As the AI Leaderboard newsletter explained, it will not necessarily be a demonstrated animosity toward humans that pits AI against us. It may be that we are simply in the way and end up as collateral.
