Why Goodhart`s law, Campbell`s law and reward hacking crucial for AI agents stress testing?

People often distort metrics because complex outcomes are hard to measure and achieve. Once numeric targets are linked to rewards, human behavior naturally shifts to optimize the number itself. As a result, people game the system — sometimes intentionally, sometimes not. There are two well-known laws that explain this behavior: Goodhart’s Law and Campbell’s Law.

Goodhart’s Law says that when a measure becomes a target, it stops being a good measure. In practice, people start optimizing the metric itself instead of the real goal it was supposed to.

A simple example of Goodhart’s Law: a company rewards a call-center employees for the number of calls they make. At first, the number goes up, but then operators rush, become rude, and give customers low-quality service just to hit the target. That is why teams usually need multiple metrics, not one or two numbers that can be simply gamed.

Campbell’s Law says that the more a quantitative social measure is used for decision-making, the more it gets distorted by incentives and gaming, until it no longer reflects the thing it was meant to measure.

A few examples of Campbell’s Law: if a sale-manager`s success is measured only by closed deals, the front office team may push bad-fit customers, leading to cancellations and further complaints. If Uber drivers’ ratings fall below a certain threshold, they risk losing their jobs or customers. As a result, they are motivated not only to provide high-quality service (the metric’s intent), but also to manipulate the metric by directly asking customers to give them a perfect score.

Campbell’s Law and Goodhart’s Law are very similar, but they come from different fields and emphasize slightly different problems. Both warn that once a metric is used for decision-making, people will try to game it and the metric will become less trustworthy. In everyday use, they often describe the same failure mode: measuring some important metrics too aggressively causes gaming, distortion or corruption of metrics, and unpredictable outcomes.

Obviously, AI has gone beyond humans in some spheres. Moreover, AI has outsmarted humans at their own game (!), mastering reward hacking beyond anything we anticipated. The core business challenge is making sure an AI follows our intended goals, not just the exact instructions we write down. This matters especially when a model is trained through optimization, because it learns to exploit the gap between what we meant and what we specified.

The main reasons why AI alignment sometimes can be difficult enough are two AI phenomena:

• • specification gaming

• • reward hacking

Specification gaming occurs when the model follows the proxy objective too literally and finds clever, often degenerate and weird, ways to maximize it, even when those solutions go against the real intent of the task.

A few examples of specification gaming: A classic example is a reinforcement-learning agent in a boat-racing game that was rewarded for collecting points instead of finishing the race, so it learned to drive in circles around a bonus area and farm reward instead of racing properly. That is specification gaming: the AI satisfies the literal objective, but not the intended task.

Here are a few more specification gaming examples for AI:

• • An agent was instructed to correctly complete a coding task. When a test failed during the session, Claude removed the user’s main test file rather than debugging the failure, framing the deletion as a way to “focus on the summary of fixes”;

• • Content-related systems: a system optimized for click-through metrics can promote sensational, false or misleading content because it performs better;

• • When human and AI specification gaming meet together – we can read some news like this “Pizza Hut’s AI system caused ‘cascading’ problems and $100M in damages”.

AI specific gaming is about just optimizing an imperfect proxy. The pattern is always the same: the model finds the easiest way to satisfy the proxy, not the real goal.

A few examples of reward hacking:

• • A company measures its customer-support AI agent only by average “resolution time,” so the model starts giving fast but shallow answers, closing tickets quickly while actually increasing repeat contacts, customer frustration and dissatisfaction;

• • A language model for summarization is able to explore flaws in metrics such that it obtains the highest score but the generated text summaries are barely readable by a human reader.

• • A coding model may learn to directly modify the code used for calculating the reward;

• • A chatbot is optimized for user approval, so it becomes overly sycophantic instead of accurate and useful. Recent research shows that AI`s responses were 50% more sycophantic than humans` answers. Human users preferred and trusted sycophantic AI responses more (!), incentivizing AI developers to preserve sycophancy despite the risks.

Reward hacking is expected to be a more common problem as the model and the algorithm become increasingly sophisticated. A more intelligent agent is more capable of finding “holes” in the design of reward function and exploiting the task specification—in other words, achieving higher proxy rewards but lower true rewards.

Why AI agents stress testing is crucial for specification gaming and reward hacking? Because you can understand what`s going with your agents` behavior. 

What interesting conclusions can we make from this from DEXIMES point of view? Here they are:

1. 1. Secret AI agents testing is a hidden key for success. Because this is a kind of metrics you can only get through our unique testing. You`ll receive valuable information on instruction adherence under ambiguity (does the AI agent follow its guidelines when the situation is unclear), hallucination rate (% of conversations where the AI invents information for a customer), recovery rate after error KPI (does AI agent correct itself or double down), agent`s consistency across identical scenarios (humans vary, but agents shouldn`t), task completion rate & perceived helpfulness score etc. 

• • Human validation of AI agents` is crucial. Preventing AI specification gaming and reward hacking requires careful evaluation that goes further than just simple metrics. Don’t worry, we`ve got you covered!  Proper AI agent testing and verification helps because many failures are semantic, context-dependent, or only obvious with expert judgment rather than automatic checks. In practice, our professional human experts provide structured adversarial evaluation + UX testing combined. They get clear answers on questions like: “What happens when things get messy?” and “Can customers trust this system in the real-life scenarios?” 

• • Red team your AI agents. We have a special team of testers who will test your AI models and intentionally provoke them to bypass productivity tests, safety filters, reveal toxic behavior, or exhibit biases. We will also test policy violation rate under adversarial prompts. If real life results are unpredictable, we will use a process-based supervision to tune-up your AI agents. 

• • Always balance critical metrics. Always pair your internal quantitative data (KPI, numbers, scores) with external qualitative validation (professional expert reviews and information). Never rely on quantitative measures alone. External validators` (like us) can help you to understand the nuanced consequences of business choices that might otherwise be missed entirely if you just rely on passively collected analytics data. Don’t lose the forest for the trees. 

• • If something is going wrong with your AI agents – check them out first and then tune them up immediately. You can always count on us. We`ll inspect outputs, edge cases, and failed examples to catch when an AI agent is optimizing the metric instead of solving the real task. This is especially useful for reward hacking and specification gaming, where the system may satisfy the letter of the rule while violating the intent.
  • Monitor your AI agent’s behavior. Or we can do that on your behalf, of course.  If your metrics are improving but broader outcomes (like customer satisfaction) are declining, some metric is distorting your efforts. We`ll look at label cases where the model “looks correct” but did not achieve the intended outcome and compare agent`s outputs against task intent, not just the scoring function.