Can a fake phishing email really prove how safe your company is? Plenty of vendors want you to believe the answer is yes. Run a monthly simulation, watch the click rate drop, and call it security. But security teams who have run these programs for years know the picture is messier. Simulations are useful tools, yet they come with blind spots that can mislead leaders, frustrate employees, and even weaken the security culture they are meant to strengthen. This article walks through the real limitations of phishing simulations, where the metrics fall short, and how to use them without fooling yourself. (Teach Employees to Avoid Phishing)
What Phishing Simulations Actually Measure
A phishing simulation sends a fake but realistic email to employees and records who clicks, who enters credentials, and who reports it. On paper, that sounds like a direct test of human defenses. In practice, it measures behavior under artificial conditions.
The first limitation is context. Real attackers choose timing, targeting, and pretext with care. They know when finance closes the quarter, when IT rolls out a new system, or when a new hire is still learning the ropes. A simulation built from a template rarely captures that precision. So a low click rate in a calm month says little about how people will react during a genuine, well-crafted attack.
There is also the awareness problem. Once employees know simulations exist, some treat every email as a test. That changes the behavior being measured. You are no longer observing natural reactions; you are observing people who suspect they are being watched.
The Click-Rate Metric Is Weaker Than It Looks
Click rate is the headline number in almost every simulation report. It is also one of the least reliable indicators of real risk.
Reported average click rates vary enormously across studies and vendors, from single digits to well over 30 percent, depending on the population, the difficulty of the lure, and how “click” is defined. That spread makes cross-company comparison close to meaningless. A 5 percent click rate at one firm and a 15 percent rate at another may reflect different simulation designs rather than different security cultures.
The distribution matters more than the average. A small group of repeat clickers often accounts for a disproportionate share of clicks. When that is true, an aggregate rate hides a skewed reality. A company might report a healthy 8 percent click rate while a handful of people in sensitive roles click almost every time. Fixing the average does nothing if the same people keep failing.
Finally, clicking is not the same as compromising. Someone who clicks a link but stops before entering credentials has behaved differently from someone who hands over a password. Collapsing both into one number throws away the most useful signal.
Reporting Rate Deserves More Attention Than It Gets
Many practitioners argue that reporting rate is a better signal than click rate. It measures detection and response rather than just failure. An employee who spots a suspicious email and flags it gives the security team early warning, which is exactly what happens in a real attack.
| Metric | What it captures | Common weakness |
|---|---|---|
| Click rate | How many people interacted with the lure | Hides repeat clickers and context |
| Credential submission rate | How many entered data | Depends heavily on lure quality |
| Reporting rate | How many detected and flagged the email | Can be inflated by easy lures |
| Time to first report | Speed of human detection | Rarely tracked consistently |
If a program only celebrates a falling click rate and ignores reporting, it rewards silence. Employees learn that the safest move is to touch nothing and say nothing, which is the opposite of what you want when a real attack lands.
Simulations Can Damage Trust and Culture
Here is where theory meets the hallway conversation. Simulations that mimic internal communications, fake urgent requests from managers, or imitate HR announcements can feel like traps. When people are shamed for clicking, or when results are used punitively, the exercise breeds resentment rather than vigilance.
The UK’s National Cyber Security Centre has cautioned that poorly designed simulations can be counterproductive, particularly when they are punitive or shaming. That warning lines up with what many security awareness leads observe: fear-based programs produce anxious employees who report less, not more.
Practical tip: never tie simulation results to discipline. Treat clicks as data about the program, not about the person. Announce the program openly, explain why it exists, and make reporting frictionless. A one-click report button in the mail client does more for detection than a dozen clever lures.
Measurement Validity: Why Results Don’t Always Generalize
A simulation is a model, and like any model it can drift from reality. Attackers use timing, targeting, and pretext quality that templates rarely match. They also escalate: a first email builds trust, a second exploits it. Most simulation programs test single emails in isolation, which misses the multi-stage nature of real campaigns.
There is also the novelty effect. New programs often show quick improvement as awareness spikes, then plateau or decay. Short-term gains in click rates do not automatically translate into durable behavior change. Some studies find improvement that fades over time, which means a program judged on its first quarter may look far better than it is.
The lesson is to treat simulation data as one input among several. Pair it with real-world reporting data, help-desk tickets about suspicious emails, and results from other exercises. No single number tells the whole story.
Vendor Benchmarks and the Comparison Trap
Annual phishing benchmark reports are everywhere, and they are tempting to cite in board decks. But this data is usually proprietary, and the methodology is often opaque. It can be useful for spotting broad trends, yet it is weak as evidence that your program is working.
If a vendor tells you the industry average click rate is X, ask how the sample was built, what counted as a click, and whether the lures were standardized. Without those answers, the number is marketing, not measurement. The same skepticism applies to claims that a specific tool “reduces risk by 90 percent.” Risk reduction claims need methodology, not adjectives.
Regulatory and Standards Context
Security awareness training sits inside a broader framework of standards and guidance. NIST publishes material on building security awareness and training programs, and those documents are worth reading before you design simulations. The details of how far any standard endorses simulations specifically should be checked against the source documents themselves rather than taken from a vendor slide.
The practical takeaway is that simulations should sit inside a security awareness program, not replace it. Training, clear reporting channels, and leadership support do the heavy lifting. Simulations are a check on how well that foundation is working, not a substitute for it.
Ethical and HR Considerations
Simulations that imitate real internal communications can erode trust, and in some jurisdictions they require consultation with works councils or unions. This is jurisdiction-dependent, so check local rules before launching a program that fakes messages from managers or HR.
Common mistakes people make:
- Running simulations without informing employees that a program exists.
- Using results to single out or discipline individuals.
- Designing lures that mimic sensitive personal or HR topics.
- Ignoring local consultation requirements.
- Launching during a crisis or major organizational change.
What actually works in practice is transparency about the program’s existence, confidentiality about individual results, and a clear promise that clicks are used to improve training, not to punish people.
Better Ways to Use Simulations
None of this means simulations are worthless. It means they need to be used with clear eyes. Here is a practical approach that holds up.
- Define the goal before you send anything. Is it measuring detection, improving reporting, or testing a specific control?
- Track reporting rate and time to first report alongside click rate.
- Segment results to find repeat clickers and high-risk roles, then target training there.
- Vary difficulty and document lure design so results are comparable over time.
- Never punish individuals. Publish aggregate trends only.
- Combine simulation data with real-world reports and help-desk tickets.
- Review the program annually and retire lures that no longer teach anything.
This approach treats simulations as a diagnostic tool. It also pairs well with broader awareness work, such as teaching staff what phishing is and how to recognize it and explaining what spear phishing looks like when an attacker has done homework on the target. Targeted attacks are where simulations struggle most, because templates rarely match the personalization of a real spear-phishing attempt. (Phishing)
It is also worth remembering that attackers now use AI to write cleaner, more convincing messages. Awareness of AI article generator features and limitations helps employees understand why some phishing emails read better than they used to. The same goes for voice: as AI voice cloning tools and their limitations improve, phone-based social engineering becomes a channel that email-only simulations never test.
The Bottom Line
Phishing simulations are a useful mirror, not a security guarantee. They measure behavior under artificial conditions, they lean on a click-rate metric that hides as much as it reveals, and they can backfire if they feel punitive. Used well, with reporting rate in the spotlight and trust protected, they earn their place in an awareness program. Used as a scoreboard, they teach employees to fear the test rather than the attack.
Frequently Asked Questions
Are phishing simulations legally required?
No. Simulations are not generally mandated, though awareness training may be expected under various standards and frameworks. Some jurisdictions require consultation with employee representatives before running them, so check local rules.
What is a good click rate for a phishing simulation?
There is no universal benchmark. Reported averages vary widely by population and methodology, so comparing across companies is unreliable. Track your own trend over time and watch the distribution, especially repeat clickers, rather than chasing an industry number.
Can simulation results be used in performance reviews?
They should not be. Tying results to discipline or reviews damages trust and discourages reporting. Use clicks as data about the program’s effectiveness, not about individual employees.
Why do employees who pass simulations still fall for real attacks?
Simulations rarely match the timing, targeting, and personalization of real attacks. Employees may also treat simulations as tests and behave differently. Passing a template-based exercise is not proof of real-world resilience.
Is reporting rate better than click rate?
Many practitioners think so. Reporting rate measures detection and response, which is what matters during a real attack. Click rate measures failure and can hide repeat clickers. Track both, but give reporting serious weight.
How often should a company run phishing simulations?
There is no fixed rule. Many programs run monthly or quarterly, but frequency matters less than design. Vary difficulty, document lures, protect trust, and review results regularly. A well-designed quarterly program beats a careless monthly one.
The FAQ above covers the questions that come up most often, but one more deserves attention: what should a program actually do with a repeat clicker? The instinct is to escalate—more simulations, mandatory training, a note to the manager. That instinct is usually wrong. Repeat clicking often signals overload, a role that receives enormous email volume, or a reporting channel that feels cumbersome. Treat it as a signal to investigate the environment, not to single out the person. A short, blame-free conversation about what made the message convincing teaches more than another lure ever will.
The broader point is that phishing simulation is a measurement tool with real limits, and maturity means knowing those limits. Measure reporting, protect trust, document your methods, and keep simulations inside a wider awareness program. Do that, and the data you collect will mean something.
