What if the very exercise meant to make a workforce safer is quietly teaching people to distrust the security team? That is the uncomfortable question behind most failed phishing simulation programs. Fake “urgent” emails are the dominant phishing pattern, and the Federal Trade Commission has long warned businesses about credential-harvesting pages dressed up as routine notices. Yet many organizations run simulations that are technically fine but strategically broken. They measure the wrong thing, punish the wrong people, and then wonder why reporting rates stay flat. (What is spear phishing) (Phishing)
This article walks through the most common phishing simulation mistakes, why each one matters, and what actually works in practice. No vendor hype, no scare tactics, just the patterns that separate programs that change behavior from programs that generate resentment. (What is phishing and how can you recognize it) (Recognize and Report Phishing)
Mistake 1: Treating Click Rate as the Only Score That Matters
The click rate is the easiest number to report. It is also the least useful on its own. A program that tracks only clicks misses the behavior that actually limits damage: how fast and how often employees report a suspicious message. A worker who clicks but reports within two minutes is far more valuable to the organization than one who never clicks and never says anything.
Industry benchmark reports from firms such as Verizon and KnowBe4 have for years shown median simulated-phishing click rates ranging from the high single digits to roughly 30 percent, depending on sector and methodology. Those figures vary enormously by year, scenario difficulty, and how “click” is defined. Quoting a single number as if it were universal is a mistake in itself.
Practical fix: build a small scorecard instead of a single metric.
| Metric | What it tells you | Why it matters |
|---|---|---|
| Click rate | How many people interacted | Baseline only, not the goal |
| Report rate | How many flagged the message | The behavior that limits damage |
| Median time to report | Speed of response | Shrinks the attacker’s window |
| Repeat-click rate | Share of habitual clickers | Targets training where it counts |
If a program reports only one number, it should be the report rate, not the click rate.
Mistake 2: Running Simulations With No Baseline
You cannot show improvement if you never measured where you started. A surprising number of teams launch a full simulation campaign in week one, then compare month six against month one and call the difference “training impact.” That comparison is meaningless when the difficulty of the scenarios changed along the way.
The fix is unglamorous but essential: run a baseline assessment before any training, keep the scenario difficulty steady for the first few rounds, and only then introduce harder lures. If you plan to change both the training and the test difficulty at once, you lose the ability to attribute any change to either.
A related trap is comparing your organization to a vendor benchmark instead of to itself. Benchmarks are useful context, but they are not a control group. Your own trend line is the only honest measure of your program.
Mistake 3: Designing “Gotcha” Simulations That Humiliate People
There is a strong practitioner consensus, echoed across security-awareness literature, that punitive or humiliating simulations backfire. When employees are publicly shamed for clicking a fake invoice, they learn one lesson fast: don’t tell anyone when you make a mistake. That is the opposite of what you want.
The classic failure mode looks like this: a simulation lands, a handful of people click, and a leader forwards the “leaderboard” to the whole department. Reporting rates drop in the next quarter. Nobody connects the two.
What works in practice is a no-blame reporting culture. Reward fast reporting publicly. Treat a click as data, not a disciplinary event. If your HR or legal teams insist on consequences, keep them narrowly scoped and clearly communicated in advance, and never spring them on people after the fact.
This is also where realistic scenario design matters. The FTC’s guidance on phishing emphasizes that fake urgency and credential-harvesting pages are the dominant pattern, so scenarios should mirror those tactics rather than inventing exotic lures that teach nothing transferable.
Mistake 4: Ignoring the Difference Between Repeat and One-Time Clickers
Not all clicks are equal. Someone who clicks once, reports it, and completes training is a very different risk profile from someone who clicks every single round. Conflating the two inflates or deflates your results depending on the mix.
Segment your population. Track repeat clickers separately. For the one-time group, a short refresher is usually enough. For the habitual group, the problem is often not knowledge but workflow: they are clicking fast because they are drowning in email, or because the simulation mimics a tool they use daily and they have stopped reading carefully.
Practical tip: before assuming a repeat clicker needs more training, look at what they clicked. If the lure imitated an internal system they use hourly, the fix may be process design, not education.
Mistake 5: Cloning Real Internal Tools Without Authorization
This one creates operational chaos. A simulation that perfectly mimics your company’s password-reset portal or HR system can trip email-security filters, generate help-desk tickets, and in some cases trigger incident-response alerts. Worse, it can violate internal policy or, in some jurisdictions, labor agreements.
Before any campaign, run a short checklist:
- Confirm the simulation domain and sending infrastructure are allowlisted with the email security team.
- Notify the help desk so they know what is coming and when.
- Check with legal and, where relevant, employee representatives about consent and notice.
- Confirm the simulation does not harvest real credentials, even accidentally.
- Agree on a kill switch in case the campaign misfires.
Skipping step one is the most common error. A well-meaning security team once discovered its own simulation had been quarantined by its own filters, which meant half the workforce never received the test and the results were garbage.
Mistake 6: Sending Simulations Too Often, or Never
Cadence is a Goldilocks problem. Run simulations once a year and people forget the training between rounds. Run them weekly and employees start treating every email as hostile, which slows real work and breeds fatigue. The sweet spot for most organizations is monthly or quarterly, with difficulty that ramps gradually.
There is a second cadence mistake: sending simulations only during quiet periods. If every test arrives on a Tuesday morning when inboxes are calm, you are training people for a world that does not exist. Real attacks arrive during quarter-end chaos, on Friday afternoons, and the day before a holiday. Vary the timing, but keep it fair and documented.
Mistake 7: Making Training a Punishment Instead of a Path
When a click triggers a mandatory 45-minute module, people learn to avoid the module, not the phish. Short, immediate micro-learning works better: a two-minute explainer that shows exactly which cue was missed, delivered right after the click while the context is fresh.
The same logic applies to reporting. If reporting a suspicious email requires filling out a five-field form, most people will delete the message instead. A one-click report button that feeds directly into the security team removes friction and produces better data.
Mistake 8: Forgetting That Phishing Is a Family of Attacks
Basic “your package is delayed” lures are the entry level, not the whole curriculum. Spear phishing, business email compromise, and credential harvesting through fake login pages all require different detection skills. A program that only ever tests generic lures leaves people unprepared for the targeted messages that cause the most damage.
Build a scenario library that covers the range: generic lures, internal-impersonation attempts, vendor-invoice fraud, and messages that imitate cloud-service notifications. Rotate through it so no single pattern becomes the only thing people recognize.
Mistake 9: Skipping the Feedback Loop With the Security Team
Simulation data is a goldmine for the security team, and most programs barely use it. Which departments report fastest? Which lures get through email filters? Which internal tools are most frequently imitated, suggesting a process worth hardening?
A quarterly review that pairs simulation results with real reported phishing attempts turns the program from a compliance checkbox into an intelligence source. It also gives leadership something more useful than a click-rate chart: a picture of where the organization is actually exposed.
Mistake 10: Never Documenting What Changed
Programs drift. A simulation that worked two years ago may now be trivially easy because everyone has seen it. Without a written record of scenarios used, dates, results, and changes made, teams repeat old mistakes and cannot explain their results to auditors or leadership.
Keep a simple log. It takes ten minutes a month and pays for itself the first time someone asks, “Why did our click rate go up in Q3?”
What Good Programs Do Differently
Strong programs share a handful of habits. They measure reporting as seriously as clicking. They run a baseline before training. They keep simulations blame-free. They segment repeat clickers. They coordinate with IT, legal, and the help desk before every campaign. And they treat the whole thing as a long-term behavior-change effort rather than a quarterly box to tick. (How to take a screenshot on a mac)
None of this requires expensive tooling. It requires discipline, a little humility, and the willingness to look at uncomfortable data without punishing the people who produced it.
Conclusion
Phishing simulations are not a test of who is smart and who is careless. They are a mirror held up to an organization’s processes, culture, and communication. The most common mistakes, from fixating on click rate to humiliating clickers to cloning internal tools without warning anyone, all share a root cause: treating the simulation as an end in itself rather than a means to a safer workplace.
Get the fundamentals right, measure reporting as carefully as clicking, and keep the program humane. The numbers will follow.
Frequently Asked Questions
What is the biggest mistake in phishing simulation programs?
Measuring only click rate. A program that ignores reporting rate and time-to-report misses the behaviors that actually reduce harm. Click rate is a baseline, not a goal.
Should employees be punished for clicking a simulated phishing email?
Practitioner consensus says no. Punitive simulations suppress reporting, because people learn to hide mistakes rather than flag them. Reward fast reporting instead.
How often should phishing simulations be run?
Most organizations do well with monthly or quarterly campaigns, with gradually increasing difficulty. Weekly tests cause fatigue; annual tests let skills decay.
Do phishing simulations need legal or HR approval?
Often yes, especially where employee representatives or labor agreements exist. At minimum, coordinate with legal, HR, the help desk, and the email security team before launching.
What is the difference between a one-time clicker and a repeat clicker?
A one-time clicker usually needs a short refresher. A repeat clicker may have a workflow problem, such as inbox overload or a lure that mimics a tool they use hourly. Treat them differently.
Can phishing simulations accidentally cause security incidents?
Yes. Simulations that clone real internal tools without authorization can trip email filters, generate help-desk tickets, or trigger incident-response alerts. Allowlist the infrastructure and agree on a kill switch first.
The final habit worth naming: close the loop with the people who were tested. A short note after each campaign explaining what was simulated, what the results showed, and what will change builds more trust than any training module. Transparency turns a surveillance exercise into a shared effort, and employees who understand the program’s purpose report faster and click less. That is the whole point.
