Quick Answer: How do I choose a conversion rate optimization company?
Choosing a conversion rate optimization company means finding a partner that can explain its research process, its testing statistics, and its pricing model in plain language before you sign anything. The practical path is to run a paid pilot or audit, ask for raw test data from past clients, and compare how each vendor handles sample size, test duration, and reporting. (Best polling rate for mouse)
Alternatives worth knowing include freelancers, in-house teams, and hybrid agencies that bundle CRO with paid media or web development. Those options differ mainly in cost structure, speed, and how much control you keep over your traffic and data.
Before committing, compare process clarity, statistical rigor, pricing model, case study transparency, data ownership terms, contract notice periods, and GDPR compliance.
Why This Decision Goes Wrong So Often

Picture two agencies pitching the same week. One promises a 40% lift in ninety days. The other says it needs six weeks of research before it can even guess at a number. Most buyers pick the first one. Then they spend a year learning why the second one was right.
Conversion rate optimization, or CRO, is the process of increasing the percentage of website visitors who take a desired action. It is a process, not a one-off audit. That single sentence, widely used by practitioners, is the whole ballgame. A company that sells you a “conversion audit” as a finished product is selling you a snapshot. A company that sells you a research and testing loop is selling you a system.
The tricky part is that both look identical on a proposal deck. This guide is about telling them apart before money changes hands.
Start With the Process Question, Not the Results Question
Ask any candidate to walk you through the last three tests they ran. Not the wins. The whole set.
A real CRO team will describe a research phase, a hypothesis, a test design, a result, and what they learned. They will mention a test that failed. They will talk about the sample size they needed and why. If they only talk about wins, you are hearing a sales pitch dressed as a case study.
The classic research loop looks like this:
- Gather data from analytics, heatmaps, session recordings, and user interviews.
- Form hypotheses tied to specific observed friction.
- Prioritize tests by expected impact and effort.
- Run controlled experiments with pre-calculated sample sizes.
- Analyze results honestly, including inconclusive ones.
- Roll learnings into the next cycle.
A vendor that skips step four is not doing CRO. They are doing redesign with extra confidence.
The Statistics Test Every Buyer Should Run
You do not need a statistics degree to spot a weak testing partner. You need three questions.
First: how do you calculate sample size? A credible answer involves baseline conversion rate, minimum detectable effect, and statistical power. Practitioners commonly use 80% power and 5% significance as working defaults, though these are conventions rather than laws of nature.
Second: what happens if a test looks significant on day three? The right answer is nothing. Stopping early because a result looks good is called peeking, and it inflates false positive rates well above the nominal 5%. This problem has been documented in practitioner writing for over a decade, and it remains one of the most common ways agencies manufacture impressive-looking wins.
Third: what do you do with an inconclusive test? Weak vendors bury it. Strong vendors treat it as information and design a follow-up.
If a candidate cannot discuss power, sample size, and peeking without getting defensive, keep looking. This is the single most reliable filter available to a non-technical buyer.
Pricing Models and What They Hide
CRO vendors price in several common ways, and each has a different incentive structure.
| Model | How It Works | Watch Out For |
|---|---|---|
| Monthly retainer | Fixed fee for research, testing, and reporting | Vague deliverable counts; paying for meetings |
| Project-based | Fixed price for a defined audit or test sprint | Scope creep; audits sold as outcomes |
| Performance-based | Fee tied to measured lift | Attribution disputes; pressure to claim wins |
| Hybrid | Base retainer plus bonus on results | Complex measurement agreements |
Performance-based pricing sounds appealing and often is not. It pushes vendors toward short tests on high-traffic pages, where variance is easiest to spin. It also creates fights over whether a lift was real or seasonal. A hybrid model with clear measurement rules can work, but only if the contract defines exactly how lift is calculated.
Whatever the model, ask what happens in a month with zero wins. Good vendors plan for this. Bad ones pad reports.
Case Studies Are Weak Evidence Until You Interrogate Them
A screenshot showing plus 62% conversion rate tells you almost nothing. Four details turn a case study into evidence:
- The baseline conversion rate before the test
- The sample size per variation
- The test duration and whether seasonality was considered
- Whether the result was replicated in a follow-up or held after rollout
Ask for those four things on two case studies. Vendors who have them will produce them quickly. Vendors who do not will change the subject toward “strategy” and “brand alignment.”
One more practical tip: ask whether you can speak to a past client directly. Most buyers never do this. The ones who do learn a lot about communication speed, reporting honesty, and how a vendor behaves when a test fails.
Data Ownership, Privacy, and the Boring Clauses That Matter
CRO work runs on your traffic and your visitor data, which makes the contract unusually important.
In the EU, the GDPR has applied since 25 May 2018, and the ePrivacy Directive constrains how cookies and visitor data are handled during testing. If your vendor runs tests on European visitors, they need a consent story that matches your own. Ask how their testing tool handles consent state, and whether users who decline tracking are excluded from experiments.
Accessibility matters too. Serving different experiences to different users can create barriers. The WCAG 2.2 guidelines, published as a W3C Recommendation on 5 October 2023, are the reference point most teams use. Ask whether the vendor has a policy on testing variations that could break keyboard navigation, screen reader output, or contrast. A vendor without an answer has probably never been asked.
Then there is the clause almost nobody reads: does the vendor use your traffic for its own benchmarking or product improvement? Ask for that in writing, along with who owns the test data and learnings when the engagement ends.
Contract hygiene checklist:
- Ownership of test data, creative assets, and documented learnings
- Notice period and exit terms
- Whether your traffic feeds the vendor’s benchmarks
- Consent and data processing terms
- Reporting cadence and access to raw experiment data
In-House, Freelancer, or Agency?
Not every company needs an agency. The choice depends on traffic volume and internal capacity.
| Option | Best For | Main Tradeoff |
|---|---|---|
| In-house team | High traffic, long horizon, data-sensitive products | Hiring cost and slow ramp |
| Freelancer | Single funnel, tight budget, defined scope | Limited bandwidth and tooling |
| Specialist CRO agency | Mid to high traffic, multiple funnels | Cost and onboarding time |
| Full-service agency | Teams wanting CRO bundled with media or dev | CRO often becomes the side dish |
A useful rule of thumb: if your site gets fewer than roughly 10,000 monthly visitors per tested page, most tests will be underpowered and no vendor can fix that with cleverness. In that case, spend the budget on research, qualitative feedback, and obvious friction fixes rather than on a retainer promising statistical wins. This is the same logic that applies when you choose a design tool as a beginner — pick the option that fits your actual constraints, not the one with the loudest feature list.
Questions That Separate Good Vendors From Great Ones
Bring this list to every call. The answers matter less than how fast and how specifically they arrive.
- What percentage of your tests produce a statistically significant win? (Industry-wide, most tests lose or tie. A vendor claiming 80% wins is either cherry-picking or measuring badly.)
- Who designs the experiment, and what is their statistics background?
- How do you handle low-traffic pages?
- What does your reporting look like in month one versus month six?
- How quickly can you start, and what do you need from us?
- What is your policy on testing changes that affect accessibility?
Notice that none of these ask about price. Get the process answers first, then negotiate. Buyers who lead with budget tend to end up with vendors optimized for selling, not testing.
Run a Paid Pilot Before You Sign a Retainer
The best de-risking move in this entire process is a small paid pilot. Ask for a four-to-six week engagement covering research plus one or two tests on a defined section of your site.
A pilot reveals things no proposal can:
- How fast they actually communicate
- Whether their research produces specific, testable hypotheses or generic best-practice lists
- Whether their reporting is honest about inconclusive results
- Whether your teams can work together without friction
Set clear success criteria for the pilot itself. Not “a winning test” — that is partly luck — but “a documented research phase, two properly powered tests, and a written learning summary.” Judge the process, because the process is what you are actually buying.
The same principle shows up in plenty of other buying decisions. When people pick AI transcription tools, the winners are rarely the ones with the longest feature lists; they are the ones that fit the actual workflow. CRO vendors are no different.
Red Flags Worth Walking Away From
Some signals are close to disqualifying on their own.
Guaranteed lift percentages. No honest tester guarantees an outcome. Variance does not care about your contract.
Best-practice audits as the core deliverable. “Move the CTA above the fold” is not a hypothesis. It is a guess with a slide deck.
No discussion of sample size. If the word never comes up, the testing is decorative.
Reports that only show wins. Ask what happened to the other tests. The answer is usually revealing.
Resistance to data access. You should be able to see raw experiment data at any time.
Vague tooling answers. Ask which testing platform they use and who pays for it. Tool limits shape what is possible.
There is no single accredited certification body for CRO vendors that can be independently verified, so treat “certified partner” badges as marketing until you know who issued them and what the criteria were.
How Traffic Volume Changes the Whole Calculation
This deserves its own section because it quietly determines whether any vendor can help you.
Statistical power depends on traffic and baseline conversion rate. A page converting at 2% with 5,000 monthly visitors will take months to detect a modest lift. A page converting at 15% with 200,000 visitors can test aggressively.
Ask candidates to estimate, using your real analytics, how many tests per quarter your traffic can support. A vendor who gives you a number grounded in your data is doing real work. A vendor who promises twelve tests a quarter regardless of traffic is selling activity, not outcomes.
If your traffic is thin, the honest path is qualitative research, usability fixes, and offers or pricing changes — things that do not need statistical proof to be obviously better. A good vendor will tell you this even though it means a smaller contract. That conversation is worth more than any case study.
Conclusion
Choosing a CRO company comes down to one shift in perspective: you are not buying lifts, you are buying a disciplined research and testing process run by people who will tell you the truth about inconclusive results. Ask about sample size and peeking. Demand case studies with baselines, sample sizes, and durations. Read the data ownership clause. Run a paid pilot and judge the process rather than the scoreboard.
The vendors worth hiring will welcome those questions. The ones who flinch are answering a different question than the one you asked — and that is your answer too.
FAQ
How long before a CRO engagement shows results?
Expect the first month to be research-heavy with little testing. Meaningful test results usually appear in months two through four, depending on traffic volume. Anyone promising wins in week two is skipping the research that makes testing worthwhile.
Should I choose a performance-based CRO contract?
It depends on how carefully lift is defined. Performance deals can align incentives, but they also create pressure to run short tests on high-traffic pages and argue over attribution. A hybrid model with written measurement rules tends to be more stable than pure pay-for-lift.
What is the difference between CRO and SEO?
SEO aims to increase qualified traffic to a page. CRO aims to increase the share of existing visitors who convert. They overlap in research and user intent work, but they are measured differently and usually need different specialists.
Can a small business with low traffic do CRO?
Yes, but not through high-volume A/B testing. With limited traffic, focus on qualitative research, session recordings, user interviews, and fixing obvious friction. Save statistical testing for pages that get enough visitors to reach a valid sample size.
Who should own the test data after the contract ends?
You should. Insist in writing that experiment data, creative assets, and documented learnings belong to your company, and confirm whether the vendor uses your traffic for its own benchmarking. Getting this clarified before signing avoids awkward conversations later.
How many CRO vendors should I evaluate?
Three to five is a practical range. Fewer than three and you lack comparison; more than five and the process drags. Use the same question list for each and score the answers side by side rather than relying on memory.


