How to hire a CRO agency for larger sites

1. Weigh up the positives and negatives of outsourcing CRO for your brand.

If you don’t have enough traffic to your site, you won’t have enough traffic to run quantitative testing, and CRO will have limited data to work with. The potential ROI of CRO also varies based on revenue. If your company doesn’t have enough revenue to benefit significantly, CRO might not be worth the investment. Peep Laja, CXL Founder, explains: “If you increase sales by 1% for a company that makes $100 million, that’s $1 million of added revenue. CRO makes sense. But if you get that same 1% increase for a site that makes $1 million a year, the ROI—$10,000—isn’t there.” Building an in-house team: Takes time. Finding seasoned CRO experts can take many months, and launching a CRO program can take even longer. Is prohibitively expensive for many companies. You’ll need at least $500,000 to build a skeletal team. Can be a bottleneck for launching multiple tests. An in-house team often can’t launch multiple tests because their plates are full. Doesn’t make sense if you have only one funnel to optimize. Once your in-house team maxes out conversions for that funnel, what will they do next? On the other hand, hiring a CRO agency isn’t always the best fit either. This is the case if you have: No clear or realistic goals. You’re not aligned on what an experimentation program needs to achieve or can realistically achieve. No one to own the relationship. You need a point person who can clear roadblocks and hold the agency accountable. No one to implement changes. Test wins become revenue only after they’re deployed across the site. The two options aren’t mutually exclusive: If you hire an agency, you can lean on them to help you build and run, or speed up, an in-house CRO program. They can also, periodically, serve as a second set of eyes on a program or come in to help resolve a particular challenge.

2. Figure out your budget and needs regarding CRO.

Once you have a sense of your needs, you should be able to answer the following questions: How much can you realistically spend? What is your budget for CRO services plus the costs to run tests? One report shows that top-converting companies spend more than 5% of their budget on optimization alone. “It’s tough to find a great agency under $10,000 a month,” says Laja. “And $3,000 a month is a major red flag” if you are seeking an ongoing program of optimization. What are your objectives? Get as specific as possible. You won’t know the percentage improvements that are possible and you may not even know what needs optimizing. But any details beyond increase conversions, like you’re planning UX changes, leads are down, and your internal team lacks the experience for a particular project, can help identify the right agency partner.  How will you measure success? This could include KPIs like customer lifetime value, average conversion rate, cart abandonment rates, and others. Which services fit your needs? Are you looking for ongoing experimentation, customer research, analytics implementation, or all of the above? What tools are you currently using? Catalog your martech stack with all the log-in details. With this handy list, it’s easy for an agency to dive in and get started right away. Who’s the point person in your company for the agency?

3. Compile your own testing history and customer research so that the agency doesn't have to cover old ground.

If your team or company has run experiments or done testing in the past, put together a comprehensive history. Include details like: A description of each test and experiment. The goal of each experiment. How and when the test was executed. The results and learnings from all testing. The history offers much-needed context and avoids waste. Collect all existing information about your ideal customers: Buyer personas. Customer surveys. Other customer research like demographic data. You can share existing documents or set up a call to walk through the information. If an agency doesn’t request historical testing data or customer info, consider it a giant red flag.

4. Ask for references from previous customers to understand more about the real experience of working with the agency.

To figure out whether an agency can help you achieve your specific goals, contact at least three of their previous clients and ask key questions: What problem did they hire the agency to solve? Did they have a process when researching and launching experiments? If so, what did that process look like? What were the tests and experiments that the agency launched? What were the results of those experiments? How were the agency’s response time and level of service? Did they always reply quickly and were they polite and helpful? Were they transparent throughout their engagement? Is the company still working with the agency? If not, why? If an agency is hesitant to provide a list of clients or claims that it’s confidential, Vahtra says, “that’s a red flag”. While an agency might not divulge info about all their clients due to non-disclosure agreements, they should be able to offer a few client references. On top of referrals and references, you can also establish an agency’s credibility with case studies. Check their website or ask your contact person for case studies, for customers within your industry of a similar size or with a similar problem.

5. Ask agencies about their data gathering and optimization processes.

“Ask about the process that’s being used to optimize websites,” Vabrit suggests. “This process should be data-driven. Clients should investigate what data will be gathered, how it will be analyzed and turned into a testing hypothesis, and how learning management is organized.” Asking about stopping rules for tests can be revealing. There are no magic numbers, like 100 conversions per test, a response that would be a major red flag. Sample size calculators should determine how many conversions are necessary for statistical validity.    Some other ways to separate the amateurs from the experts: Big, upfront promises. If they predict big lifts in conversions without running any tests, those results are unlikely to materialize. No process details. A good agency will be transparent about all the testing details before launching. For example, they’ll work with you to decide how long specific tests should run. Too many clients per analyst. How many clients does each CRO analyst manage at an agency? It’s a revealing question. “They should have one analyst per 3–4 clients max,” suggests Laja. “Anything more and quality suffers.”

6. Ask about package costs and what services are included or excluded.

“Ask about all costs related to running a CRO program with a specific agency,” Vabrit says. “What’s included in the service, what’s not. Should they allocate an additional budget to develop and QA tests? What are different tool costs needed for optimization?” For example, one agency’s retainer maybe half that of a competitor but if that fee doesn’t include QA testing and development costs, you may end up paying more in the long run. Ask whether an agency’s fee includes: QA testing. Multiple variations for each test. Ask how many variations are included in a package. Optimization and analytics tools. Developer time. Designer time. If one agency comes in vastly lower than others, ask why. As Laja notes, “some young agencies are just hungry, everyone starts somewhere, and they may be willing to take on clients for less money.” However, others may try to close the revenue gap with client volume, meaning that analysts are managing 10 or 12 clients at a time.

7. Ask about their team structure and the sort of consultancy service you should expect.

Does the agency team function like a factory, with many clients per analyst, or would you receive a personalized, consultant-like service? Will you have a single contact, and, if so, who?  Some aspects of team structure aren’t necessarily better than others, but knowing how the team is structured can help you establish expectations. “The main piece of advice that I would have is making sure that all of the parts of the company that an agency will touch are aligned,” Codeacademy’s Layfield said. “Communicating everything is key.” Also, ask about their human resources. Do they have developers, designers, customer researchers, and data specialists in-house, or do they outsource their projects to external experts?

8. Run additional checks on shortlisted agencies to single out better candidates, like reading their content and testimonials.

Here are a few final methods to help you narrow down your list further: Customer testimonials. Read any testimonials on their website, or ask for them, if none are published. Let your team ask questions. Leake had plenty of questions while vetting JungleScout’s CRO agency. She recommends letting relevant team members ask questions as well. This engages the entire team and helps them understand how the agency would affect their role. Ensure the agency specializes in CRO. Vahtra cautions against hiring an agency that focuses on SEO, for example, but does CRO as an add-on. Outsourcing your CRO as an add-on service often leads to slower testing, inaccurate data interpretations, and poor processes. Read their content. Is the agency publishing thought-leadership content that highlights new, innovative ideas?  Educational opportunities. CRO agencies will often help teach a company to fish. Do they educate your team so that you can eventually run your own in-house CRO program?