A/B test Facebook ad creatives

1. Decide what and how many ad creatives to test.

There’s no one-size-fits-all solution to this, what and how many ad creatives you test ultimately depends on your overall budget and capacity to create and analyze A/B tests. Larger budgets and more resources will allow you to test more ad creatives at once.  This study by AdEspresso shows that, when it comes to ad creatives, images, ad text, placement, landing page, and headline provide the biggest gains. Taking that into consideration, a good starting point would be to focus on testing at least three different images and one or two different sets of ad text. Always test different creative types – Images, Video and Carousel ads. Different audiences will prefer different types of creative, and you want to identify this early.

2. Understand the way Facebook treats and automatically A/B tests multiple ads in a single ad set.

Facebook will determine a “winner” using its own algorithm and automatically scales back the number of impressions delivered to the “losing” ad. However, Facebook often picks its “winner” based on a very small sample size. While this isn’t a major issue, it can sometimes make testing multiple ad creatives at the same time somewhat ineffective, as Facebook will often allocate the majority of impressions to what it determines to be your best-performing ad  and stop showing your lower-performing ads. For example, in the image below you can see just how early Facebook made a decision on which ad it determined to be the best-performing. However, by taking into consideration the cost per website click (the goal of this campaign), you can see that the ad Facebook deemed the “loser” was actually delivering cheaper results. The ad Facebook deemed the “loser” was actually delivering cheaper results.

3. Create separate ad sets, each targeting the same audience

Place up to 3 ads in each ad set. More than 3 often results in under-spending on some ads This allows you to collect enough data to make intelligent decisions without being limited by Facebook’s own A/B testing.

4. Run ads for at least 72 hours without making any changes to give the Facebook algorithm enough time to optimize.

72 hours is the minimum you should run your test for. If you don’t have a statistically significant results after 72 hours you should continue to run the test.

5. Ensure you have enough data for your results to be statistically significant.

The amount of time it takes to collect enough data varies, but a minimum test period of 5 days is recommended. It’s possible to deliver tens of thousands of impressions and get hundreds of results in a shorter time period, but that requires a relatively large budget, and also allows for anomalies on any given day to skew the results. You can use a tool like A/B Test Guide’s A/B Test Calculator to help determine when your test achieves significance.

6. After you identify the winners of your test, focus your next round of testing on improving that winning formula

For example, if you find that carousel ads vastly outperform your other creative types, you should focus on testing more carousel ads. Likewise with video and images Within each creative type you will also notice that creatives with certain attributes outperform others. Look to continue testing around those top performing attributes.