Run marketing experiments
Run marketing experiments
1. Create a document with the following sections: Hypothesis, Dependent variables, Sample size, Analysis plan, Testing execution plan, and Dependencies.
Divide the Hypothesis list into two subsections: Testing Hypothesis(HA) and Null Hypothesis(HO).
2. Create and add your testing hypothesis statement to the testing hypothesis (HA) subsection.
Use a testable and falsifiable statement starting with “I believe”. For example: I believe my company will gain a 50% increase in revenue for every 50% increase in price. I believe changing our brand colors to violet and gray will increase our customer retention rate by 10%. I believe increasing our ad spend will improve our ads reach. These are falsifiable and testable, while “I am an occult mystic because I can see ghosts with my third eye” is not.
3. Create and add your null hypothesis statement to the null hypothesis (HO) subsection.
Your null hypothesis states the opposite of your testing hypothesis. For example: Our revenue will not increase by 50% if we increase our price by 50%. Changing our brand colors to violet and gray will not improve our customer retention rate. Increasing our ad spend will not improve our ads reach.
4. Add your testing dependent variables (testing outcomes) to the dependent variables section.
5. Calculate your sample size from your significance levels and power levels.
The standard significance levels is 95%, while the standard power level is 80%. You can calculate your sample size easily with the CXL A/B test calculator.
6. Add the segments of data that you will be analyzing to the analysis plan section.
7. Add your testing plan to the testing execution section.
Add how your test will be run, where the test will be run, and when your tests will be run to this section.