Improve A/B tests with session recordings on Lucky Orange

1. Create a behavior tag for the URLs of your test variants.

Navigate to Settings and click on Behavior Tagging and Add New Rule. Select Match URLs that contain this text from the drop down list. Enter the URL of the first variant in the URL to match field. Enter text that is unique to the page, so it returns only results for those pages. Enter a descriptive name in the Tag recording with field. For example, you might name your behavior tag Home Test A.  Click on Add Rule to save. Screenshot showing how to set up a behavior tag to filter Lucky Orange recordings.

2. Repeat the step above to add behavior tags for additional variants.

Create a new behavior tag for each of your test variants, such as HomePageTestB or LandingPageBFCM1.

3. Return to the Recordings tab and filter by the behavior tags you created.

Click on Filter by Tags and enter your behavior tag name in the with: field. A list of session recordings in the Lucky Orange dashboard showing the recordings filtered by behavior tag

4. Click on Any Activity to find recordings with a high level of activity and filter by visitors with High and Very High activity.

These visitors were more engaged with your page and will give you more behavior to evaluate. The indicator bar in the activity column will show as full and have warmer colors (oranges and reds) for recordings with a high number of clicks, scrolls and moves. A list of session recordings in the Lucky Orange dashboard filtered by those recordings that have high visitor activity.

5. Watch recordings and make note of important behaviors to see if visitors were more likely to engage with the changes you made.

Those changes can be: Button placement Color changes Imagery changes Different word choice between variants.

6. Decide whether changes between variants contributed to differences in conversion rates.

If you don’t see differences in engagement with the changes you made, you may need to re-run your test with a bigger audience sample to make sure the winning variant didn’t happen by chance.