How to A/B test

One of our top tips for anyone getting started with personalization would be to define your metrics and then TEST, MEASURE and OPTIMIZE.

The more content you test, the more effectively you can measure differences and then go on to fine-tune and optimize it. Within Fresh Relevance you have the ability to go one step further and really enhance your marketing by A/B testing your content - making sure it’s performing at its best to deliver you those all important results.

This How to Guide details the steps to take in order to get testing!

How to A/B test TRIGGERS

Icon - Triggers.png

Step 1 - Navigate to your Trigger

  • On the left nav click on Triggers
  • In the Signal dropdown, select the Signal where you'd like to run your test
Triggers - 1.png
Icon - Rules.png

Step 2 - Build your Programs and rules

  • Create your Trigger Programs you want to test
  • Build out any Rules as needed
Triggers-2.png
Icon - Split Testing.png

Step 3 - Setup your tests

  • Pull the first program you'd like to test into the rule tree
  • Pull the 2nd piece program you'd like to test on top of the first piece of content (in the same position)

This automatically creates a A/B Split test. Click on the header to:

  • Edit the proportions each program should be sent
  • Optionally set the test to Auto-end, where the winning program is implemented after a set date or number of sends
Triggers - 3.png
Icon - Reporting.png

Step 4 - Review Results

  • On the left nav click Optimize to access the Optimize center. 
  • Find your test in the list and click on the 3 dots, then the Chart icon to Open Report
  • In this report, use the Current Report dropdown to compare performance on various measures (including revenue & AOV)
Triggers & Web - 4.png

How to A/B test SMARTBLOCKS

Icon - Web & Email.png

Step 1 - Build your smartblocks

  • On the left nav click on Content | SmartBlocks
  • Create the 2 or more pieces of content you'd like to test. These can be different SmartBlock types (e.g. Banner & Countdown timer)
Web-1.png
Icon - Rules.png

Step 2 - Create your Slot & Rules

  • On the left nav click on Content | Slots 
  • Create or edit the Slot where you would like to run your test
  • Click on the Rules tab and build out any Rules as needed
Web-2.png
Icon - Split Testing.png

Step 3 - Setup your tests

  • Pull the first SmartBlock you'd like to test into the rule tree
  • Pull the 2nd SmartBlock you'd like to test on top of the first SmartBlock (in the same position)

 

This automatically sets the content to be randomly selected. Click on the header to:

  • Select an A/B Split test
  • Edit the proportions each SmartBlock should be shown
  • Optionally set the test to Auto-end, where the winning program is implemented after a set date or number of impressions
Web - 3.png
Icon - Reporting.png

Step 4 - Review Results

  • On the left nav click Optimize to access the Optimize center. 
  • Find your test in the list and click on the 3 dots, then the Chart icon to Open Report
  • In this report, use the Current Report dropdown to compare performance on various measures (including revenue & AOV)
Triggers & Web - 4.png