Modifying product listings during a test, ruins data!
It is best practice to never modify a product listing outside of our app when you have a running test for that same product. This could lead to multiple variable changes during the duration of a test, which would make it impossible to know which variable change helped or hurt the overall test performance.
“In an experiment, scientists only test one variable at a time to ensure the results can be attributed to that variable. If more than one variable is changed, scientists cannot attribute the changes to one cause.”
-Google AI
Related Articles
Test Conflict/New Test Conflict?
If you received a test conflict error, this means that you are trying to schedule or run more than one test at a time for the same product. Running or scheduling more than one test for the same product can lead to multiple variable changes during the ...
Why can I only test one variable at a time?
Testing multiple variables for the same product skews test data which makes it impossible to know which test variable helped or hurt the overall test performance. Instead, Optimal A/B requires that you only test one variable at a time for each ...
Setting up Test Variants?
On the right portion of the “Variants” page is where you configure different variations of: Prices, Images, Titles, or Descriptions, depending on what type of test you are running: syncX: A/B Testing will automatically cycle through all the ...
Test progressing slowly?
If you are not receiving enough traffic to your product listings, paid advertising campaigns on social media can be an excellent way to drive traffic, which will increase the overall effectiveness of your A/B testing. Shopify also has a very helpful ...
What do the test analytics mean?
Avg Revenue Per View: The average revenue generated by a single view for a given test variant. This is the default metric used to determine the winning variation for Image, Title, and Description tests. Avg Profit Per View (only available for Price ...