Split Tests (A/B Testing)
Split Testing (A/B testing) is vital for an online store because it allows businesses to compare two versions of a webpage to determine which one performs better.
By testing different elements such as headlines, images, calls-to-action, or layouts, stores can identify what resonates most with their audience. This data-driven approach helps optimize the user experience, increase conversion rates, and make informed decisions that enhance overall sales performance.
Ultimately, A/B testing is a key strategy for continuous improvement and maximizing the effectiveness of an online store and pages.
The Split Testing feature allows you to test:
A ZP Landing Page vs another ZP Landing Page
A ZP "Full Customization" Product Page template vs one of your Theme's product page templates
A ZP "Full Customization" Product Page template vs another ZP "Full Customization" Product Page template
To help identify which changes made increases or maximizes an outcome of your interest. Split Tests are a great way to improve your page's conversion rate performance by testing two of your pages against each-other to find a winner and to continuously improve page performance:
The data collected for both pages allows you to identify which page variant has a higher purchase conversion rate (or other goal) so you can drive traffic to most winning versions of your pages.
📝 The Split Testing functionality relies on data from your Shopify store collected by the Zipify Analytics pixel. You must include a product featured in a Buy Box section on your split test pages and drive traffic directly to the split test URL to record conversion rate and other ecommerce data successfully.
Creating a Split Test
To create a Split Test, follow the steps below:
1. Ensure that you've enabled Zipify Analytics at Settings > General > Tracking within the Zipify Pages app:
2. Click on the "Split Tests" link (1) in the sidebar of the app, then on the "Create A Split Test" button (2) there:
It's recommended to prepare both pages for the test before starting the creation process. For landing pages, we recommend to duplicate the main page and then apply minor changes to the duplicated version (variant) which you wish to test against the original page (control).
3. Choose the type of page you want to split test (Landing Pages or Product Pages) then click the "Continue" button:
Landing Pages
On the next screen, Name your test (1) and choose page A (Control) by clicking the "Choose" button (2) and making your selection in the "Choose the Control Page" popup:
📝The test name field is pre-filled with an Untitled {time} title.
Next, choose page B (Variant) for the test. You can either "Copy Page A" directly or click "Choose Existing" make your selection in the "Choose the Variant Page" popup:
If the "Copy Page A" option is chosen, the page will be copied on-the-fly in an unpublished state. Click the "Edit Page" button next to the copy to edit it in the Builder and then Publish it.
Finally, set a Test Goal (1), set the Traffic Distribution (2) and click the "Start" button (3) to start the test. The traffic will be distributed once the test is launched:
The Public Test URL is the same as Page A (Control). This is the URL that traffic should be driven to from your Ads/Campaigns to be included in the test.
Product Pages
On the next screen, Name your test (1) and click the "Choose Product" button (2), then make your selection in the "Choose the Product" popup for the product page you want to split test:
📝 The test name field is pre-filled with an Untitled {time} title.
Next, choose the Zipify Pages product page template to use for Page A (Control). Only a product page template created in the Zipify Pages app can be used for page A. In most cases, it will be the template which is currently assigned to the product as noted by a 🏷️ next to the template name:
Next, choose the product page template for Page B (Variant) for the test. You can either "Copy Template A" directly if you want to test the template against itself with some small edits and changes or click "Choose Existing" to choose a completely different product page template from the "Choose the Variant Page" popup:
Copy Template A
If the Copy Template A option is chosen, a copy of the product page template will be created on-the-fly in an unpublished state with 0 products assigned to it:
Click the "Edit Page" button next to the copy to edit it in the Builder and then Publish it:
📝The copied product page template should be published with 0 assigned products as there is no ability to have 2 product page templates published at the same time for the same product on a Shopify storefront.
Finally, set a Test Goal (1), set the Traffic Distribution (2) and click the "Start" button (3) to start the test. The traffic will be distributed once the test is launched:
Choose Existing
If the Choose Existing option is chosen, then choose the variant product page template to use for the test. You can either choose a Zipify Pages product page template or a product page template from your Theme as page B.
There you can also click the "Update Themes List" button (1) to sync existing product page templates from your theme to the list, use the Search field (2) to search for a specific template and/or click the template type dropdown (3) to sort your templates by:
All product pages
Zipify product pages
Shopify product pages
Once you've found and selected the variant product page template, click the Save button:
Finally, to edit the product page template in the builder if needed before starting the test, click the "Edit Page" button (1). Set a Test Goal (2), set the Traffic Distribution (3) and click the "Start" button (4) to start the test. The traffic will be distributed between the two templates once the test is launched:
Test Goals
Test Goal (Metric) | Description |
Page Value | The average value of this page or set of pages. Page Value is (Transaction Revenue divided by Unique Pageviews for the page or set of pages). |
Conversion Rate (default) | The percentage of sessions that resulted in an e-commerce transaction. (Number of Transactions / Total Number of Unique Pageviews) |
Sales | Total number of completed purchases. |
AOV | Total revenue / Number of orders. |
Product Revenue | The total amount of money generated from sales on a page. |
Revenue per View
| Revenue / Total Number of Views. |
* 📝 | Revenue is being tracked as the total amount the customer paid (including Shipping + Taxes - Discounts) |
Important Notes:
You can only split test pages which have the same setting for the “Use Theme Header/Footer” option in the Page settings. This means this option should be either enabled or disabled for both pages included in the test.
A specific page (landing pages) or template (product pages) can only be used to run one Split Test at a time.
The default traffic distribution is set to 50/50% which is the most precise way to split test pages. Optionally, you can change the traffic distribution model by sliding the scale towards A or B to adjust the distribution.
It's not recommended to apply changes to the pages (landing pages) or templates (product pages) actively involved in a running split test. Doing so can affect the trustworthiness of stats already collected for the test (your changes will affect the conditions that the previous stats were collected for). We recommend to create another test instead which guarantees the integrity of the experiment.
The split test captures data up until the checkout completion event. Any actions or events occurring after the checkout process, including post-purchase offers, are not tracked.
Split Test Dashboard
After you create a test and it's been running for some time, you can view its performance by going back to the Split Tests section of the Zipify Pages app. The split test dashboard section lists all your Split tests ( Running, Draft and Stopped statuses).
The Split Tests table contains stats (Test Goal (metric), Total Views, Unique Views and Status) for each page of your tests (excluding ones you haven’t started yet) as well as the Duration of the test. You can stop the Split Test at any time and then start it again later if you'd like. Keep in mind that we’re not gathering statistics for a test which has been stopped.
It can take a few hours to see the initial stats for your split test pages after a test is started because Zipify Analytics needs some time to collect and process it. Stats typically update every few hours, however there can be delays at times when the overall volume of data across all stores is very large.
*Conversion Rate shown refers to "Purchase" conversions only.
If you wish to apply any changes to a running Split Test you should Stop the test first. Then click on the “Edit Test” action within the Actions drop-down next to the test you want changes applied to. Apply your changes and then start the test up again by clicking the "Run" button.
Split Test Details View
To access the split test details view, click directly on the split test name (1) or on the "Test Details" button (2):
There you can observe and analyze traffic per device type (1), main metrics for each Split Test page (2) and Total Statistics (3) in general:
Actions available for Split Tests
"More Actions" for your split tests can be accessed by clicking the three dots icon in the "Actions" column:
Preview Test - takes you to Page A's URL in a new tab.
Edit Test - takes you to the Edit Split Test screen to apply changes. The test needs to be stopped before edits can be applied.
Select Winner - choose either Page A or Page B as the winner and stop the test. If you choose the variation page (Page B) as the winner, then the original page (Page A) will become unpublished and the variation page will receive the original page's URL. This means that the traffic will arrive at the original URL, but that the content of the page would be the variation page's content instead.
Copy Test Link - copies Page A's URL to your clipboard.
Delete - deletes the split test. The action must be confirmed in the popup. Once confirmed, the action can't be undone.
⚠️IMPORTANT⚠️
If the losing page is being used for the:
"ZP Landing Page" button/image/link Click Action for any existing pages
"Page" destination for header/footer links for any existing pages
...the URL for those will break (return a 404 page not found) due to the losing page's URL becoming unpublished when you pick the winner. In this case, make sure to update any "ZP Landing Page" and/or "Page" destinations you may be using on existing page(s) in sections after picking the winner.
Important notes about running your Split Tests
You can’t delete or unpublish a page that is involved in a running split test. You have to stop the test first.
Be careful when updating a page that is involved in a running split test since this changes the page stats the data is collected for and makes the collected data irrelevant.