Split Tests (A/B Testing)
This feature allows you to Split Test (A/B):
A ZP Landing Page vs another ZP Landing Page
A ZP "Full Customization" Product Page template vs one of your Theme's product page templates
A ZP "Full Customization" Product Page template vs another ZP "Full Customization" Product Page template
To help identify which changes made increases or maximizes an outcome of your interest. Split Tests are a great way to improve your page's conversion rate performance by testing two of your pages against each-other to find a winner.
The data collected for both pages allows you to identify which page variant has a higher purchase conversion rate (or other goal) so you can drive traffic to the winning versions of your pages.
*Notes:
The Split Testing functionality relies on data from the built-in Google Analytics integration for Shopify found in the Online Store > Preferences > Google Analytics section of your Shopify admin. Only the Universal Analytics (UA) Property type is supported currently (GA4 support coming soon). If you are not using this integration for your Shopify store, then Split Test functionality will not work correctly.
Total Views and Unique Views data is being pulled from the Behavior > Site Content > All Pages section of your GA account.
Transactions and Conversion Rate data is from the Behavior > Site Content > Landing Pages section of your GA account. The definition of "Landing Pages" in GA are the pages through which visitors entered your site I.e. traffic must be driven directly to your split test URL to be recorded in this section and for Conversion Rate data to be reported.
Product Revenue data is from the Conversions > Ecommerce > Sales Performance section of your GA account.
You must include a product featured in a Buy Box block (not including the "Collection Block") on your split test pages and drive traffic directly to the split test URL to record conversion rate data successfully.
Connect your Google Analytics account
Zipify Pages doesn’t collect any Split Test stats in-app, but uses data collected by your own Google Analytics account instead and the Shopify integration.
If you don't already have Google Analytics connected to your Shopify store and Enhanced Ecommerce enabled in both your GA account and Shopify admin, please do so before proceeding further. The details on how to set that up can be found in this help article from Shopify: https://help.shopify.com/en/manual/reports-and-analytics/google-analytics/google-analytics-setup
Once you have your Google Analytics account connected properly to your Shopify store with Enhanced Ecommerce enabled, you need to connect your Google Analytics account to the Zipify Pages app. To do that, simply follow the steps below:
1. Navigate to the Settings > General > Tracking section of the app. Click the "Connect Google Analytics" button. Sign in to your Google account that your Google Analytics account is listed under:
2. After you sign in to your Google account, use the drop-down menus to select your Google Analytics Account, Property and View associated with your Shopify store and click the Save button. Be sure to select the correct account, property and view which contains data about your store and subsequently the pages you want to split test:
If you need to connect a different GA account, click the "Connect a Different Account" button and then sign in to your other Google account instead.
Creating a Split Test
To create a Split Test (after connecting your Google Analytics account), follow the steps below:
1. Go to the "Split Tests" section of the app and click the "Create A Split Test" button there:
It's recommended to prepare both pages for the test before starting the creation process. For landing pages, we recommend to duplicate the main page and then apply minor changes to the duplicated version (variant) which you wish to test against the original page (control).
2. Choose the type of page you want to split test (Landing Pages or Product Pages) then click the "Continue" button:
Landing Pages
On the next screen, Name your test (1) and choose page A (Control) by clicking the "Choose" button (2) and making your selection in the "Choose the Control Page" popup:
📝The test name field is pre-filled with an Untitled {time} title.
Next, choose page B (Variant) for the test. You can either "Copy Page A" directly or click "Choose Existing" make your selection in the "Choose the Variant Page" popup:
If the "Copy Page A" option is chosen, the page will be copied on-the-fly in an unpublished state. Click the "Edit Page" button next to the copy to edit it in the Builder and then Publish it.
Finally, set a Test Goal (1), set the Traffic Distribution (2) and click the "Start" button (3) to start the test. The traffic will be distributed once the test is launched:
The Public Test URL is the same as Page A (Control). This is the URL that traffic should be driven to from your Ads/Campaigns to be included in the test.
Product Pages
On the next screen, Name your test (1) and click the "Choose Product" button (2), then make your selection in the "Choose the Product" popup for the product page you want to split test:
📝The test name field is pre-filled with an Untitled {time} title.
Next, choose page B (Variant) for the test. You can either "Copy Template A" directly or click "Choose Existing" make your selection in the "Choose the Variant Page" popup:
Copy Template A
If the Copy Template A option is chosen, a copy of the product page template will be created on-the-fly in an unpublished state with 0 products assigned to it:
Click the "Edit Page" button next to the copy to edit it in the Builder and then Publish it:
The copied product page template should be published with 0 assigned products as there is no ability to have 2 product page templates published for the same product on a Shopify storefront.
Finally, set a Test Goal (1), set the Traffic Distribution (2) and click the "Start" button (3) to start the test. The traffic will be distributed once the test is launched:
Choose Existing
If the Choose Existing option is chosen, then choose the variant product page template to use for the test on the "Choose the Variant Page" popup.
There you can also click the "Update Themes List" button (1) to sync existing product page templates from your theme to the list, use the Search field (2) to search for a specific template and/or click the template type dropdown (3) to sort your templates by:
All product pages
Zipify product pages
Shopify product pages
Once you've found and selected the variant product page template, click the Save button:
Finally, to edit the product page template in the builder if needed before starting the test, click the "Edit Page" button (1). Set a Test Goal (2), set the Traffic Distribution (3) and click the "Start" button (4) to start the test. The traffic will be distributed between the two templates once the test is launched:
Test Goals
Test Goal (Metric) | Query in the GA API | Description |
Page Value | 'ga:pageValue' | The average value of this page or set of pages. Page Value is ((Transaction Revenue + Total Goal Value) divided by Unique Pageviews for the page or set of pages). |
Conversion Rate (default) | Current Implementation | The percentage of sessions that resulted in an e-commerce transaction. (Number of Transactions / Total Number of Sessions) |
Sales | 'ga:transactions' | Total amount of completed purchases. |
AOV | 'ga:revenuePerTransaction' | Total revenue / Number of orders. |
Product Revenue | ga:transactionRevenue | The total amount of money generated from sales on a page. |
Revenue per View
| 'ga:transactionRevenue' / 'ga:pageviews' | Revenue / Total Number of Views. |
Add to Cart | 'ga:productAddsToCart' | The number of times a product was added to a shopping cart. |
Important Notes:
You can only split test pages which have the same setting for the “Use Theme Header/Footer” option in the Page settings. This means this option should be either enabled or disabled for both pages included in the test.
A specific page (landing pages) or template (product pages) can only be used to run one Split Test at a time.
The default traffic distribution is set to 50/50% which is the most precise way to split test pages. Optionally, you can change the traffic distribution model by sliding the scale towards A or B to adjust the distribution.
It's not recommended to apply changes to the pages (landing pages) or templates (product pages) actively involved in a running split test. Doing so can affect the trustworthiness of stats already collected for the test (your changes will affect the conditions that the previous stats were collected for). We recommend to create another test instead which guarantees the integrity of the experiment.
Split Test Dashboard
After you create a test and it's been running for some time, you can view its performance by going back to the Split Tests section of the Zipify Pages app. The split test dashboard section lists all your Split tests ( Running, Draft and Stopped statuses).
The Split Tests table contains stats (Test Goal (metric), Total Views, Unique Views and Status) for each page of your tests (excluding ones you haven’t started yet) as well as the Duration of the test. You can stop the Split Test at any time and then start it again later if you'd like. Keep in mind that we’re not gathering statistics for a test which has been stopped.
It can take 1 - 2 hours to see the initial stats for your split test pages after a test is started since it depends on how quickly Google Analytics sends the data back. Data in the split test section updates once per hour for running split tests.
*Conversion Rate shown refers to "Purchase" conversions only. We take Conversion Rate data from the Behavior > Site Content > Landing Pages section of your connected Google Analytics account. You must include a product featured in a Buy Box block on your split test pages to record conversion rate data successfully.
If you wish to apply any changes to a running Split Test you should Stop the test first. Then click on the “Edit Test” action within the Actions drop-down next to the test you want changes applied to. Apply your changes and then start the test up again by clicking the "Run" button.
Split Test Details View
To access the split test details view, click directly on the split test name (1) or on the "Test Details" button (2):
There you can observe and analyze traffic per device type (1), main Google Analytics metrics for each Split Test page (2) and Total Statistics (3) in general:
Actions available for Split Tests
"More Actions" for your split tests can be accessed by clicking the three dots icon in the "Actions" column:
Preview Test - takes you to Page A's URL in a new tab.
Edit Test - takes you to the Edit Split Test screen to apply changes. The test needs to be stopped before edits can be applied.
Select Winner - choose either Page A or Page B as the winner and stop the test. If you choose the variation page (Page B) as the winner, then the original page (Page A) will become unpublished and the variation page will receive the original page's URL. This means that the traffic will arrive at the original URL, but that the content of the page would be the variation page's content instead.
Copy Test Link - copies Page A's URL to your clipboard.
Delete - deletes the split test. The action must be confirmed in the popup. Once confirmed, the action can't be undone.
⚠️IMPORTANT⚠️
If the losing page is being used for the:
"ZP Page" button/image/link Destination for any existing pages
"Page" Destination for header/footer links for any existing pages
...the URL for those will break (return a 404 page not found) due to the losing page's URL becoming unpublished when you pick the winner. In this case, make sure to update any "ZP Page" and/or "Page" destinations you may be using on existing page(s) in blocks after picking the winner.
Viewing Split Test data in your Google Analytics account
Landing Pages
You can see the transaction count and other data for each page in the split test. It can be seen either in the Statistics section of ZP or in the connected GA account.
In the ZP statistics section, you can search (ctrl + f) for ?zpstest and that will bring up any page that is involved in a ZP split test. Then you can check the Transactions column for the number of conversions:
In Google Analytics, go to Behavior > Site Content > Landing Pages and search for ?zpstest to see the data:
You will be able to see Page A and Page B URLs and the data for each page this way. The text string at the end of the URLs is the split test ID and will be the same if both pages are from the same Split Test:
*Note:
Zipify Pages and Google Analytics calculate Total Conversion Rate a little bit differently, so figures between the two platforms can differ slightly. We actually do it in a more accurate way!
For the in-app Conversion Rate calculation for the split test pages, we ignore the additional page URL parameters which are appended to end of the split test page URLs in Google Analytics.
When we're gathering this data from GA, we consider all of these URLs as the same one URL, because we understand that visitors can go to the page not only via a direct URL, but also from different websites/social medias with ads which append additional URL parameters automatically.
Product Pages
To check the Total View and Unique Views related to Product pages (Full Customization), navigate to Behavior > Site Content > All Pages AND apply an Advanced filter (Exclude pages with URLs with view=product
) like shown here in GA admin:
Comparing to the Split Test list in Zipify Pages:
To check the Conversion Rate related to Product pages (Full Customization) navigate to Behavior > Site Content > Landing Pages AND apply an Advanced filter (Exclude pages with URLs with view=product
) like shown here in GA admin:
Comparing to the Split Test list in Zipify Pages:
Similar steps have to be done for Product pages (Partial Customization) BUT the 'Include' option within the Advanced Filter (for product pages with URLs that contain view=product
) should be selected and applied in GA admin like displayed here:
Total & Unique Views:
GA admin - https://i.imgur.com/HY3TnX2.png
Split Tests list - https://www.screencast.com/t/Y9hw1hQ5L
Conversion Rate:
GA admin - https://www.screencast.com/t/ybZrmqbc
Split Tests list - https://www.screencast.com/t/2VVWdiUFXq
NOTES:
1. To update GA tables with the most relevant data, you can click on the "Refresh Report" button here in the bottom-right of the GA reports screen:
2. Previous GA data can be cached on the Split Test list page in the Zipify Pages app. To update split tests tables within the ZP app immediately with relevant data, remove the zp.ga
_reports_cache
file in Dev tools > Application > Local Storage and refresh the page as shown here: https://www.loom.com/share/42e84b3a4e6f4c17bf486f6afc5ebdfc
*Dev tools can be opened by "Ctrl + Shift + I" keyboard combination or by pressing the F12 key.
Split Test Troubleshooting
If you notice that no data is being reported back for the Views or Conversion Rate statistics for your running split tests, the troubleshooting information below can help.
*Important Notes:
Total Views and Unique Views data are being pulled from the Behavior > Site Content > All Pages section of your connect GA account.
Conversion Rate data is being pulled from Behavior > Site Content > Landing Pages section of your connect GA account. The definition of "Landing Pages" in GA are the pages through which visitors entered your site I.e. traffic must be driven directly to your split test URL to be recorded in this section and for Conversion Rate data to be reported.
So, there could be situation where data is mismatched between Landing Pages and All Pages and can affect Conversion rate data. It’s connected to how GA tracks a page and matches it as a Landing Page view or All Pages.
Since the Statistics and Split Test sections within Zipify Pages relies on your connected Google Analytics account, please ensure that you have everything configured properly within your GA account and that it is connected to your Shopify store.
Specifically, please check on the following:
1. You have connected Google Analytics and enabled Enhanced Ecommerce within your Shopify admin at Preferences > Online Store > Google Analytics.
2. You have enabled Enhanced Ecommerce tracking within your Google Analytics account.
3. You have not added an extra Google Analytics tracking code as a Global/Local script to your Zipify Pages. The Split Testing and Statistics sections in Zipify Pages uses the native Google Analytics integration in your Shopify admin at Online Store > Preferences > Google Analytics by default. Adding an extra GA tracking script to your Zipify Pages can cause issues with Split Test and Statistics tracking.
4. You have connected the correct Google Analytics Account, Property and View which contains data about your store to the Zipify Pages app.
If all of those things are not currently implemented, please do so and you should then start to see conversion and revenue stats show up within the Statistics and Split Test section(s) of the Zipify Pages app.
*Note: Ecommerce tracking for Google Analytics starts to work from the time enhanced ecommerce is enabled within your account. It won't work to back-populate data for sales made prior to enabling it.
If you've done this and you're still not seeing any data, you may need to clear your cache. Caching can cause a lack of reporting. So please check via a private browsing window or clear your cache and check the split test again.
Important notes about running your Split Tests
You can’t remove or unpublish a page that is involved in a running split test. You have to stop the test first.
Be careful when updating a page that is involved in a running split test since this changes the page stats the data is collected for and makes the collected data irrelevant.
Once a test is started it will generate a special URL for the pages which contains additional parameters (ex: store_domain/pages/page_title?zpstest={{token}}).
It will only appear this way for the pages in the Statistics section of ZP and in Google Analytics and all metrics will be collected for it. This means that when a split test is running, GA will stop collecting stats for the initial page URLs (store_domain/pages/page_title) and will collect stats for the special URLs (store_domain/pages/page_title?zpstest={{token}}) instead.
IMPORTANT: Split Test pages will be available through a page's initial URL (store_domain/pages/page_title) not through the special one, so you shouldn’t worry about any changes to the links/URLs you already setup and are using in Ads/Campaigns.