On this page

Split Tests (A/B Testing)

This feature allows you to split test (A/B test) two different ZP Landing Pages or a "ZP Layout" Product page layout against your Theme's product page layout, to identify which changes made increases or maximizes an outcome of your interest. Split Tests are a great way to improve your page's conversion rate performance by testing two of your pages against each-other to find a winner. 

The data collected for both pages allows you to identify which page variant has a higher purchase conversion rate (or other goal) so you can drive traffic to the winning versions of your pages. 

*Notes:

  • The Split Testing functionality relies on data from the built-in Google Analytics integration for Shopify found in the Online Store > Preferences > Google Analytics section of your Shopify admin. If you are not using this integration for your Shopify store, then Split Test functionality will not work correctly.

  • Total Views and Unique Views data is being pulled from the Behavior > Site Content > All Pages section of your GA account.

  • Conversion Rate data is from the Behavior > Site Content > Landing Pages section of your GA account.

  • Product Revenue data is from the Conversions > Ecommerce > Sales Performance section of your GA account.
    📝You must include a product featured in a Buy Box block (not including the "Collection Block") on your split test pages to record conversion rate data successfully.

Connect your Google Analytics account

ZipifyPages doesn’t collect any split-test stats in-app, but uses data collected by your own Google Analytics account instead.

If you don't already have Google Analytics connected to your Shopify store and Enhanced Ecommerce enabled in both your GA account and Shopify admin, please do so before proceeding further. The details on how to set that up can be found in this help article from Shopify: https://help.shopify.com/en/manual/reports-and-analytics/google-analytics/google-analytics-setup

Once you have your Google Analytics account connected properly to your Shopify store with Enhanced Ecommerce enabled, you need to connect your Google Analytics account to the Zipify Pages app. To do that, simply follow the steps below:

1. Navigate to the Settings > General > Tracking section of the app. Click the "Connect Google Analytics" button. Sign in to your Google account that your Google Analytics account is listed under:

2. After you sign in to your Google account, use the drop-down menus to select your Google Analytics Account, Property and View associated with your Shopify store and click the Save button. Be sure to select the correct account, property and view which contains data about your store and subsequently the pages you want to split test:

Note: If you need to connect a different GA account, click the "Connect a Different Account" button and then sign in to your other Google account instead. 

Creating a Split Test

To create a Split Test (after connecting your Google Analytics account), follow the steps below:

1. Go to the "Split Tests" section of the app and click the "Create A Split Test" button there.

Note: Be sure to prepare both pages for the test before starting the creation process. For landing pages, we recommend to duplicate the main page and then apply minor changes to the duplicated version (variant) which you wish to test against the original page (control).

2. You will then be directed to the Select Pages screen where you choose the pages to include in the test. A is the original page and B is the variation of the original that you want to test against the original.

Click the "Choose" button for Page A to select the type of pages you'd like used, Landing Pages or Product Pages (ZP Layout):

If "Landing Pages" is chosen then select Page A first. Then click the "Choose" button next to Page B to choose the variation page:

If "Product Pages" is chosen, choose the "ZP Layout" product page you'd like used as Page A. Your theme's default product page template will be used as Page B by default, so there is no need to manually choose Page B for the test. However, if you'd like a different product page template in your theme to be used for Page B instead, click the "Choose Layout" button to select the alternate template:

Choose a Test Goal from the available dropdown menu:

Test Goal (Metric)

Query in the GA API

Conversion Rate (default)

Current Implementation

Sales

'ga:transactions'

AOV

'ga:revenuePerTransaction'

Product Revenue

ga:transactionRevenue

Revenue per View

'ga:transactionRevenue' / 'ga:pageviews'

Add to Cart

'ga:productAddsToCart'

Optionally, you can change the traffic distribution model by sliding the scale towards A or B to adjust the distribution. The default for traffic distribution is set to 50/50% which is the most precise way to split test pages.

Important Notes: 

  1. You can only split-test pages which have the same setting for the “Use theme header/footer” option in the Page settings. This means this option should be either enabled or disabled for both pages of test at the same time.

  2. A specific page can only be used to run one Split Test at a time.

3. Once you've selected your pages and set your traffic distribution, click the "Save & Start" button to both save your split test and start running it. The Public Test URL is the URL that you will want to drive your traffic to from your ads/campaigns. Visitors to that URL will be redirected to control (A) and variation (B) pages based on the traffic distribution set by you. 

If you're not ready to start the test just yet, click the "Save Draft" button instead to add the test to the Split Test dashboard in a "Draft" state. You can then come back to it later without losing your initial setup work. Click "Cancel" to discard the test.

IMPORTANT NOTES:

1. Zipify Pages collects no statistics in-app, but uses data from the built-in Google Analytics integration for Shopify instead (Online Store > Preferences > Google Analytics in your Shopify admin).

2. You need to connect your Google Analytics account first by clicking on the “Connect Google Analytics” button within the Settings > General > Tracking section of the Zipify Pages app. Then use the drop-down menus to select your Account, Property and View which contain your store's data and subsequently the data about the pages you want to split-test. 

3. It's not recommended to apply changes to the pages involved in a running split test. Doing so can affect the trustworthiness of stats already collected for the test (your changes will affect the conditions that the previous stats were collected for). We recommend to create another test instead which guarantees the integrity of the experiment.

Split Test Dashboard

After you create a test and it's been running for some time, you can view its performance by going back to the Split Tests section of the Zipify Pages app. The split test dashboard section lists all your Split tests ( Running, Draft and Stopped statuses).

The Split Tests table contains stats (Test Goal (metric), Total Views, Unique Views and Status) for each page of your tests (excluding ones you haven’t started yet) as well as the Duration of the test. You can stop the Split Test at any time and then start it again later if you'd like. Keep in mind that we’re not gathering statistics for a test which has been stopped.

Note: It can take 1 - 2 hours to see the initial stats for your split test pages after a test is started since it depends on how quickly Google Analytics sends the data back. Data in the split test section updates once per hour for running split tests. 

*Conversion Rate shown here refers to "Purchase" conversions only. We take Conversion Rate data from the Behavior > Site Content > Landing Pages section of your connected Google Analytics account. You must include a product featured in a Buy Box block on your split test pages to record conversion rate data successfully.

If you wish to apply any changes to a running Split Test you should Stop the test first. Then click on the “Edit Test” action within the Actions drop-down next to the test you want changes applied to. Apply your changes and then start the test up again by clicking the "Run" button.

Split Test Details View

To access the split test details view, click directly on the split test name (1) or on the "Test Details" button (2):

There you can observe and analyze traffic per device type (1), main Google Analytics metrics for each Split Test page (2) and Total Statistics (3) in general:

Actions available for Split Tests

"More Actions" for your split tests can be accessed by clicking the three dots icon in the "Actions" column:

  • Preview Test - takes you to Page A's URL in a new tab.

  • Edit Test - takes you to the Edit Split Test screen to apply changes. The test needs to be stopped before edits can be applied.

  • Select Winner - choose either Page A or Page B as the winner and stop the test. If you choose the variation page (Page B) as the winner, then the original page (Page A) will become unpublished and the variation page will receive the original page's URL. This means that the traffic will arrive at the original URL, but that the content of the page would be the variation page's content instead.

  • Copy Test Link - copies Page A's URL to your clipboard.

  • Delete - deletes the split test. The action must be confirmed in the popup. Once confirmed, the action can't be undone.

⚠️IMPORTANT⚠️
If the losing page is being used for the:

  • "ZP Page" button/image/link Destination for any existing pages

  • "Page" Destination for header/footer links for any existing pages

...the URL for those will break (return a 404 page not found) due to the losing page's URL becoming unpublished when you pick the winner. In this case, make sure to update any "ZP Page" and/or "Page" destinations you may be using on existing page(s) in blocks after picking the winner.

Viewing Split Test data in your Google Analytics account

Landing Pages

You can see the transaction count and other data for each page in the split test. It can be seen either in the Statistics section of ZP or in the connected GA account.

In the ZP statistics section, you can search (ctrl + f) for ?zpstest and that will bring up any page that is involved in a ZP split test. Then you can check the Transactions column for the number of conversions: 

In Google Analytics, go to Behavior > Site Content > Landing Pages and search for ?zpstest to see the data:

You will be able to see Page A and Page B URLs and the data for each page this way. The text string at the end of the URLs is the split test ID and will be the same if both pages are from the same Split Test:

*Note:

Zipify Pages and Google Analytics calculate Total Conversion Rate a little bit differently, so figures between the two platforms can differ slightly. We actually do it in a more accurate way!

For the in-app Conversion Rate calculation for the split test pages, we ignore the additional page URL parameters which are appended to end of the split test page URLs in Google Analytics.

When we're gathering this data from GA, we consider all of these URLs as the same one URL, because we understand that visitors can go to the page not only via a direct URL, but also from different websites/social medias with ads which append additional URL parameters automatically.

Product Pages

To check the Total View and Unique Views related to Product pages (Zipify Layout), navigate to Behavior > Site Content > All Pages AND apply an Advanced filter (Exclude pages with URLs with view=product) like shown here in GA admin:

Comparing to the Split Test list in Zipify Pages:

To check the Conversion Rate related to Product pages (Zipify Layout) navigate to Behavior > Site Content > Landing Pages AND apply an Advanced filter (Exclude pages with URLs with view=product) like shown here in GA admin:

Comparing to the Split Test list in Zipify Pages:

Similar steps have to be done for Product pages (Shopify Layout) BUT the 'Include' option within the Advanced Filter (for product pages with URLs that contain view=product) should be selected and applied in GA admin like displayed here:

Total & Unique Views:
GA admin - https://i.imgur.com/HY3TnX2.png
Split Tests list - https://www.screencast.com/t/Y9hw1hQ5L

Conversion Rate:
GA admin - https://www.screencast.com/t/ybZrmqbc
Split Tests list - https://www.screencast.com/t/2VVWdiUFXq

NOTES:
1. To update GA tables with the most relevant data, you can click on the "Refresh Report" button here in the bottom-right of the GA reports screen:

2. Previous GA data can be cached on the Split Test list page in the Zipify Pages app. To update split tests tables within the ZP app immediately with relevant data, remove the zp.ga_reports_cache file in Dev tools > Application > Local Storage and refresh the page as shown here: https://www.loom.com/share/42e84b3a4e6f4c17bf486f6afc5ebdfc
*Dev tools can be opened by "Ctrl + Shift + I" keyboard combination or by pressing the F12 key.

Split Test Troubleshooting

If you notice that no data is being reported back for the Views or Conversion Rate statistics for your running split tests, the troubleshooting information below can help.

*Important Notes:

  • Total Views and Unique Views data are being pulled from the Behavior > Site Content > All Pages section of your connect GA account.

  • Conversion Rate data is being pulled from Behavior > Site Content > Landing Pages section of your connect GA account.

    So, there could be situation where data is mismatched between Landing Pages and All Pages and can affect Conversion rate data. It’s connected to how GA tracks a page and matches it as a Landing Page view or All Pages. 

Since the Statistics and Split Test sections within Zipify Pages relies on your connected Google Analytics account, please ensure that you have everything configured properly within your GA account and that it is connected to your Shopify store.

Specifically, please check on the following:

1. You have connected Google Analytics and enabled Enhanced Ecommerce within your Shopify admin at Preferences > Online Store > Google Analytics.

2. You have enabled Enhanced Ecommerce tracking within your Google Analytics account. 

3. You have not added an extra Google Analytics tracking code as a Global/Local script to your Zipify Pages. The Split Testing and Statistics sections in Zipify Pages uses the native Google Analytics integration in your Shopify admin at Online Store > Preferences > Google Analytics by default. Adding an extra GA tracking script to your Zipify Pages can cause issues with Split Test and Statistics tracking. 

4. You have connected the correct Google Analytics Account, Property and View which contains data about your store to the Zipify Pages app. 

If all of those things are not currently implemented, please do so and you should then start to see conversion and revenue stats show up within the Statistics and Split Test section(s) of the Zipify Pages app. 

*Note: Ecommerce tracking for Google Analytics starts to work from the time enhanced ecommerce is enabled within your account. It won't work to back-populate data for sales made prior to enabling it. 

If you've done this and you're still not seeing any data, you may need to clear your cache. Caching can cause a lack of reporting. So please check via a private browsing window or clear your cache and check the split test again.

Important notes about running your Split Tests

  1. You can’t remove or unpublish a page that is involved in a running split test. You have to stop the test first.

  2. Be careful when updating a page that is involved in a running split test since this changes the page stats the data is collected for and makes the collected data irrelevant.

  3. Once a test is started it will generate a special URL for the pages which contains additional parameters (ex: store_domain/pages/page_title?zpstest={{token}}).

    It will only appear this way for the pages in the Statistics section of ZP and in Google Analytics and all metrics will be collected for it. This means that when a split test is running, GA will stop collecting stats for the initial page URLs (store_domain/pages/page_title) and will collect stats for the special URLs (store_domain/pages/page_title?zpstest={{token}}) instead.

    IMPORTANT: Split Test pages will be available through a page's initial URL (store_domain/pages/page_title) not through the special one, so you shouldn’t worry about any changes to the links/URLs you already setup and are using in ads/campaigns. 

Did this answer your question?