A/B Testing with Apple Product Page Optimization

Creating and designing your mobile app is just half of the work you need to do. If you want to have a successful mobile app, you must ensure that your Apple app store product page is fully optimized to attract new users and keep existing ones.

However, it is easier said than done. How do you know which settings and configurations give you the best results? How can you be sure that you have the best app icon, app previews, and screenshots displayed on your product page?

Thankfully, you don’t have to guess anymore. With A/B testing, you can try different product page configurations and identify which works and which doesn’t work.

What is Apple Product Page Optimization?

The Apple Product Page Optimization (PPO) allows you to create different configurations or treatments (alternate product page versions) of your app store product page elements. From there, you can then test them to better understand which of them leads to the best level of user engagement.

Source

The PPO was announced in June 2021 during the World Wide Developer Conference (WWDC). It’s quite interesting though that a similar feature has been available in Google Play Store since 2015.

How does Apple Product Page Optimization work?

The PPO allows you to test various elements on your app page. You can test different app page variations with a maximum of 3 different treatments.

While it is great to test all elements of your product page, you can only do it with icons, app previews, and screenshots. You can run your test for up to 90 days.

You submit your A/B test variations to the Apple team. They would then review your variations.

Obviously, you would need a way to analyze the results of your A/B test. Thus, in line with the Product Page Optimization, there is a newly added App Analytics tool that you will see on your dashboard. The tool lets you evaluate conversion rates, impressions, and improvement features.

What are the benefits of Apple Product Page Optimization?

Many Apple app store developers and marketers have long been requesting a way to run A/B tests on iOS. While they can do it on Google Play, it is a feature that would surely improve the performance of their app if they can also have it on the Apple App Store.

So, what’s so great about A/B testing with Apple Product Optimization? Here are some of its advantages:

Gain actionable insights into users’ behavior

First and foremost, A/B testing allows you to understand how users view your product page. 

Remember, in most cases, your users will first see your app product page. Therefore, your product page produces the first impression and you know what they say, “first impression lasts.”

Your app page can either convince your user to download and use your app or cause them to hit the back or close button within five seconds.

Make decisions based on data

Source

Another significant benefit of PPO is helping you make data-driven decisions. With A/B testing, you can test certain alternate variations, also known as treatments, and see which one of them works best.

If you have multiple app icons, video previews, and screenshots, you can use A/B testing to identify the ones that lead to the highest conversion rate. Since you are making choices based on data, you can potentially increase installs multiple times.

Understand users based on region

You may not know it but people living in the United States are different from people living in China, Japan, Europe, or any other part of the globe. Because of different cultural backgrounds and trends, your users may see your product page differently. 

One color or design may be attractive to one regional group while it is not as attractive as it should be to others.

With PPO, you can test treatments based on specific regions. This would further improve your conversion rate and marketing efforts around the world.

Saves you time and money

The last thing you want to do is heavily invest in something that didn’t produce positive results. You can minimize your losses with A/B testing.

For example, before launching a new app version or app page treatment, you can perform A/B testing first to see whether the new changes you made will work or not. 

This saves you countless amounts of time running useless app page treatment. Likewise, saving time means saving resources and energy as well.

How to do A/B testing with Apple Product Page Optimization?

If it is your first time doing Product Page Optimization, you may find the interface of Apple a little confusing. However, with practice and experience, you should easily get the hang of it.

Here are the steps to take:

Step 1:

Go to App Store Connect.

Step 2: 

Be sure you’re on the “App Store” tab. You should then find the Product Page Optimization page. If it is your first time starting a test, you would find the “Create Test” button. 

If you already have a test, you’ll find a (+) next to the PPO.

Step 3: 

Add reference names, which can include up to 64 characters. 

While you can choose any reference name, you may want to include a description, type of asset, and date of the test to easily recognize your tests. 

This should help you to easily identify your test once you start monitoring your results.

Step 4:

Create a product page test. You would need to specify the following:

• Number of treatments – the maximum number of treatments you can test per batch is three. The more treatments you add, the longer the test would run.
• Traffic proportion – this determines the percentage of how many users would be included in your test. 
• Localizations – choose the region which you want to include in your test. Initially, the app store would select all supported regions. So, you need to deselect the regions that you don’t need.

Step 5: 

If you wish to see how long your test may take, you can click on the dropdown menu, “Estimate Your Test Duration.” Apple uses your current performance data to give this estimate. Data include new downloads and daily impressions.

In the dropdown menu of Estimate Your Test Duration, you can indicate the improvement estimate, which is the percentage of how much you want your app page to improve.

The estimate does not affect your test, but rather, it just gives you a guide on how the test would proceed.

Please note that if you don’t have enough data, Apple may not be able to give you an estimate. While you can still create and initiate the test, your test may run for more than 90 days.

Step 6: 

Your test can run up to 90 days. You can manually stop tests. In most cases, tests can be completed within 90 days, especially if you’re just running fewer treatments.

Step 7: 

Review your test and click “Create Test” to proceed.

How to monitor your A/B test for mobile apps

You don’t have to wait until your test is completed to view the data accumulated so far. You can even view data as early as a day after you have initiated your test.

To view your app store A/B testing data, go to the App Analytics section in App Store Connect.

Once you’re on App Analytics, you’ll find various data that include percent improvement, conversion rate, number of impressions, and confidence level based on your chosen baseline. 

The baseline is usually the original product page unless you have changed it. You can change the baseline and switch it to other treatments. This way, you can compare not just one treatment against your product page but also one treatment to another.

To help you monitor closely your tests, your App Store Connect page shows indicators as long as your tests are running. This will help you know the current status of your test. 

For example, an indicator can tell you when you need more data. An indicator can also tell you how your treatments are performing compared to the set baseline.

If you wish to better monitor your data and even learn how to use it to achieve your mobile app marketing goals, you can seek the professional help of ShyftUp. They have the relevant skills and expertise in creating more business opportunities for your app.

How to apply a test treatment

Based on the results you received from your A/B testing for mobile apps, it’s time to make a choice. It is best to make a decision once there’s a clear winner in your test. At best, you want to hit more than 90% confidence.

If ever you believe that a treatment is better than your existing product page, then you can choose to use that.

Source

Here are the steps to apply a test treatment:

1. Go to My Apps.
2. Select the app involved in the test.
3. Click Product Page Optimization under Features, which can be found in the left sidebar.
4. Search for the specific test and click on its name.
5. Find the treatment you want to apply.
6. Select Apply Treatment to Original Product Page.
7. Choose which app versions you want your treatment to take effect.

Quick notes about Apple Product Page Optimization

Before you end reading this article, there are a few things you need to remember:

• You can apply treatments not just on your App Store page, but also to various versions of your Ready for Sale or Prepare for Submission. Be careful when applying treatments. Once you proceed to apply treatments, you won’t be able to undo it anymore.
• If you don’t see the option for Product Page Optimization, one common reason is that your app’s status isn’t considered as “Ready for Sale.” Moreover, avoid submitting a new app version when you are currently running a test. Unless you intend to do so, this will automatically halt your test.
• Please note as well the Product Page Optimization is not available for Apple Watch and iMessage app store page. It is not available as well for custom product pages.
• Finally, if your treatment includes an app icon, you would need to submit and set it as a default icon in your new app version. However, for app previews and screenshots, you can apply them directly as part of your chosen treatment.

Go deeper with your analytics

As wonderful as the new Apple Product Page Optimization is, you can still get additional help from ShyftUp.

ShyftUp is among the best user acquisition agencies in San Francisco that work with hundreds of companies worldwide. We have the right technology, tools, and experience to exponentially grow your mobile app’s user base.

Contact us today and schedule a demo. See how they can help you amplify your app’s reach and success.

Is Product Page Optimization free?

Yes, Product Page Optimization is free as long as your app is already Ready for Sale.

How to manually stop an A/B test?

To stop a test, you can go to My Apps > Features > Product Page Optimization > click the name of the test > select Stop Test.

Are A/B tests available on Custom Product Pages?

No, but you can use paid UA campaigns to test various variations of your product pages.