ASO Academy - Mobile A/B Testing Best Practices

August 31, 2022

ASO Academy - Mobile A/B Testing Best Practices

There’s a lot of emphases placed on keywords and metadata in App Store Optimization (ASO). While critical in gaining visibility in the app stores, there are ways you can engage both metadata and creative optimization practices that all help drive conversion.

If you’ve ever asked questions like…

How does the user decide to download on the product page?

Can I influence my users to download outside of copy CTA’s and value propositions?

How do I know my product page is performing the best it can be?

…you are not alone. The rhetoric around App Store Optimization often directs developers and marketers toward focusing on the impact of keywords alone. While text and copy do have conversion effects, your creative set is directly tied to conversion.

So how can you improve conversion through an optimized creative set? A/B testing! But first, let’s get down to the basics of how the conversion funnel works in ASO.

App Store Optimization & User Conversion – How it Works

ASO emphasizes a three-step process of optimizing reach, installs, and action. Search optimization is attributed to reach as the first entry point to improving your presence in the app stores. Next, the process leads to installs, which your creative set is directly responsible for. In this step, optimizing for conversions helps drive installs. After the installation phase, ASO seeks to optimize actions – which prolongs the utility and the value of your newly onboarded users.

Notice how each of these steps helps drive the entire conversion process from inception to drop-off. It’s easy to get excited about keywords and visibility, but ultimately, conversion and prolonging the value of your users are the final steps in optimization.

It’s actually quite simple. Through relevant keyword optimization, users are able to find you. Through engaging creatives, users can be influenced by you. And finally, through an improved onboarding process, users will want to stay with you.

Mobile A/B Testing Throughout App Store Optimization

You may be wondering how split testing or A/B testing applies to any of the information from the previous section, but it’s important to reiterate that ASO is a process, and the means, methods, and purpose of A/B testing cannot be understated.

From search to action, all of the steps in an ASO strategy can (and should) be tested. Most ASO tools and native A/B testing platforms have seismic limitations in what they can do, which may dissuade their use at all, or cause developers to use them ineffectively. However, ASO tools and technology like Splitcube can help developers test all product page assets without having to deploy them in the stores in addition to a lot of other great features that simplify the A/B testing process.

Whether an app is in pre-launch, or maturity, mobile A/B testing platforms will always prove to be a valuable practice to increase conversions across the app stores.

Mobile A/B Testing Best Practices

1. Always use a control for each test

This may seem like a no-brainer, but many tests are often performed on a hunch and treated as one in the testing process without a control. Remember, all testing should be backed by a data-driven thesis. Don’t deploy changes just for the sake of making them, or that they look “cool”. Know what you’re testing, and why you’re testing, and continuously adjust.

2. Account for Known Limitations

Whether you’re using Apple’s PPO and CPP, or Google’s Store Experiments, make note of what these platforms can and cannot do. While it’s convenient to have native split testing platforms, there are still high hurdles to jump over in testing what you need to. There are limitations to understanding traffic sources, simultaneous multivariate testing, international testing, and other notable differences.

3. Iterate and Plan for Changes

User behavior trends and competitive environments never stand still – neither should your app keywords and creatives. Optimizing frequently, yet consciously to plan for future changes is a necessary step in ASO. Remember, optimization should NEVER stand still or be "one and done" if you want to continue growing.

4. Differentiate Between Paid & Organic Behavior

Know which campaigns are powered by paid or organic performance – good paid performance doesn’t necessarily suggest organic success.


A/B testing is one of the most effective and data-driven solutions to improving conversion rates. Running an A/B test can show developers what elements of their app listing perform the best. However, to properly conduct an A/B test, developers must first understand how each A/B testing platform works, what they offer, what limitations they pose, and how to address them.

Want to learn more about App Store Optimization? Contact Gummicube and we’ll help get your strategy started.