No traffic to test? No problem!

Hi there ,

Every person in growth will tell you to be testing.

I’m like a broken record, advocating about the power of experimentation.

But to see the results of a certain type of experimentation A/B testing, you need people to try each version. You need the traffic to support either A or B (or C, D….).

Otherwise you end up running tests that are unreliable or seem to show no significant difference. Which doesn’t get you any further in the process and just wastes your time.

So if you don’t have the traffic just yet, which is common with start-ups, how can you still test and improve your business?

Let’s dive in.

How do I know if I have the traffic to A/B test?

When it comes to A/B testing, you don’t have to simply guess. You can calculate your Minimum Detectable Effect, meaning the minimum effect you can measure with a test.

The Minimum Detectable Effect depends on 7 main factors:

1. Traffic

2. Conversions

3. Conversion rate

4. Number of variants

5. Confidence level

6. Statistic power

7. Test duration

Basically, the lower your traffic or other variables, the longer you’ll need to run the test.

 Calculate it for yourself to see what you get.

But don’t stress, I’ve got 4 other ways you can test without the required traffic.

No traffic? No problem

Often, you’ll be advised not to bother testing without enough traffic, or to focus on metrics earlier in the funnel. But even in those early days of low traffic, testing can make such a big difference for your growth.

And it can feel like an endless cycle, as you need those tests to improve your traffic so that you can run tests.

So let’s not shy away from that challenge, let’s climb this mountain together.

1. Preference testing

With a tool like UsabilityHub, you can display two variants and then ask follow-up questions about their preference between the two.

This allows you to test both copy and design changes. You have the option to use their panel or use your own list to create a relevant demographic for feedback.

Here is an example of a messaging test:

It’s not linked to a direct sale, but it’s an opportunity for learning.

You can also choose to run a five-second test where you only show one variant and ask for feedback, which is ideal for identifying clarity issues.

2. Meta ads

With meta ads, you can test everything upfront. It’s particularly useful for copy changes. But ensure it remains a fair test by testing the actual copy vs the image and copy in one.

When it comes to the expected budget for ads, I look at the average CTR and CPC combined with the Minimum Detectable Effect calculator I mentioned previously. Usually, it works out to between £500 and £1,200 for a test.

Don’t forget to use the experiment feature of Meta ads to set it up as a fair test!

3. Email testing

With email testing, you can measure the differences either in your open rate or click-to-open rate (CTOR) for two variants.

You tend to see higher volumes, as well as open rates and CTOR, than you do for your conversion rate. This allows you to go further with a smaller sample size (usually you'll still need at least 2,000 e-mail subscribers)

This is again ideal for testing copy and adapting your messaging angle. I usually run these tests on non-purchasers and run several.

4. User testing

Finally, one that I think does not get the attention it rightfully deserves!

A few months ago, a client did 15 user testing and the insights were mind-blowing. We got so many insights and were able to perform wonders on improving the landing page.

Assuming you don’t have the traffic to measure before/after with user testings, it’s important to consistently conduct rounds of testing to see how your changes are perceived.

And as for finding participants, tools like Userfeel can provide you with individuals for either moderated or unmoderated user testing.

Most people think user testing is time-consuming and a big project, it doesn't have to be.

Here is a complete guide to user testing in just one week that I created together with Principal Experience Designer of Kooth, Georgia Sugarman:

Learn faster without the traffic

Even though these 4 types test earlier in the funnel (no purchases, just looking or giving feedback), they can allow you to learn a huge amount in a short period of time.

I recommend using 2-3 methods or rounds of tests to improve your reliability, especially if it’s a major change.

Need a hand implementing them?

Check out my Message-Market Fit Programme which covers all of these techniques in-depth, with practical application tips. Tests like these are integral in finding the right messaging.

(If you would like the testing technique part of the programme, Week 7 - 10, hit reply and I'm happy to offer that separately)

Recommendation

In every edition of Growth Waves, I also share a related book, individual or newsletter to check out related to the week's topic. No affiliates, just content I love or can recommend.

In this hands-on guide, Jaime Levy helps you and your team craft innovative multi-device products that people actually want to us.

Plenty of UX/UI designers and product managers have recommended this book to me, and I think it’s just as relevant for entrepreneurs!

Low traffic can often feel like a huge hurdle for start-ups, and make testing feel impossible. But all it takes is a little effort and a lot of creativity.

In moments like these, don’t simply follow the rules that were set, but say screw them and find another way to your goal.

See you next week!

Daphne

P.S. Did someone forward you this e-mail? Sign up for weekly actionable growth tips here.