In this article, I’ll talk about the importance of setting up Marketing experiments, measuring them and keeping track of your key metrics. We’ll look at example experiments and I’ll explain how you can get insights from them. This system is built upon my main mantra for data informed marketing:

Everything you think is only an assumption. You need to verify your assumptions by running experiments and delivering evidence.

Data-Informed Marketing

You should look at the outcome of your experiments on at least a weekly basis. We want to keep track of the data as it helps us realize very quickly when things are working for us. If they don’t, we use our insights to choose a different route.

The aim of data-informed marketing is to learn fast, to not repeat your mistakes and to not waste time. We don’t want to run in circles.

We always want to document our learnings so the whole team can benefit from them and we can reference them at any point at a later time.

Following this process ensures we don’t just work on things because we think they work but because they actually do.

The Importance of Analyzing Marketing Experiments

When building a business or a product you want to make sure you use your time wisely. Use it for things that actually move the needle.

The most important thing when running experiments is the ability to analyze them. If you think about it… Only 20 years ago, this was something that marketers really struggled with. You would have a great idea and use many resources (time and money) to make it come to life. But how would you track your target audience’s actions? How would you know how far down the funnel they actually made it? Marketers had to rely on people’s good will to respond to letters or to answer questions truthfully on cold calls.

Thankfully, nowadays we have products like Google Analytics, Optimizely, Heap Analytics, HubSpot, and many more that enable us to follow a user’s behavior. It’s not about the products though; it’s what they allow us to see. We have the possibility to look at the actions of specific users at any given point in time.

These tools guide us in defining funnel stages, their key metrics, setting email automation trigger points, and bringing attention to problems in our product. In the end, they help us create a better product for our users. And from a budget perspective, they also help us Marketers justify our marketing spend.

Numbers are people and they deserve to be understood.

The experiments that we run are not intended to immediately generate X% more revenue or bring in Y% more unique visitors. Their main purpose is to help us understand our customers, where they come from, what they struggle with, and how we can best attract them.

Working within such a system, analyzing our experiments, gaining learnings from them and consequentially making data-informed decisions gets a lot easier with time and is actually a lot of fun!

As a result, you will feel more confident that you’re headed in the right direction and what your next steps should be.

So, very importantly: Make sure to frequently make the time to analyze and gain learnings from your experiments.

Estimating the Outcome of your Experiments

The easiest way to learn is by recording estimations prior to launching new campaigns. This forces you to think about potential outcomes and makes you commit to a goal you sincerely believe you can achieve.

Don’t be too hard on yourself in the beginning. Making precise estimations is not easy, but you will improve over time.

It is important to give yourself a threshold of how far off you allow yourself to be. If you are way off you should dig deeper and find out what didn’t go as expected (be that in the positive or the negative direction). Let’s look at an example:

You offer a service that offers German-speaking Hamburger cooking classes. You produced beautiful banners and you use some of your budget for paid advertisement. You expect a certain amount of click-throughs and signups to your classes. After the experiment, you analyze your numbers and realize that you had the expected amount of unique visitors, but a very bad conversion rate for signups – it is way outside of the threshold you set.

So you start digging deeper: You quickly discover that the setup of your paid ads was wrong! You targeted all languages. English speaking cooking aficionados clicked on your banners because they saw something about “Hamburgers”. So did German speaking visitors, as they are using the same word for this food. But, your landing page is written entirely in German! You paid for all of those clicks from non-German-speaking visitors and they literally didn’t understand a word you were saying.

You would not have discovered this problem (at least not as quickly) had you not analyzed your experiment and estimated an outcome prior to setting up your campaign.

I’ve seen some of my clients turn this process into an actual “game” that makes sure people keep motivated and actually evaluate their actions before they run an experiment. Everybody from the Marketing team has to commit to his/her estimations and when the experiment is finished the person closest to reality receives a special gift from the rest of the team.

As mentioned above, it’s critical that as you conduct experiments, you take notes of your estimations, results, and learnings. This way, your teammates (and future you) can learn from them. Information like this is also invaluable to new team members as they are able to reconstruct winning formulas and take a historical look at previous activities, which should speed up onboarding orders of magnitude.

How to Keep Track of Your Marketing Experiments

Now that we know experiments are important, that we should track them, and that we should have estimations in place prior to launching our campaigns how do we actually track our experiments?

It’s important that you are tracking your experiments, the method doesn’t necessarily matter.

As you can see on my about page I love spreadsheets, so that’s what I - personally - feel most comfortable with. I’ve had clients set up this system within Trello boards or also on good old analogue paper though.

I like using spreadsheets but you can conduct these experiments using whatever tools you’re most comfortable with.

How to Set Up a Data Informed Marketing System

Firstly, it is important to define the channels you’ll be working with. For example: Blog, Newsletter, Podcast Sponsorship, LinkedIn, or others. If you’re looking for inspiration on channels I recommend reading my article on Creating Repeatable, Measurable Marketing Building Blocks where I talk about how you can identify profitable marketing channels and campaigns.

Secondly, you need to make sure you have the most important stages of your funnel represented and tracked. For example, if you run a SaaS product you most likely will want to look at metrics like Unique Visitors, Signups, Activated Users, and Paying Customers.

Thirdly, get the numbers for a certain period of time for these stages of your funnel. I like to look at them in a month-by-month basis because with a lot of activities (especially paid) it’s easier to map expenses (and your given budget) and outcome together this way.

With the previously mentioned products, all it takes is the correct setup (ask your developers to help you) and some time to create a structure that might look something like this:

August – Twitter:

September – Twitter:

Now that you have these metrics, you can calculate Conversion Rates:

125 Signups / 2,140 Unique Visitors, for example, gives you a conversion rate of 5.84% in September. Do the same thing for the remaining stages of your funnel. Look at these numbers frequently and keep track of them for your most important channels. After only a few months you will get meaningful insights. This is where your main marketing learnings will come from.

When you set up experiments, measure them and record estimations prior to launching your campaigns all these questions will quickly turn into meaningful answers. The numbers (=people) are telling us what to work on.

Data-Informed Marketing: A Real-Life Example

Now that you’re tracking your channels month-by-month, it’s easy to calculate their average performance. If we are defining a new experiment, we now have data to base our estimations upon.

Let’s say you want to increase the Signup rate for one of your Landing Pages.

How about this experiment: Start putting Twitter testimonials on the Landing Page to provide social proof. Our assumption is that this should help create trust. And more trust should lead to more signups.

Previously, this Landing Page got 1,000 Unique Visitors and 70 Signups per month giving you a Conversion Rate of 7%. The amount of people hitting your Landing Page won’t change that much by running this experiment. But we think our signup rate might increase. Looking at your data, you can base your estimation on an average of 7% and commit to a predicted rise in Conversion Rate of 2.5%, estimating 95 Signups next month.

Actionable Tasks as a Result of Your Experiments

Did this experiment work? Did it increase your Conversion Rate to Signups? Document it for your team and define actionable tasks.

In this particular case: From now on every Landing Page will show Twitter testimonials.

Conclusion

We are living in exciting times in which you can visualize the depths of your funnels and track customers on their journey from 1st visit towards loyal, long-term customer.

It’s on you to use this to your advantage and gain learnings from the tools available.

Most importantly, you need a process in-place. This article describes this process. But, your team may approach things differently. That’s fine, my system is not meant as a strict prescription. It’s important you make sure though you don’t stop at making assumptions. Always try to get learnings from what you are doing.

Lastly, don’t get caught in the trap of exclusively executing on experiments. As mentioned above, frequently make time to re-evaluate your channels and their performance.