Creative Experimentation vs. Evidence Experimentation - Actively exploring new growth paths

Let’s think about a new way to describe two areas of Experimentation: “Creative Experimentation” and “Evidence Experimentation.” Creative is meant in the sense of creating something new or discovering something new. Showing how experimentation can not only guide but lead growth.

‘Evidence Experimentation’ to gather evidence and reduce risk

Evidence Experimentation refers to the A/B tests we run to check and verify the impact of new features and ideas. Whether it’s a new feature for the product, a new marketing campaign, a new slogan, or a layout to test. These kind of tests are run for any changes any team wants to introduce.

The goal of Evidence Experiments is to verify the impact of different changes that are introduced into a product or website, or at least to prevent newly introduced changes from having a negative impact on our customers. In this case, Evidence Experimentation mostly is a ‘protective’ measure to reduce risks.

‘Creative Experimentation’ to actively find new growth opportunities

Creative experimentation, in contrast, is actively learning how to improve things by rapidly testing new ideas and measuring customer feedback. We focus on bringing new ideas in front of a small group of our customers quickly in order to learn what they think about them and find new growth opportunities.

In this case, experimentation is an ‘active’ measure to find new improvements and independently drive growth for the company.

Distinguishing features

Evidence Experimentation	Creative Experimentation
When new changes are introduced, monitor their impact. These changes are often planned without ‘Experimentation’ in mind.	Specifically introduce new changes for the purpose of testing them.
💬 “We think we have a good understanding of what the customer wants and already know what the solution is. Let’s implement the solution and then AB-test to measure the impact.”	💬 “We don’t have a good understanding of what the customer might prefer. Let’s create a couple of hypotheses in different directions and run tests to explore what the customer thinks.”
⏱️ Variations for these kinds of tests are usually prepared with some lead time, and a lot of research goes into designing them.	⏱️ Variations for these kind of tests are created and approved faster.
⭐ Variation quality before the test starts is close to 100%.	⭐ Variation quality before the test starts is around 70-80%. Once the hypothesis is confirmed, the quality can be improved.
🎯 A process that guides which new changes are introduced and provides feedback to teams.	🎯 A process that leads which new changes are introduced and creates learnings to share with the whole company.

Creative Experimentation at pagent

At pagent, we apply this principle to text, image, and (soon) layout on websites. Using a recommendation algorithm, pagent creates different strategies and implements these to appeal to different emotions, problems, and benefits that the customer has. With an easy review process, adherence to compliance and guidelines is ensured.