Experimental Mind: Mind the gap

Hi everyone,

In this weekly newsletter I share the best experimentation related articles, jobs and events, that I have found during the week. This week I have selected 9 articles for you. Plus 38 job opportunities & over 15 events for you and your team.

So, grab your cup of coffee (or tea), sit down and take it all in.

🔎 What I’ve been reading last week

Mind the gap

A well, researched blog post by Davis Treybig on the gap between companies who truly get experimentation, and the rest.

… most companies can not, despite their best efforts, realistically achieve an experimentation program anywhere close to what the aforementioned companies have.

If you read one article this week, let it be this one.

Build vs. Buy is a false dichotomy

In Ronny Kohavi’s latest A/B testing class, three agencies were asked to present their views on the different tools. If you are looking into buying a new tool or building (parts of) an experimentation platform, you should definitely read the slides that were shared.

Build vs. Buy is a false dichotomy. You may buy some components and build others into a hybrid.

Best online resources to get into Behavioural Science

Merle van den Akker created this list of online resources to learn more about behavioural science. This is a follow-up of her books & papers post.

Duolingo's Expurrimenter tool

Blog post about Duolingo’s experimentation tool for copy tests.

… there was a big payoff once the changes were made and the tool was built — within the first three months with Expurrimenter, we more than doubled our all-time volume of copy tests.

How to wisely use social proof
How to wisely use social proofnesslabs.com

From one of my favorite blogs: Ness Labs.

Social proof can lead us to act in ways that do not align with our beliefs, and that morally may not be defensible. Learning when and how to use social proof to make decisions will allow you to benefit from the wisdom of others, without encountering the associated pitfalls.

How to correctly interpret p-values?

Often misunderstood concept, so good reminder:

P-value is a probability value corresponding to the likelihood of obtaining a data value (‘test statistic’), which is at least as “extreme” as the actually observed data value (observed ‘test statistic’), under the assumption that Null Hypothesis is correct.

P-value provides an answer to: Assuming no-difference (i.e. Null Hypothesis), what is the likelihood of seeing the difference we are observing in the data.

Rory Sutherland: how do people behave?

Post by Rory Sutherland linking direct marketing, behavioral science and the importance looking at people’s behavior.

David Ogilvy used to say that every copywriter should start off by working four years in direct marketing before they move into general advertising – because in direct marketing you find out what works.

Direct marketing is basically a vast series of behavioral science experiments performed at scale.

In every circumstance, we must not assume that what we think we know or is always true – we should start by asking, how do people behave?

The Experimentation Journal
The Experimentation Journaltwitter.com

Joe Goodman created this Experimentation Journal to keep track of all those personal experiments in Notion. Get your free copy here.

Book review: 'Feature Management with LaunchDarkly'

Ronny Kohavi reviewed the book ‘Feature Management with LaunchDarkly’, including what he liked and didn’t like.

🚀 Job opportunities

CRO Specialist at RTL
CRO Specialist at RTLexperimentalmind.com

DUTCH: Wil jij grote digitale platformen zoals Videoland, RTL Nieuws of Buienradar helpen optimaliseren? RTL is op zoek naar jou.

All featured roles:

Or checkout the other 28 open roles.

Hiring? Post your open roles to the job board (from €0 for you)

📅 Upcoming events

This is a running list of upcoming events:

See the full overview of events to find 15+ other events and conferences for you and your team.

💬 Quote of the week

From 100 ways to make your working life a little easier:

“Adopt an experimental mindset that aims to try new things in the pursuit of learning.” — James Chudley

😎 Cool thing of the week

In 2021, Cees Bassa’s camera took a photo of the Dutch sky every 15 seconds.

This image combines the 2.1 million images into a year-long keogram, showing the length of the night change during the year, the motion of the Sun and Moon, and lots and lots of clouds.

[tweet https://twitter.com/cgbassa/status/1479480655822172170]

🥑 Thanks for reading

If you’re enjoying the Experimental Mind newsletter, buy me a coffee, beer or an avocado. Do you prefer to use Paypal? Send it to ‘kevin@experimentalmind.com’. Thanks to the 20+ people who have already done it. Much appreciated.

Or share this newsletter with a friend or colleague (just like Luka and Siddhartha did). You can point people to:getrevue.co/profile/kevinanderson

Have a great week — and keep experimenting.


ps. this is a tool I recently discovered 👇

💌 A message from The Sample

Discover newsletters with The Sample
Discover newsletters with The Samplethesample.ai

The Sample is a newsletter discovery service. After you sign up, they pick a different newsletter to forward you each day. If you like it, you can hit “subscribe in 1 click”. Great idea, great service.