Experimental Mind: Kitchen Mindsets, Data Habitat, Experiment Capacity, Detecting SRM …

Hi friends,

Last week me and my family spent some time in the German Black Forest area enjoying the autumn colours. But I also managed to read some work related stuff. So in this week’s newsletter: 5 articles, 17 job opportunities and an updated list of upcoming events.

Enjoy!


🔎 What I’ve been reading this week

1. The test kitchen mindsets | Seth Godin

Many times, we optimizers follow the incremental path: make tiny changes and measure the impact. But as Seth Godin explains, there is also an alternative mindset:

Do things that might not work. Develop new assertions. Go past the edges to unexplored territory. Try to figure out why things are the way they are. Fail often. Blaze a trail. After all, it’s a test kitchen, not a Michelin restaurant.

2. Ideal habitat for data | Avinash Kaushik

In Avinash Kaushik’s latest newsletter he reflects on how to create this ideal, healthy habitat where data informed decision making can thrive.

I’m sharing four patterns that occur most frequently in non-ideal habitats for data, and data-people. Across those four patterns, I’m sharing six questions to spark introspection and ideas to identify your organization’s specific issues. Fix the four patterns, unlock a difficult to replicate competitive advantage!

3. Behavioural science in the sign-up flow

Kai Mackie explains how he and his team at Gousto are applying techniques from behavioral science in improving the sign-up flow.

4. Principles to increase experiment capacity

DoorDash’s Assignment team adopted an experiment-driven framework to determine the outcome of any change with scientific precision and run this analysis at scale. 

5. Detecting Sample Ratio Mismatch in your A/B Tests

In this article Frank Hopkins will breakdown how to perform a SRM test in Python, as both a post-hoc test and for individual increments throughout your experiment.

🚀 Job opportunities

Are you looking for a new opportunity? There are now 17 open roles on the job board.

This week’s featured jobs:

  1. Experimentation Expert at ING (Amsterdam, NL)
  2. VP, Digital Experimentation at Chase (Wilmington DE, US)
  3. Product Manager, Experimentation at Adyen (Amsterdam, NL)

And did you know? You, as subscriber of this newsletter, can post for free to the job board. Post a job and use this code at the last step: SUBSCRIBER200

📅 Upcoming events

This is a running list of upcoming events:

💬 Quote of the week

“Customers don’t know what they feel, don’t say what they know, and don’t do what they say. Market research is three steps removed from real behaviour.” — David Ogilvy

😉 Fun of the week

Jono Hey from Sketchplanations.com on ‘sampling bias‘:

if you hope to receive responses to your questions by surveys then you get responses from people who happen to not mind doing surveys — from anyone who throws them in the bin you won’t receive a single data point.

🔁 Any feedback?

I like to hear what you think of this newsletter, Share your feedback, it takes less than a minute.

🙏 Thanks for reading

If you’re enjoying the Experimental Mind newsletter you can buy me a coffee or beer. Thanks to the 8 people who already did it. Much appreciated.

And share this email with a friend or colleague. I would love this community of experimenters to grow. You can point them here to subscribe.

Have a great week — and keep experimenting.

Kevin