Experimental Mind: GitLab, Change, Data PM’s, DoorDash, Risky Bets …

This is your weekly roundup of interesting reads around experimentation with updates from Gitlab, DoorDash and Statsig. I also added three new events to the list. Enjoy!

🔎 What I’ve been reading

Experimentation at GitLab
Experimentation at GitLababout.gitlab.com

This documentation describes GitLab’s Growth engineering process for running experiments. Extensive read.

Top 10 Non-Controversies in A/B Testing

Great LinkedIn post by Matt Gershoff sharing his top 10 non-controversies in A/B testing. Number 10: Two Tailed vs One Tailed Tests.

Generalist vs Subject Matter Expert: Companies Grapple with How to Use Data Product

Eric Weber on the upcoming role of Data Product Manager, also for experimentation:

… the ability to have a real product experimentation platform is transformational for many companies. An experimentation platform enables companies to identify causal relationships between product changes and user behavior (or other metrics). This space is *extremely* competitive right now, and I’m a huge fan of many startups that have launched here. For data product managers, experimentation platform requires a vast array of context: from experimental design, to cohort allocation, to logging, to guardrails metrics, to experiment readouts.

How change happens
How change happensseths.blog

Working in the experimentation space can sometimes feel like an uphill battle. Reminder from Seth Godin: persist. This is how change happens.

Focused, persistent community action is how systems change. And systems concretize and enforce cultural norms.

If you care, keep talking. Keep acting. Stay focused. And don’t get bored.

The Statistics Engine of Experimentation at DoorDash

DoorDash’s centralized library for statistical analysis of experiments.

We needed a way to enforce standardization and facilitate knowledge sharing to improve the velocity of our experiment analysis.

Pulse Time Series for Deeper Insights
The Statsig Pulse results page offers a snapshot of all the metric movements driven by an experiment. Sometimes, a brief scan of the color coded score card is enough to validate that all metrics…

Experimentation Culture Awards 2022
Experimentation Culture Awards 2022experimentationcultureawards.com

I’ll be one of the jury members at the 2022 Experimentation Culture Awards. Join the award session and listen to inspiration stories.

🚀 Job opportunities

All featured roles:

Also checkout the job board for more open roles in the field of experimentation, CRO and analytics.

📅 Upcoming events

Final call for joining Ronny Kohavi’s new class, where he will teach you everything he knows about A/B testing in 5 two hour sessions. First session is May 31. Subscribe via this link to receive a $50 Amazon gift card.

All upcoming events in the next months:

Second batch of tickets will release soon:

See the full overview of events to find other events and conferences for you and your team.

💬 Quote of the week

Amazon’s Risky Bet, according to BusinessWeek in 2006.

[tweet https://twitter.com/JeffBezos/status/1527003895393812480]

☕ Thanks for reading

If you’re enjoying the Experimental Mind newsletter and want to say thank you? Buy me a coffee (or a beer). Or share this newsletter with a friend or colleague.

Have a great week — and keep experimenting.