Experimental Mind: Decision Making, Lighthouse Metric, Dark Patterns, Thinking Modes …

Hi, in this week’s Experimental Mind newsletter 8 articles to read and 20 job opportunities to explore. Enjoy.

ps. as you might have noticed, this email looks a bit different. I’m testing out another email provider (Revue). Let me know if you see any strange things.

What I’ve been reading this week

1. Decision Making at Netflix

Part 1 in a multipart series about decision making and experimentation at Netflix.

At Netflix, we believe there’s a better way to make decisions about how to improve the experience we deliver to our members: we use A/B tests. Experimentation scales. Instead of small groups of executives or experts contributing to a decision, experimentation gives all our members the opportunity to vote, with their actions, on how to continue to evolve their joyful Netflix experience.

ps. Netflix is looking for an Experimentation & Causal Inference Intern

2. A/A’/B Testing: Evaluating Microsoft Teams across build releases

A/B testing is rarely used for comparison between whole builds. That’s because each build integrates multiple feature changes and it is hard to figure out which features cause regressions, if any. However, the Microsoft Teams Experimentation team did make an attempt to use A/B testing as an integration testing tool for builds comparison.

3. Why the term 'North Star Metric' is a terrible metaphor for product success
3. Why the term ‘North Star Metric’ is a terrible metaphor for product successmedium.com

Bhavik Patel suggests we start calling it ‘Lighthouse Metric’, not NSM.

4. The use of Dark Patterns [paper]

In this study the authors look at 50 types of dark patterns across the desktop web, mobile web, and mobile app modalities of 105 services. Most used dark patterns are privacy related, see table below.

5. Outdated belief: A/B testing is only for online systems

Jan Bosch explains why:

A/B testing is relevant in any context where a mathematical optimum can’t be established easily. As such, it’s relevant for virtually any company and system. With the increased connectivity of systems and a continuously growing expectation of improvements to systems throughout their entire economic life, it’s clear that A/B testing is a capability that every industry, company and team needs to build up.

6. The Framework You Need for Successful Marketing Experimentation

Jason Widup on how to get a systematic plan to run effective marketing experiments and learn from them.

7. Lessons from 7,000 people about ads and privacy

Google asked Ipsos to interview thousands of people about their personal privacy and data. Three tips for defining your data strategy:

  1. Make it meaningful
  2. Make it memorable
  3. Make it manageable

8. Adam Grant explains the 4 modes of thinking.
8. Adam Grant explains the 4 modes of thinking.www.inc.com

I think too many of us spend too much time thinking like preachers, prosecutors, and politicians … We need to think more like a scientists.

Job opportunities

This week’s featured jobs:

Don’t forget to check out the other 15 opportunities on the job board. Are you hiring? Simply add the vacancy to the board.

Upcoming events

Quote of the week

If you’re not experimenting, your baseline assumption is that whatever you’re doing now is perfect — Richard Mathera

Fun of the week

As always, if you’re enjoying this email , I’d love it if you shared it with a friend or two.

You can send them here to sign up. I try to make it one of the best emails you get each week, and I hope you’re enjoying it. And should you come across anything interesting this week, send it my way! I love finding new things to read through members of this newsletter.

Have a great week — and keep experimenting.