Kevin Anderson

This is an overview of articles that I read this week and shared via my weekly newsletter. It covers the best of analytics and experimentation related content.

1. One Nation, tracked
Massive investigation into the smartphone tracking industry by The New York Times. Read the article.

2. Biased Algorithms Are Easier to Fix Than Biased People
Racial discrimination by algorithms or by people is harmful — but that’s where the similarities end. Also from NYT. Read the article.

3. Refuted Causal Claims from Observational Studies
Ronny Kohavi shared a new chapter from his upcoming book (available for pre-order). “We review famous examples where causality was claimed as likely in observational studies, but later refuted in studies higher in the hierarchy of evidence, such as randomized controlled experiments.” Read the article.

4. How Imperfect Foods built a culture of experimentation
Read the article.

5. Extending monitoring from application performance to features
Lizzie Eardley from Split explains the advantages of feature monitoring. Read the article.

6. The 3 A’s of becoming a data-driven organization
Accurate, Accessible and Actionable. Read the article.

7. Build a Career in Data Science (book)
Emily Robinson and Jacqueline Nolis have released their new book.
Read more.

This is an overview of articles that I read this week and shared via my weekly newsletter. It covers the best of analytics and experimentation related content.

1. Recap Conversion Hotel conference 2019
Two weeks ago I visited the ‘Conference Formerly Known as Conversion Hotel’, now known as #CH2019. This was already my fourth visit to what I believe is the best conference in The Netherlands for people into data, analytics and experimentation. Read/watch links to all video summaries, slides, notes, questions and pictures (and get on the notification list for next year).

2. Cookiestatus.com
Earlier this year Simo Ahava already wrote a good blogpost on Intelligent Tracking Prevention (ITP). He recently released Cookiestatus.com: “a knowledge sharing resource for the various tracking protection mechanisms implemented by the major browsers and browser engines”. Mandatory reading material for everyone collecting and/or analysing online data. Go to cookiestatus.com

3. Creating the Experimentation Organization
A short interview with Stefan Thomke about experimentation and his forthcoming book, ‘Experimentation Works‘. “Companies better get on this [experimentation] if they don’t want to be at a major competitive disadvantage.” Read the article

4. The North Star Playbook
John Cutler from Amplitude put together this nice overview of how to define a North Star Metric. Including this checklist with the characteristics of a strong North Star:

• It expresses value. We can see why it matters to customers.
• It represents vision and strategy. Our company’s product and business strategy are reflected in it.
• It’s a leading indicator of success. It predicts future results, rather than reflecting past results.
• It’s actionable. We can take action to influence it.
• It’s understandable. It’s framed in plain language that non-technical partners can understand.
• It’s measurable. We can instrument our products to track it.
• It’s not a vanity metric. When it changes, we can be confident that the change is meaningful and valuable, rather than being something that doesn’t actually predict long-term success—even if it makes the team feel good about itself.

Read the article

5. The 11 best A/B Testing tools in 2020
… according to Alex Birkett. Read the article

6. Using R to synthesise long-term findings with Meta-analysis at the BBC
Frank Hopkins from BBC’s Experimentation & Optimisation Team writes why meta-analysis of A/B tests are important. And how they do it. Read the article

7. The 2019 State of Conversion Optimization Report
CXL’s fourth edition of the State of Conversion Optimization report. Biggest challenges: better processes and buy-in from decision-makers. Download the full report