Yesterday I listened to the Masters of Scale podcast with Marc Zuckerberg, co-founder and CEO of Facebook. Highly recommended.
Lots of great stuff around starting the company, growing it, hiring people and … my favourite topic: experimentation (A/B testing). Here are some quotes from the transcript.
“I think the strategy of Facebook is to learn as quickly as possible what our community wants us to do—and that requires a culture that encourages people to try things and test things and fail.”
Marc Zuckerberg, timestamp: 03:28
“At any given point in time, there isn’t just one version of Facebook running, there are probably 10,000. Any engineer at the company can basically decide that they want to test something. There are some rules on sensitive things, but in general, an engineer can test something, and they can launch a version of Facebook not to the whole community, but maybe to 10,000 people or 50,000 people—whatever is necessary to get a good test of an experience. And then, they get a readout of how that affected all of the different metrics and things that we care about. How were people connecting? How were people sharing? Do people have more friends in this version? Of course, business metrics, like how does this cost the efficiency of running the service, how much revenue are we making?”
Marc Zuckerberg, timestamp: 22:23
“It can even kick off qualitative studies and ask people how happy they are with this version. And then at the end of that, the engineer can come to their manager, and say, “Hey, here’s what I built, these are the results. Do we want to explore this further and do this?” And giving people the tools to be able to go get that data without having to argue whether their idea’s good through layers of management before testing something, frees people up to move quicker. If the thing doesn’t work, then we add that to our documentation of all the lessons that we’ve learned over time. If it does work, then we can incorporate those small changes into the base of what Facebook is—that now everyone else who is trying to build an improvement, that’s the new baseline that they need to get against.”
Marc Zuckerberg, timestamp: 24:05
