1

I got feedback about experimenting, and I'm unsure how to process it.

Profile picture
Mid-Level Software Engineer [E4] at Meta2 years ago

I had a feedback session with my manager, and they gave me feedback that I was doing well overall but they would like to see me become more familiar with Meta's experimentation tools. They mentioned that this wasn't bad feedback as this unfamiliarity is expected with relatively new Meta engineers like myself. That being said, I'm unsure how to process it: Should I process it urgently and if so, how do I become familiar with Meta's gigantic experimentation infrastructure?

139
1

Discussion

(1 comment)
  • 3
    Profile picture
    Robinhood, Meta, Course Hero, PayPal
    2 years ago

    I don't think you need to process it urgently, but I heavily recommend familiarizing yourself with it in the next 3 months before PSC starts. If you can run a few experiments end-to-end by then (come up with goal metrics, set up the experiment, launch it, analyze the data, and probably write a nice Workplace post about it), that would be great.

    Experimentation is part of the lifeblood of Meta (this is what you get in a company that's both so big and so metrics-driven), so it's effectively a requirement to understand for almost all software engineers there. Being very skilled at it is very necessary going from E4 -> E5 in most orgs.

    That being said, the experimentation infra at Meta is so, so complicated. I really don't recommend trying to understand on your own by playing around with things and reading the documentation - Even at Meta's current scale, it evolves too fast that the documentation/resources can't really catch up. I heavily recommend just having a 30-45 minute meeting with a tenured engineer on your team where they just explain all of Meta's experimentation concepts to you - That's how I learned to run experiments at Meta. Before this meeting, come up with a good set of questions to ask. To help you out, here's some questions you can potentially use:

    1. How does our team use Deltoid and Scuba? Which is more important?
    2. How do we use gatekeepers, quick experiments, and launchers?
    3. What is a universe experiment?
    4. What's the point of a holdout group?
    5. What's a backtest?

    And with each of these questions, you can ask for an example (ideally from your team's past) as well.