Measurement of impact for MLE?

Question

I joined my team in June this year right after bootcamp. When I joined this team, we set the goals for the half, and then got reorg-ed to a different domain (think ML for ads vs ML for recommendations).

Our models had only shown limited success in the previous domain before the reorg, we spent around 2 months (July and August) just building new versions of these models for the new domain.

It's October already, the model hasn't shown any significant success in any of our projects with XFN. We are getting closer and closer as we understand the problems better. However with code freeze in November, December - it is unlikely it will reach production or even online experiments by then.

Does that mean I would have "no impact" at my first PSC? This would be the case for all of my teammates which seems bonkers.

I thought about writing a long note with all of the progress we've made in understanding the problem (which will result in a model that's cheaper than the current one and easier to understand), what are some results we have seen already, and hypothesis on where to go next.

Still to be honest I'm scared the results I got won't be good enough to get to production by PSC-time, and thus I'll be marked as no impact. In retrospective I should have studied the problem more when I joined but I was so new to Meta.

How can I mitigate this? Looking for a side-project now I can fully own (as E5, I don't think attaching myself to a teammate's project is good enough) is unlikely to get any results with the current model we have.

Alex Chiou · Accepted Answer

First, talk to your manager. This is pretty different from your scenario, but I recommend this Q&A about staying in sync with your manager about your performance as a lot of the concepts still apply.

I'm not an MLE, but back at Instagram, I ran a lot of experiments (many of which failed) and I also took on large projects that would span >1 performance cycle (this was back when PSC was every half), so I have some thoughts here.

Does that mean I would have "no impact" at my first PSC?

Short answer: Almost certainly not.

Long answer: So I don't know if you know this, but "delayed impact" is a thing at Meta. At E5, and especially at E6, you're going to take on these more "long-pole" projects that, well, take a long time. It wouldn't be fair to punish these engineers for not realizing impact within the cycle (PSC is every year now, but you have only been at the company for 1 half so if you do get rated, you are essentially doing "old-school" PSC). In these scenarios, these engineers would get "delayed impact", which is evaluated through project milestones and quite a lot of subjectivity.

One of the ways to get delayed impact is to generate learnings - At Instagram, we got impact credit for failed experiments as long as we were able to extract clear learnings from it. It's delayed impact as the learnings will help us find a successful experiment in the future.

I thought about writing a long note with all of the progress we've made in understanding the problem...

You should 100% do this. In order to get delayed impact credit, you need to eloquently show that the work you've already done wasn't a complete waste of time and is actually leading up to something. Ideally, you can concretely and reasonably connect the current results of your model to future action items and attributes of the future, ideal model. Conversely, you can share in your learnings which categories of hypotheses you have learned to not be viable anymore (saving time). I would literally link Workplace notes on my PSC self-review - They're that important and are a huge part of your rating if done well.

How can I mitigate this?

Come up with additional scope! Here's a bunch of my ideas on how to do this. For Meta in particular, you should look into BE projects - Due to the code freeze, November and December were pretty much Better Engineering season back for me at Instagram. How much it's valued depends on org, but I feel like the company as a whole has valued it more as stability has become more important. If there aren't E5-scope BE projects in your team/org, I would be very surprised.

(as E5, I don't think attaching myself to a teammate's project is good enough)

Not with that attitude! Expanding the scope of a teammate's project is so powerful as it not only gives you work, but it helps build that relationship as well. I have some thoughts (and examples) on how to do that here.

Measurement of impact for MLE?

Discussion