I just did a live Ask-Me-Anything (AMA) but we can also collect questions here that I'll answer afterward.
I recently joined Databricks as a Senior Director of Engineering. Most of my career has involved Machine Learning, and I've managed teams ranging from 2 to 200 engineers/research scientists.
Previously, I was the Head of FB Reels and Video Recommendations (Director of Engineering) at Meta, from 2019 - 2023. Before that, I was a Senior Staff Software Engineer and Tech Lead working at Google in Mountain View. I hold a PhD in Computer Science from Penn State, and my publications during that time have accumulated almost 9K academic citations.
Some questions I've heard:
I've started writing detailed notes in blogs titled TLDR0. Slightly cheesy, but it stands for Tech Leadership - Direct, Respectful, Zero Ego. See the full details on my LinkedIn profile.
How can we develop an executive / leadership presence, when you are trying to grow from Senior to Staff engineer?
How did you learn the art of communicating effectively ?
How did you know the "right" people to connect to for career growth?
In the past year or so, there's been a "flattening" of managers throughout the industry and increasing importance on IC engineers. Do you think this is the new normal for the software industry?
How do you think about long-term career stability on the IC vs manager path?
In the past year or so, there's been a "flattening" of managers throughout the industry and increasing importance on IC engineers. Do you think this is the new normal for the software industry?
How do you think about long-term career stability on the IC vs manager path?
I think the industry is definitely grappling with the relative value of engineering managers versus engineers at the same levels (since they 'cost' roughly the same). Basically, the question company leaders are asking is -- what would happen if managers who don't build stuff themselves either weren't there at all or had to support many more engineers. I think the answer is mixed. It's not super clear, and in the current environment, not being super clear is a problem.
My advice is to aspire to be either be (a) a very technical engineer who codes or contributes technically in other ways, and has a few reports (the bonafide TLM archetype) or (b) a less hands-on manager who is capable of scaling up massively to support many teams and charters, including other TLMs. Those somewhere in between these two are at risk IMHO.
What are your thoughts on balancing engagement with communications and meetings for career growth with maximizing focused time at work, along with work-life balance? We're all ambitious and want to get promoted, but it's also important to work in a sustainable way, and also to make the most of our time. Finally, we all have lives and demands on our time outside of work as well. How can we balance these priorities most effectively?
How did you approach learning throughout your career.
What are the actions and steps you took at your job and outside your job in order to learn and grow as a software engineer?
My learning is always motivated by the work I am doing at a given point. This makes the learning clearly motivated, and also no learning technique works for me better than 'practicing on the job'.
That would be an essay in an of itself. The one thing that has always resonated is focusing on the end user for everything I did as a software engineer. I always prioritize doing work that impacts users versus work that feels technically 'interesting'.
When considering your next job, how do you distinguish b/w companies like Robinhood, Dropbox etc. that were hyped talent magnets but failed to live up to their promise v/s Facebook, Google, and maybe Databricks? Is there a test for this?
Execution velocity is emphasized a lot everywhere. we want to keep up our words but at the same we can only do so much. How do I navigate this kind of expectations at work, what is possible and the hidden risks (unknowns)?
When considering your next job, how do you distinguish b/w companies like Robinhood, Dropbox etc. that were hyped talent magnets but failed to live up to their promise v/s Facebook, Google, and maybe Databricks? Is there a test for this?
I think there are two different aspects to that choice:
#1 ties more to short-medium term learning and development. #2 ties to more to long term wealth creation and long term employment.
Almost nobody can predict with high certainty whether a company would turn out to be Google or Robinhood. I bet that when Robinhood or Dropbox were hyped, the talent pool was so good that it must have been rewarding to be part of those companies. But at the same time, wealth creation story is very different for long term Googlers versus Robinhood (outside of the earliest employees). Likewise, when you join Robinhood during the early days, you are placing a bet that may or may not pay off -- huge upside, huge risk. Google at this point is low risk but also low upside - sublinear growth at best.
Practically, most people would want a mix of 1 and 2, but their mix would be different. If you start with what mix you are optimizing for at a given point, you'll find it easier to decide between Google and Dropbox.
Hello just continuing the thread from the Q and A. About addressing feedback that comes across as complaints rather than being actionable. In my review cycle, I was not meeting expectations and it seemed like when my former teammate told me one thing then they complained to my manager about another thing. With complaints that I don't think I could have acted on cause it seem. Also, I wanted to ask for feedback on something how would you approach an assignment you feel your manager gave you as a sink-or-swim assignment? How would you move forward with the info and the situation?
I don't fully understand the first part of the question, but let me try the second part.
Sink or swim situations aren't very common, and when your manager is putting you in that situation, unfortunately they have already started to imagine a team without you. So the odds are against you that you may not survive the process. In this situation I would really do everything to reduce risk of being unemployed by (a) doing my best to survive the project, and (b) starting to explore other opportunities inside or outside the company. I think b is necessary because even after your best effort, your manager may still make a judgment call that 'it's not working out', so it's best to immediately start exploring backup plans.
One meta point is that you should try to understand the reasons behind 'not meeting expectations' on your current job -- are there skillsets you are lacking? Or do you find the area boring? Or is the environment toxic? Any other factors? As you explore option b, consider the things that may not have worked out in the current gig, and use that to inform your next play.
- What has been the biggest weakness in quality control over hiring in both companies?
- What are the similarities, differences, and focuses in infrastructure's practices between FB and G? At scale, it seems like what they are trying to do is fairly similar, especially with the rewrite of Buck and the open sourcing of Bazel.\
- How do you decide allocation of headcount and resources when it comes to product development vs developer support vs open source community support?
I'm curious to hear about the early stages of your career. How did you decide on what company and what niche of software engineering that you intended to pursue? Did you work at Google straight out of college?
In the ML/AI space - who are you paying attention to? What are some exciting technologies coming out that could help Software Engineers become better Software Engineers (or even help them have a stronger career / impact)? ChatGPT for example seems like a pretty good game changer, but is there anything else?
What are some of the most common people problems/conflicts you encounter as a technology leader and what do you do to avoid or minimize them for your organization?
The biggest ones by far are overlapping scope, insufficient recognition for work done, and related to both but slightly different, a lack of promotion or path to promotion.
One of the secrets of great career growth is being amongst good / great peers or co-workers. Such folks are usually at top-tier firms that have a high hiring bar. How could folks who are working at smaller firms or not unicorn companies build good relationships & grow in their career (apart from jumping ship) ?
This is a good question; it falls under a group of 'chicken and egg' questions such as -- how do you get a job as a new grad if all the jobs require experience. I'd say there are two avenues to explore:
What are your thoughts on balancing engagement with communications and meetings for career growth with maximizing focused time at work, along with work-life balance? We're all ambitious and want to get promoted, but it's also important to work in a sustainable way, and also to make the most of our time. Finally, we all have lives and demands on our time outside of work as well. How can we balance these priorities most effectively?
I think I answered this during the live session, but I agree this is a good question and a good pursuit. One thing I'll say is that the balance is going to be dynamic over months and years as your personal situation evolves and the intensity surrounding your job varies over time. Career growth can be achieved through short-medium bursts of intense work followed by breaks during which you can spend more time with family / outside of work as needed. It's usually unhealthy to sustain high intensity workload for too long, so it just makes sense to balance it out, and set the right expectations with your leadership team accordingly. Very practically, the run up to a promotion is usually the period during which your productivity gets most scrutinized, thanks to recency bias everywhere.
As you've recently joined Databricks, What are your tips for someone joining at the senior / staff IC level to do in their first 60 / 90 days to make an impact ?
Also answered during Live Q&A, but instead of being overly specific to Databricks, I'd say the biggest thing that'll pay off is to build good relationships with lots of people, great relationships with a small subset of them, and be very very perceptive about the company culture and progressively mould yourself to that culture.
I'm curious to hear about the early stages of your career. How did you decide on what company and what niche of software engineering that you intended to pursue? Did you work at Google straight out of college?
I graduated with a PhD and 0 years of experience during a period of recession 2009-2010. I didn't really have a choice -- Google was the only offer I had, as far as I can remember.
Regarding niche of software engineering -- I was fascinated by probability, statistics, and its impact on the world. I was also good at coding and understood theoretical computer science well. So the intersection ended up being Applied Machine Learning.
Execution velocity is emphasized a lot everywhere. we want to keep up our words but at the same we can only do so much. How do I navigate this kind of expectations at work, what is possible and the hidden risks (unknowns)?
If I understand the question correctly, it's about live up to the expectations of high velocity of execution amidst unknowns that can slow you down.
In my field of applied AI, the unknowns are pervasive -- every idea is at best an experiment, you implement it, you try it out, and it fails. So in the field of AI it's very common to have a ton of hidden risks - everything is inexact.
I think one way to navigate this would be to spend a lot of time with your leadership team explaining to them these risks and unknowns; sometimes the expectations are unreasonable because there's this lack of understanding. Don't assume that they know about these risks that could slow you down. Repeat the message around uncertainty till it gets ingrained. In addition, discuss mitigation strategies. e.g. how can you reduce risk, increase chances of success, or have backup plans that align with business goals in case the unknowns slow you down.
What are some of the most common people problems/conflicts you encounter as a technology leader and what do you do to avoid or minimize them for your organization?
One of the secrets of great career growth is being amongst good / great peers or co-workers. Such folks are usually at top-tier firms that have a high hiring bar. How could folks who are working at smaller firms or not unicorn companies build good relationships & grow in their career (apart from jumping ship) ?
As you've recently joined Databricks, What are your tips for someone joining at the senior / staff IC level to do in their first 60 / 90 days to make an impact ?
- How would you start increasing influence in a new company?
- What things should one focus on growing from Senior Eng to Staff Eng & Senior Eng to Eng manager level?
In the ML/AI space - who are you paying attention to? What are some exciting technologies coming out that could help Software Engineers become better Software Engineers (or even help them have a stronger career / impact)? ChatGPT for example seems like a pretty good game changer, but is there anything else?
Not the best person to be answering this, but in general, coding assistants like copilots are going to become increasingly critical tools in your software development toolkit, including those that interface via natural language.
Hello just continuing the thread from the Q and A. About addressing feedback that comes across as complaints rather than being actionable. In my review cycle, I was not meeting expectations and it seemed like when my former teammate told me one thing then they complained to my manager about another thing. With complaints that I don't think I could have acted on cause it seem. Also, I wanted to ask for feedback on something how would you approach an assignment you feel your manager gave you as a sink-or-swim assignment? How would you move forward with the info and the situation?
How did you approach learning throughout your career.
What are the actions and steps you took at your job and outside your job in order to learn and grow as a software engineer?
Too much emphasis on Leetcode-style coding challenges
I've often felt that if you Leetcode properly and approached the challenge the same way you might as a proper engineer, you would consistently do well on these. Incidentally, I've faced a lot of problems in infra that actually seem similar to some Leetcode style problems.
Since i've always been within product development and that's usually most heavily staffed, that allocation problem is usually solved at a higher level in the organization. But in general, I find developer support to be relatively understaffed.
I've often felt the same way about the industry for a few years now. It seems like because a lot of the platform-level stuff is solid and the standards are well defined, it is very easy to churn out products. I have a pet theory that companies could cut costs by simply paying product developers less since the technical skill needed is not as high as companies think. I'm curious how this will play out over the next few years now that "growth for growth's sake" is no longer the mantra.
As for developer support, yes its very understaffed across the industry. But it's also relatively new from what I can tell (circa 2016, it was formally identified and defined). How many people do you think/speculate would have the confidence and skill to move from product to infra?