How to transition from back-end development to distributed systems?

Question

I'm currently employed at a startup where we're building bookkeeping tools. My work mainly involves Python, some database work, and a lot of business logic wrangling. Due to the small size of our app and engineering org, I don't have many opportunities to gain the necessary experience in distributed systems. I'm seeking advice on how to transition to distributed systems. Any guidance would be greatly appreciated.

Alex Chiou · Accepted Answer

When it comes to career pivots, here's my advice to everyone: Transfer internally. Here's why:

Every external hire is inherently risky due to the nature of interviews. Most interviews are 4-10 hours long, but you need to evaluate this human to see if they'll perform well at your company for 1.5 to 2 years on average. This is a massive time disparity.
As an employer, I am taking double risk if I'm hiring a new employee who also wants to use this job switch to pivot their stack/role/specialization. Not only will they be onboarding into my company's culture, they'll be onboarding into a new field as well. This is a recipe for disaster, which is why most (good) companies don't allow this. Tech companies are also taking way fewer risks in this crappy market.

Given that your current company literally doesn't have the scope for distributed systems work, things are a bit trickier (i.e. your transition plan will have extra steps). So here's the plan I would follow to transition:

Get a job at a bigger tech company with your current expertise - Ideally it's a Big Tech company, but this is much easier said than done obviously. It seems like you're somewhere in the backend engineer and data engineer realm, so you can apply as those.
Establish yourself in the new company as a high performer - Show your manager and org that you're just a solid engineer who understands the company culture, works well with others, and learns fast. You will need this trust for Step #3, and it will probably take 1 year+ to build.
Switch to distributed systems internally at this company - Talk to your manager about this transition and how you're following your passion (so that you'll have bigger impact for the company long-term).
Do whatever - You can stay at this company or join another company as a distributed systems engineer. You made it!

Distributed systems is unfortunately tough as not every company will have this scope (like yours), so you have the extra steps of #1 and #2. This, combined with the fact that career pivots are hard in general, means that this plan will take a long time. I recommend doing some more soul-searching to figure out if distributed systems is truly what you want to do as all of this is a hard process.

Of course, if you do get an interview for a distributed systems position while lacking the experience, you should take it. But I would temper expectations and expect to be down-leveled from mid-level to junior.

To help with Step #1, I recommend these:

To help with Step #2, I recommend these:

How to transition from back-end development to distributed systems?

Discussion