How to handle Junior Engineers who don't listen to you?

Your credibility is at much greater risk trying to manage them when you aren’t ready and aren’t supported than it would be to come out and be honest that while you are ready to technically grow to an L5-level IC responsibilities, you aren’t ready or interested in managing people.

You need to dramatically shift your perspective. They cannot self-direct work, you are too hands-off, and have no clear goals or check in structures. I am sure your reports have no idea you think they are under-performing. 1:1s should be weekly at least, more frequent if they are just ramping up.

If you refuse to do what is right for them and yourself, and let your manager know you cannot manage others now, then you need to take a lot more ownership, this is your move, you tell them: The expectation is you complete these tasks by this time. If there are things blocking your progress bring this to me immediately and we will work through it and adjust the expectations of completion. If you are not doing this, and are instead working on other things, this would constitute underperformance and we will have to document this. I have not done a good job managing expectations so far, or checking in and supporting you. Anything until now is my fault, and I am committed to providing better support and expectations from now on. What happened before won’t be held against you, but we need to work together so we can make up for the set backs I caused you.

You don’t have the luxury of doing deep research and learning what you want to learn until you fix the issues with your mismanagement of your reports. If you can’t manage them well, and take on your tasks, and take time to learn, you have to prioritize them before yourself, and it is further evidence that this is not the time in your career to be a “team lead manager” or whatever this hybrid role is.

Also, “lead” and “manage” are different. You are not ready to manage. Leadership would be admitting what you’re good at (and bad at) and focusing on how you can best contribute. That is as an IC. Learning to lead as an IC is a different skill, and you may be better at that. Certainly there’s less at stake.

Can you put more light on to "lead" vs "manage".

The levels in this org is slight different SDEs can level up from 1 to 4 and post that they become an Engineering Manager. I've read this that, EM is a position slightly different than a technical lead, but because of this hierarchy the path to leadership is where you've to manage projects and people from get go.
There are many engineers at my level who are simply limited to their tech stack and contribute just as an IC. I levelled up because I've multidisciplinary skills and the leadership thinks since I can go beyond and deliver a complete feature, I'm the right person to have a team of engineers.

Though why my team is not listening is as I mentioned one is very close to PiP and even after frequent 1:1s they fail to deliver.

The second engineer is good, but is currently underperforming because he is not completing tasks that are assigned. If someone above from my level gives him another task he will overwrite my directive and do that and again this is making us come back to zero.

The company mostly behave as up and out. I think I've to be assertive and be very frequent with 1:1s

Leading is anything you do that sets others in the right direction or makes them better. Anyone can lead. You don’t need a title or direct reports.

Managing people is taking responsibility for their delivery, keeping them from being blocked or distracted, providing them specific support if they are struggling, giving them clear direction on their performance and promotional potential, assessing their performance, and delivering difficult news (poor perf, no promotion, project is getting killed) with grace. Ideally a manager is a leader, certainly if they are good at their job, but that is not always the case.

What you seem not to be hearing is that you need to stop blaming your reports and start assuming blame yourself. If they do things that higher level people ask (why wouldn’t they? Doesn’t that person have more authority?) and you don’t like that, you need to let them know that any requests of their time can’t be discretionary, no matter who is asking. If the person wants to tell you your report must do a new task, they can. Otherwise the default is always no, because it will require dropping an existing priority. You should be making it clear to others that your reports can’t be available for ad hoc tasks, and telling your reports that YOU are the “bad” one. They never need to say no. They say “I’d love to help, but my manager said I can’t break focus on my current tasks. If you want to see if this can preempt existing work, please talk to them”. Or just “My manager asked me not to accept outside work, and it needs to be pulled in through him” or whatever.

Despite all of this… you being L2, and having two L1 reports, and that after L4 engineer (senior-staff level) the only next step being a full time people manager, is sort of a demonstration of where value is placed. If the company thinks the only distinguishing behavior between levels is direct people management, that’s just a problem. If the best engineers have to leave, stagnate, or move to management, that forces brain drain.

If there are no options but for you to manage people, i think you need an internal mentor that has a lot more experience managing to help you. You also need to dramatically change your attitude. “They are not listening” is not a good assessment. They are listening. Then someone else is giving them a more compelling pitch and they accept that. You are not communicating in a way they can hear and understand. That is shared responsibility, but falls more on you as the manager. You need to get through to them. Figure out how to ensure they take away what you intend. Ask for a written summary, something. If they are getting it wrong you can correct it quickly and adapt your communication.

I want to +1 Lee's

lead” and “manage” are different.

Leadership is about "doing the right thing". You can absolutely lead without being a manager. And you can manage without being a leader, though that's usually a recipe for unhappy reports. In fact, plenty of tech leads "lead" multiple engineers on a project, all without direct authority, because they are able to convince other people to follow their direction.

The problem is they don't listen easily get influenced by other engineers and frequently change what I've asked them to do.

I don't have full context here, but this sounds like case of attempting to manage without influence. You reports don't seem to have bought in what you are asking them to do, and they instead agree w/ some other engineers's proposals, so they just went ahead and did that. It doesn't matter how many 1:1s you've had with your reports, if you reports don't trust you and don't agree with what you are suggesting, they probably won't follow what you say. In other words, despite being a manager, you are not actually demonstrating leadership.

Lee I’m unable to focus on my own learning goals, because I feel there is a lot of gaps of what an L5 engineer technically would need before they manage anyone else. I fear conveying it now will decrease my credibility as someone who can manage teams and projects.

You don't sound happy. Your reports certainly don't either. And I'm not sure your management is either. Sure, you might lose some credibility now by owning up to this. But it's likely better than 1 year from now when your leadership realize you aren't yet ready to manage and you didn't realize that on your own and then have to force you into a different position.

Ultimately, you are not happy. Screw everyone else and just do the right thing for you. You want to be an IC? Go be IC. Before you can lead others, you need to listen to your own feelings and lead yourself.

You might be surprised how often that's just what everyone else wanted too :)