I'm a Data Engineer. Within the data engineering realm, there are a lot of tools, just like in the software engineering realm. The modern data stack is pretty popular these days. It includes things like Spark for ETL at scale, Docker for virtualized environments, Airflow for orchestration, dbt (data build tool) for transformations in SQL, Fivetran for automated data connectors, Snowflake for data warehousing, and more.
I'm far from knowing all of these tools well, primarily because I use very few of them in my day job. The main reason I want to change jobs is because of this.
I'm worried I'm caught in a catch-22 situation where I don't know the tools so I can't get jobs that have them, which I guess is similar to the new-grad cold start problem.
My question is, how should I think about learning new tools for job interviews? My current instinct is to learn via failure. That is, I have almost all of the above tools on my resume. If someone asks me about them and I'm not able to give a good answer, I will learn that part about the tool so if I'm in the same situation I can answer properly.
Another approach I can think of is to do Udemy courses of them so I have a deeper understanding of how they work. I've learned to be wary of course not tied to projects, though, so I'm hesitant.
I guess I could do projects to learn more about them, but those take time and right now I'm focused on applying to jobs.
I imagine some answers might focus on what my current problem is: can I get interviews or am I failing interviews? I don't think my issue is with failing interviews right now, and certainly not because of specific knowledge people have called me out for for not knowing these tools. I think my issue is more with sourcing interviews currently.
If there's general advice regarding how to think about prepping for an interview when you only have some of the requirements on the Job Description, would love to hear that too.
Thanks!