Profile picture

Side Projects Q&A and Videos

About Side Projects

Transitioning into the compiler engineering field (or any other domain) if you are unemployed and don't have prior experience in the field

Senior Software Engineer at Taro Community profile pic
Senior Software Engineer at Taro Community

I am currently seeking to transition into a career as a compiler engineer, a field I find deeply fascinating. The interdisciplinary nature of compiler engineering, bridging areas such as computer architecture and graph theory, intrigues me greatly. Additionally, the sector offers promising financial rewards, especially with companies like Meta, Nvidia, and AMD that are at the forefront of hardware accelerators experiencing significant growth. I am convinced this growth trajectory will continue, making this career path an ideal blend of intellectual fulfillment, professional growth, and competitive compensation.

Due to recent layoffs, I find myself unemployed, and I am seizing this moment to pivot towards compiler engineering. However, I acknowledge that there is a steep learning curve to becoming an ideal candidate for such positions. The required skill set typically includes:

  • Proficiency in C++
  • Experience with GPUs
  • Knowledge of an Intermediate Representation Language (e.g., LLVM)
  • Understanding of computer architecture

Previously, I worked as a senior backend engineer, specializing in tool development using functional programming languages such as Scala and Ocaml. My experience spans across FAANG companies and two startups.

To bridge the gap in my skill set, I have been actively contributing to open-source projects similar to LLVM and honing my C++ skills through consistent practice on Leetcode. Despite securing a few interviews for compiler engineering positions, I have not been successful, primarily due to difficulties with compiler-specific questions.

I seek advice on the following:

  1. How can I enhance my chances of entering the compiler engineering field, especially without being part of a compiler project community or holding a position of authority within such a project?
  2. What strategies can I employ to prepare for and succeed in domain-specific interviews, considering my lack of prior experience in this area?

Any guidance or insights from those who have navigated a similar path would be immensely appreciated.

Show more
Posted 10 months ago
132 Views
2 Comments

Looking for advice on fine-tuning LLMs as a side project

Entry-Level Data Scientist at Flatiron Health profile pic
Entry-Level Data Scientist at Flatiron Health

I'm a Data Scientist looking to switch company and move to a role closer to ML/LLMs. My plan is to build a side project fine-tuning LLMs to familiarize myself with this field and leverage that experience on my resume. I was wondering if anyone here has experience building similar projects or went through a similar learning process - it would be very helpful to get some insights on skill acquisition and finding a job in this area. Here're some examples of what advice I'm looking for, but please feel free to share other aspects as well - anything will be greatly appreciated:

  1. What are some good resources to learn about building LLMs? (currently mostly learning from HF, reddit, and googling)
  2. What's the best tech stack to build personal fine-tuned LLM projects? (I'm planning to use Runpod or similar services like Vast for training and inference, but was wondering if there's other better options)
  3. I'm looking to get into an early stage company in this field. What kind of project should I build to maximize my chance at getting into such companies? My plan rn is to fine tune a model using literature works (novels, poems, proses, etc.) since training data is relatively abundant and it's aligned with my interests. Are there more impactful use cases (for job hunting) out there?
  4. What are some things I should keep in mind when producing deliverables to better showcase my technical and learning abilities? I'm planning to make a series of blog/social media posts documenting my experience building this project. Is there anything in specific that would draw companies' attention?

Thanks in advance and please feel free to share your thoughts!

Show more
Posted 10 months ago
130 Views
3 Comments

What are the things I should look out for when designing a file storage system?

Anonymous User at Taro Community profile pic
Anonymous User at Taro Community

I am working on a side project. I may use GCP or AWS for it.

tldr of the system : Create a web ui to process input files and generate a tabular output giving the link to the dashboard .

I have a bunch of files generated , about 2GB in size in total , Account ID is the high level directory and inside each account there 50 log files each having specific content .

I want to create a UI using Angular framework . Using the UI I will upload the files to GCS /AWS S3 . Then the backend should trigger complex backend scripts, written will be written in Python or Java and then output file is generated. Using the output file a dashboard with few images embedded is generated.

In the end the UI should have a table which should indicate status of processing to COMPLETE and in another column show point to url of the dashboard.

I want to deal with least amount of security work as I want to avoid studying Oauth 2.0 authentication .

Few additional questions , non-system design questions =>

  1. What are the things I should learn in Angular to cover most bases in this project, I am a backend engineer and going over youtube videos . I want to stick with Angular as I have few work projects going to come in coming months so it will help me to learn it .
  2. I am unfamiliar with typescript . How much effort will it need to learn in order to code for this project. I am most familiar with Python only.
Show more
Posted 2 years ago
107 Views
2 Comments