Senior Data Engineer
Description For Senior Data Engineer
Intellectsoft, a prominent software development company since 2007, is seeking a Senior Data Engineer to join their team. The role focuses on an AI-powered platform that leverages advanced analytics and machine learning to help businesses make data-driven decisions. The platform serves various industries, including healthcare, and is designed to optimize processes and improve customer experiences.
The ideal candidate will have 4+ years of professional experience with strong expertise in data engineering, particularly with Apache Spark and SQL. They will be responsible for designing and building scalable data pipelines, collaborating with data science teams, and creating standardized data models. The position requires proficiency in Python, PySpark, and experience with ETL pipelines and distributed computing.
Working with Intellectsoft offers the opportunity to contribute to innovative solutions for prestigious clients like Jaguar Motors, Universal Pictures, and Harley-Davidson. The company provides an excellent work-life balance with 35 paid absence days annually, health insurance, and professional development opportunities through Udemy courses.
The role offers the flexibility of remote work with the chance to work on cutting-edge AI and data solutions. You'll be part of a team that delivers impactful solutions across multiple industries and regions. This position is perfect for a seasoned data engineer who wants to work with advanced technologies while maintaining a healthy work-life balance.
Responsibilities For Senior Data Engineer
- Design and build highly reliable and scalable data pipelines using PySpark and big data technologies
- Collaborate with the data science team to develop new features that enhance model accuracy and performance
- Create standardized data models to improve consistency across various deployments
- Troubleshoot and resolve issues in existing ETL pipelines and optimize workflows
- Conduct POCs to evaluate new technologies and integrate additional data sources
- Follow and promote best practices for software development
- Document development updates and maintain clear technical documentation
Requirements For Senior Data Engineer
- 4+ years of professional experience, including 2+ years of data engineering with Apache Spark and SQL
- Proficiency in Python for data processing and automation
- Knowledge of PySpark, distributed computing, analytical databases and other big data technologies
- Expertise in designing and managing ETL pipelines and distributed data processing frameworks
- Strong knowledge of database systems, data modeling, and analytical databases
- Hands-on experience with workflow orchestration tools such as Apache Airflow
- Familiarity with cloud platforms like AWS, GCP, or Azure
- Solid understanding of software development lifecycles
- Bachelor's or master's degree in Computer Science or related field (nice to have)
- Familiarity with data science and machine learning development process (nice to have)
- Understanding of Machine Learning pipelines or frameworks (nice to have)
Benefits For Senior Data Engineer
- 35 paid absence days per year + 1 additional day for each following year
- Up to 15 unused absence days can be added to income after 12 month
- Health insurance
- Depreciation coverage for personal laptop usage
- Udemy courses of choice
- Regular soft-skills trainings
- Excellence Centers meetups
Jobs Related To Intellectsoft Senior Data Engineer
