Sr Data Engineer Databricks and Python
Description For Sr Data Engineer Databricks and Python
Multiplica Talent is seeking a Senior Data Engineer with expertise in Databricks and Python to join their data team. The role focuses on building, optimizing, and maintaining large-scale data processing solutions using cutting-edge tools. The position is fully remote, offering the flexibility to work from any Latin American country. The ideal candidate will work with international teams, requiring strong English communication skills. The role involves designing data pipelines, implementing ETL processes, and collaborating with cross-functional teams. The company offers competitive compensation in USD, along with various benefits including language learning opportunities, healthcare discounts, and access to a vibrant freelancer community. The initial project duration is 2 months with potential extension to 6 months, and there's another project opportunity for a minimum of 12 months.
Responsibilities For Sr Data Engineer Databricks and Python
- Design, develop and maintain efficient and scalable data pipelines in Databricks
- Use Python and associated tools for processing and analyzing large volumes of data
- Integrate, transform and load data (ETL) from various sources to storage platforms (Data Lakes, Data Warehouses)
- Collaborate with data scientists, business analysts and other technical teams
- Optimize data pipeline performance and ensure data quality and consistency
- Participate in cloud data infrastructure design and maintenance using platforms like AWS, Azure or Google Cloud
- Implement data validation tests, perform data quality analysis and ensure database integrity
- Document processes, architectures and data integration procedures
- Stay updated with best practices and new tools related to data engineering
Requirements For Sr Data Engineer Databricks and Python
- Solid experience in Databricks, especially in creating workflows and data pipelines
- Extensive experience in Python for data processing, including libraries like pandas, numpy, pySpark
- Strong knowledge of data architecture and data storage (Data Lakes, Data Warehouses)
- Experience working with Apache Spark or similar technologies
- Familiarity with ETL solutions implementation in production environments
- Knowledge of SQL and NoSQL databases
- Good English level (oral and written)
- Cloud knowledge (AWS, Azure, Google Cloud)
- Experience with version control tools (Git)
- Ability to work autonomously and in team collaborative projects
Benefits For Sr Data Engineer Databricks and Python
- 100% remote work from any Latin American country
- Payments in USD
- 60% discount on language courses (English, French, German, Portuguese, and Italian)
- Special discounts on medical, psychology, nutrition, and physical training plans
- Personalized support from an Account Manager throughout the project
- Access to freelancer community and exclusive projects in 5+ countries including USA
- Challenging project with important technology company
Jobs Related To Multiplica Talent Sr Data Engineer Databricks and Python
