Big Data Administrator/Engineer
Description For Big Data Administrator/Engineer
Capco Poland, a leading financial services technology consultancy, is seeking a Senior Big Data Administrator/Engineer to join their team in Warsaw. This role combines big data expertise with DevOps practices, requiring 5+ years of experience in Hadoop administration and strong automation skills. The position offers a unique opportunity to work with cutting-edge technologies in the financial services sector, managing and engineering Big Data solutions across multiple open-source platforms including Hadoop, Kafka, HBase, and Spark.
The role demands expertise in performing critical Big Data administration, troubleshooting, and ensuring seamless operation of various data processing frameworks. You'll be responsible for managing Hadoop clusters, implementing monitoring systems, and developing automation scripts to streamline operations. The position offers a blend of technical challenges and opportunities for innovation in a dynamic financial services environment.
Capco provides an inclusive working environment where individuality is celebrated, offering flexible work arrangements including remote options. The company provides comprehensive benefits including medical care, life insurance, and extensive learning opportunities through their Udemy platform. The flat organizational structure ensures direct collaboration with senior partners and clients, fostering professional growth and innovation.
The ideal candidate will possess strong DevOps skills, proficiency in Python programming, and extensive experience with big data technologies. This role offers the perfect opportunity for a seasoned professional looking to make a significant impact in financial services technology while working with a global team of experts.
Responsibilities For Big Data Administrator/Engineer
- Administer and manage Hadoop clusters, ensuring high availability, performance, and scalability
- Maintain and troubleshoot Hadoop ecosystem components such as HDFS, MapReduce, YARN, and related tools
- Ensure Kafka, HBase, and Spark systems are optimized and running smoothly
- Implement monitoring and alerting for Big Data infrastructure
- Develop and automate scripts using tools such as Ansible, Shell scripting, or Python
- Create reusable automation frameworks to reduce manual efforts
- Work on CI/CD pipelines for deployment automation and system integration
- Build and manage automated deployment processes for Big Data clusters and services
- Identify, troubleshoot, and resolve issues related to Big Data platforms
Requirements For Big Data Administrator/Engineer
- 5+ years of hands-on experience in Big Data administration, automation, and DevOps practices
- Proficient in managing and troubleshooting Kafka, HBase, and Spark
- Strong experience with automation tools such as Ansible, Shell scripting, and Python scripting
- Solid understanding of DevOps concepts and practices, including CI/CD
- Proficiency in at least one programming language, with a preference for Python
- Strong debugging skills and experience in resolving complex issues across distributed systems
- Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience)
Benefits For Big Data Administrator/Engineer
- Employment contract and/or Business to Business options
- Remote work possibility
- MyBenefit Cafeteria
- Private medical care
- Life insurance
- Access to 3,000+ Business Courses Platform (Udemy)
- Access to required IT equipment
- Paid Referral Program
- Ongoing learning opportunities
- Flat, non-hierarchical structure
Jobs Related To Capco Poland Big Data Administrator/Engineer
