Senior Data Engineer (Big Data Processing)

CSIT develops products to advance national security interests of Singapore, focusing on Counter-terrorism and Computer Network Defence.
Singapore
Data
Senior Software Engineer
In-Person
5+ years of experience
Cybersecurity

Description For Senior Data Engineer (Big Data Processing)

CSIT, an agency under Singapore's Ministry of Defence, is seeking a Senior Data Engineer to join their mission-critical data analytics systems team. The role focuses on developing and maintaining high-performance data pipelines crucial for national security operations. The position involves working with cutting-edge technologies to process large datasets for threat identification, analysis, and investigation.

The ideal candidate will have extensive experience in data engineering, particularly in building large-scale data pipelines. They should be proficient in Scala or Java and familiar with modern big data technologies like Spark, Kafka, and Hadoop. The role requires both technical expertise and leadership skills, as you'll be mentoring junior engineers and leading technical discussions.

Working at CSIT offers a unique opportunity to contribute directly to Singapore's national security while working with modern technology stacks. The organization promotes an excellent engineering culture with a strong focus on work-life balance. They value innovation and provide an environment where engineers can make meaningful contributions to critical security systems.

Key responsibilities include designing scalable data pipelines, optimizing performance for low-latency and high-throughput, ensuring data integrity, and implementing best practices in data engineering. The role requires collaboration with various stakeholders, including product managers and engineering managers, to deliver impactful data products that meet crucial business needs.

Note: This position is exclusively open to Singapore Citizens due to the sensitive nature of the work and the organization's affiliation with the Ministry of Defence.

Last updated a month ago

Responsibilities For Senior Data Engineer (Big Data Processing)

  • Design and build scalable, robust and maintainable data pipelines to ingest crucial data
  • Work with product managers, engineering managers and stakeholders to deliver impactful data products
  • Optimise performance of large-scale data pipelines for low-latency, high-throughput and reliability
  • Ensure data integrity, quality and governance through robust design and operational monitoring
  • Troubleshoot and resolve complex issues in critical data pipelines
  • Implement best practices in data engineering, including testing, orchestration and monitoring
  • Lead technical discussions with the team and stakeholders
  • Mentor junior data engineers and contribute to building a strong data engineering culture

Requirements For Senior Data Engineer (Big Data Processing)

Java
Scala
Kafka
  • Bachelor's degree in computer science or equivalent
  • 5+ years of experience in data engineering, with focus on building large-scale data pipelines
  • Proven experience in writing optimised ETL code to handle large datasets
  • Proficient in programming languages such as Scala or Java
  • Familiarity with Spark, Kafka, Hadoop, Elastic stack, Kudu
  • Strong understanding of modern data architectures and data modelling paradigms
  • Able to communicate technical details effectively and clearly
  • Good interpersonal, analytical and problem-solving skills

Benefits For Senior Data Engineer (Big Data Processing)

  • Meaningful and purposeful work
  • Work with talented and driven people
  • Modern technologies and tech stacks
  • Excellent engineering culture and work-life balance
  • Engineering and operational excellence
  • Empowerment to innovate

Interested in this job?

Jobs Related To CSIT Senior Data Engineer (Big Data Processing)

Data Engineer II

Senior Data Engineer position at Microsoft focusing on cloud data infrastructure, requiring 8+ years of experience in data engineering and big data technologies.

Analytics Engineer (L5) - Content and Studio

Senior Analytics Engineer role at Netflix focusing on content and studio analytics, requiring 4+ years of experience in data analysis and engineering.

Senior Data Engineer - Data Science

Senior Data Engineer position at LinkedIn focusing on building and managing data systems for the Data Science team, offering competitive pay and hybrid work environment.

Senior Data Engineer

Senior Data Engineer position at Airbnb, building and maintaining data pipelines for marketing technology platform, offering remote work and competitive compensation.

Senior Data Engineer

Senior Data Engineer position at Simply Business, working on data platforms and ML systems with modern tech stack in a hybrid work environment in London.