Data Engineer II, Data Pipelines

World's most advanced and largest intelligence company with 1,000 intelligence professionals, over $300M in sales, serving 1,800+ clients worldwide.
Data
Mid-Level Software Engineer
Contact Company
501 - 1,000 Employees
3+ years of experience
AI · Cybersecurity

Description For Data Engineer II, Data Pipelines

Recorded Future, the world's most advanced and largest intelligence company, is seeking a Data Engineer II for their Data Pipelines team. With 1,000 intelligence professionals, over $300M in sales, and serving over 1,800 clients worldwide, Recorded Future is at the forefront of cyber intelligence.

As a Data Engineer II, you'll be part of a motivated team building reliable data pipelines that ingest raw data from various public web sources and structured data feeds. Your role will be crucial in converting this data into actionable analytics to identify and combat cyber security threats.

Key responsibilities include:

  • Leading development, productionalization, and maintenance of high-volume data extraction systems
  • Creating well-monitored production data pipelines
  • Maintaining and upgrading existing data flow production systems
  • Collaborating with junior and senior engineers through various software engineering best practices

The ideal candidate should bring:

  • 3+ years of Python programming experience, with the ability to write production-grade data processing infrastructure
  • 3+ years of high-volume data management experience, including working with large, complex data structures
  • Strong skills in transforming large datasets, querying modern datastores like MongoDB and ElasticSearch, and developing corresponding APIs
  • A well-rounded skill set in both computer programming and data engineering

Recorded Future offers a diverse and inclusive work environment, with employees (known as "Futurists") representing over 40 nationalities. The company's core values include high standards, inclusion, and ethical practices. Their dedication to empowering clients with intelligence to disrupt adversaries has earned them a 4.8-star user rating from Gartner and more than 45 Fortune 100 companies as clients.

Join Recorded Future to be part of a company that's making a significant impact in the world of cyber threat intelligence. With opportunities for growth, a commitment to diversity, and a focus on cutting-edge technology, this role offers an exciting chance to contribute to a rapidly evolving field while working with a team of passionate professionals.

Last updated 6 months ago

Responsibilities For Data Engineer II, Data Pipelines

  • Lead development, productionalization and maintenance of high volume data extraction
  • Create well monitored production data pipelines
  • Maintain and upgrade existing data flow production systems
  • Collaborate with junior and senior engineers
  • Participate in design reviews, code reviews, and pair programming

Requirements For Data Engineer II, Data Pipelines

Python
MongoDB
  • 3+ years of python programming
  • 3+ years of high volume data management
  • Experience with large, complex data structures
  • Experience transforming large datasets
  • Experience with MongoDB and ElasticSearch
  • API development experience
  • Strong debugging skills

Benefits For Data Engineer II, Data Pipelines

  • Diverse and inclusive work environment
  • Opportunity to work on cutting-edge technology
  • Potential for growth and development

Interested in this job?

Jobs Related To Recorded Future Data Engineer II, Data Pipelines

Data Engineer

Data Engineer role at Recorded Future, building data pipelines for cyber threat intelligence. 1+ years Python experience required.

Sr. Data Engineer I

Senior Data Engineer position at DoubleVerify, working on ETL processes and data pipeline development for digital campaign activation.

DataOps/ML Engineer - Remote, Latin America

DataOps/ML Engineer position at Bluelight, focusing on Python, AWS, and ML technologies, offering remote work and professional development opportunities.

Data Engineer

Data Engineer position at Truecaller, building and maintaining scalable analytics platform, working with big data and ETL pipelines in Stockholm.

Data Engineer

Data Engineer position at XP Inc., working with data pipeline development, ETL processes, and cloud technologies in a leading Brazilian financial services company.