Data Engineer II, Data Pipelines

World's most advanced and largest intelligence company with 1,000 intelligence professionals, over $300M in sales, serving 1,800+ clients worldwide.
Data
Mid-Level Software Engineer
Contact Company
501 - 1,000 Employees
3+ years of experience
AI · Cybersecurity

Description For Data Engineer II, Data Pipelines

Recorded Future, the world's most advanced and largest intelligence company, is seeking a Data Engineer II for their Data Pipelines team. With 1,000 intelligence professionals, over $300M in sales, and serving over 1,800 clients worldwide, Recorded Future is at the forefront of cyber intelligence.

As a Data Engineer II, you'll be part of a motivated team building reliable data pipelines that ingest raw data from various public web sources and structured data feeds. Your role will be crucial in converting this data into actionable analytics to identify and combat cyber security threats.

Key responsibilities include:

  • Leading development, productionalization, and maintenance of high-volume data extraction systems
  • Creating well-monitored production data pipelines
  • Maintaining and upgrading existing data flow production systems
  • Collaborating with junior and senior engineers through various software engineering best practices

The ideal candidate should bring:

  • 3+ years of Python programming experience, with the ability to write production-grade data processing infrastructure
  • 3+ years of high-volume data management experience, including working with large, complex data structures
  • Strong skills in transforming large datasets, querying modern datastores like MongoDB and ElasticSearch, and developing corresponding APIs
  • A well-rounded skill set in both computer programming and data engineering

Recorded Future offers a diverse and inclusive work environment, with employees (known as "Futurists") representing over 40 nationalities. The company's core values include high standards, inclusion, and ethical practices. Their dedication to empowering clients with intelligence to disrupt adversaries has earned them a 4.8-star user rating from Gartner and more than 45 Fortune 100 companies as clients.

Join Recorded Future to be part of a company that's making a significant impact in the world of cyber threat intelligence. With opportunities for growth, a commitment to diversity, and a focus on cutting-edge technology, this role offers an exciting chance to contribute to a rapidly evolving field while working with a team of passionate professionals.

Last updated 2 months ago

Responsibilities For Data Engineer II, Data Pipelines

  • Lead development, productionalization and maintenance of high volume data extraction
  • Create well monitored production data pipelines
  • Maintain and upgrade existing data flow production systems
  • Collaborate with junior and senior engineers
  • Participate in design reviews, code reviews, and pair programming

Requirements For Data Engineer II, Data Pipelines

Python
MongoDB
  • 3+ years of python programming
  • 3+ years of high volume data management
  • Experience with large, complex data structures
  • Experience transforming large datasets
  • Experience with MongoDB and ElasticSearch
  • API development experience
  • Strong debugging skills

Benefits For Data Engineer II, Data Pipelines

  • Diverse and inclusive work environment
  • Opportunity to work on cutting-edge technology
  • Potential for growth and development

Interested in this job?

Jobs Related To Recorded Future Data Engineer II, Data Pipelines

Data Engineer II, Attack Surface Module

Data Engineer II position at Recorded Future, focusing on building and maintaining data pipelines for security teams, requiring 3+ years of Python experience.

Data Engineer

Data Engineer role at Recorded Future, building data pipelines for cyber threat intelligence. 1+ years Python experience required.

Business Intelligence Engineer, ORC (ORC- Operations Risk Compliance) Program Analytics

Business Intelligence Engineer role at Amazon focusing on ORC Analytics, combining statistical analysis, data engineering, and business intelligence expertise in London.

Data Engineer

Data Engineer position at WorldQuant focusing on developing data pipelines and engineering solutions for financial strategies.

Data Engineer

Data Engineer position at G-P, developing solutions for their Global Employment Platform, working with Python, SQL, and modern data technologies in a remote environment.