Data Engineer II, Data Pipelines

World's most advanced and largest intelligence company with 1,000 intelligence professionals, over $300M in sales, serving 1,800+ clients worldwide.
Data
Mid-Level Software Engineer
Contact Company
501 - 1,000 Employees
3+ years of experience
AI · Cybersecurity

Description For Data Engineer II, Data Pipelines

Recorded Future, the world's most advanced and largest intelligence company, is seeking a Data Engineer II for their Data Pipelines team. With 1,000 intelligence professionals, over $300M in sales, and serving over 1,800 clients worldwide, Recorded Future is at the forefront of cyber intelligence.

As a Data Engineer II, you'll be part of a motivated team building reliable data pipelines that ingest raw data from various public web sources and structured data feeds. Your role will be crucial in converting this data into actionable analytics to identify and combat cyber security threats.

Key responsibilities include:

  • Leading development, productionalization, and maintenance of high-volume data extraction systems
  • Creating well-monitored production data pipelines
  • Maintaining and upgrading existing data flow production systems
  • Collaborating with junior and senior engineers through various software engineering best practices

The ideal candidate should bring:

  • 3+ years of Python programming experience, with the ability to write production-grade data processing infrastructure
  • 3+ years of high-volume data management experience, including working with large, complex data structures
  • Strong skills in transforming large datasets, querying modern datastores like MongoDB and ElasticSearch, and developing corresponding APIs
  • A well-rounded skill set in both computer programming and data engineering

Recorded Future offers a diverse and inclusive work environment, with employees (known as "Futurists") representing over 40 nationalities. The company's core values include high standards, inclusion, and ethical practices. Their dedication to empowering clients with intelligence to disrupt adversaries has earned them a 4.8-star user rating from Gartner and more than 45 Fortune 100 companies as clients.

Join Recorded Future to be part of a company that's making a significant impact in the world of cyber threat intelligence. With opportunities for growth, a commitment to diversity, and a focus on cutting-edge technology, this role offers an exciting chance to contribute to a rapidly evolving field while working with a team of passionate professionals.

Last updated 20 days ago

Responsibilities For Data Engineer II, Data Pipelines

  • Lead development, productionalization and maintenance of high volume data extraction
  • Create well monitored production data pipelines
  • Maintain and upgrade existing data flow production systems
  • Collaborate with junior and senior engineers
  • Participate in design reviews, code reviews, and pair programming

Requirements For Data Engineer II, Data Pipelines

Python
MongoDB
  • 3+ years of python programming
  • 3+ years of high volume data management
  • Experience with large, complex data structures
  • Experience transforming large datasets
  • Experience with MongoDB and ElasticSearch
  • API development experience
  • Strong debugging skills

Benefits For Data Engineer II, Data Pipelines

  • Diverse and inclusive work environment
  • Opportunity to work on cutting-edge technology
  • Potential for growth and development

Interested in this job?

Jobs Related To Recorded Future Data Engineer II, Data Pipelines

Cloud Support Engineer - Analytics

AWS Cloud Support Engineer position focusing on analytics technologies, providing technical solutions and support for AWS services including Kinesis, OpenSearch, and Redshift.

Business Intelligence Engineer, AWS Strategy and Operations

Business Intelligence Engineer role at AWS focusing on data analysis, visualization, and strategic insights for global services operations.

Business Intelligence Engineer

Business Intelligence Engineer role at Amazon focusing on data solutions for External Fulfillment operations across Europe

Business Intelligence Engineer, GS S&O - T&C

Business Intelligence Engineer role at AWS T&C, focusing on data analytics and visualization, offering competitive salary and benefits.

Business Intelligence Engineer, Project Kuiper Production Operations

Business Intelligence Engineer role at Project Kuiper, developing data analytics solutions for satellite technology production operations.