Data Engineer

Prophecy is the data copilot company, enabling Fortune 500 enterprises to accelerate AI and analytics by delivering clean, trusted, and timely data.
Data
Remote
AI · Finance · Enterprise SaaS...

Description For Data Engineer

Prophecy is the data copilot company, revolutionizing how enterprise teams handle their data transformation. We're seeking a solid data engineer to join our high-growth startup, working with Fortune 500 enterprises in banking, insurance, healthcare, life sciences, and technology.

As a Data Engineer at Prophecy, you'll be a resident expert in data engineering using Prophecy with Spark & Airflow. You'll solve complex customer issues related to Apache Spark and Data Warehouses, build automated tooling, and demonstrate ownership to drive solutions. You'll also educate customers on best practices for pipeline runtime performance, CI/CD, and scheduling.

Your role involves building reusable Spark, SQL, and Airflow packages that solve repeated problems in data engineering. You'll write high-performance Spark code and have the opportunity to sculpt the next wave of data tools.

The ideal candidate has customer-facing experience, preferably in solution engineering, with the ability to form relationships with executives and present convincing points-of-view. Technical experience with Apache Spark, Python or Scala, and complex data engineering (ETL) is crucial. A Bachelor's in CS is preferred.

Prophecy offers competitive benefits, including comprehensive health insurance, flexible PTO, wellness allowance, and professional development reimbursement. Join us in revolutionizing data engineering and make your mark on an innovative platform with end-to-end project ownership.

We're committed to merit and inclusion, fostering a diverse culture where people from all backgrounds can excel. Prophecy is an equal opportunity employer, considering candidates without regard to protected characteristics.

Last updated 2 months ago

Responsibilities For Data Engineer

  • Be a resident expert in data engineering using Prophecy with Spark & Airflow
  • Address complex customer issues related to Apache Spark and Data Warehouses
  • Build automated tooling to solve repeatable problems
  • Demonstrate ownership to drive to solution, be pro-active
  • Participate in on-call rotation
  • Help customers onboard to the product, answering queries about Prophecy, Spark, and Airflow
  • Provide best practices and guidance around pipeline runtime performance, usage of third-party libraries, CI/CD, and scheduling
  • Build reusable Spark, SQL, and Airflow packages that solve repeated problems in data engineering
  • Write high-performance Spark code

Requirements For Data Engineer

Python
Scala
  • Experience in Apache Spark, and ability to write Python or Scala code
  • Experience in complex data engineering (ETL) and solving performance issues with Spark jobs
  • Experience in a technical customer-facing role, preferably solution engineering
  • Ability to form relationships with executives and influencers
  • Ability to present a convincing point-of-view to important decision-makers
  • Bachelor in CS (preferred)

Benefits For Data Engineer

Medical Insurance
Dental Insurance
Vision Insurance
  • 99% coverage of employee health insurance and 75% for dependents
  • Very competitive compensation
  • $200 per month towards wellness, gyms, massages, facials, and more
  • Birthday and anniversary day off
  • Flexible PTO
  • $500 professional development reimbursement every year
  • Company sponsored Long Term Disability and Life Insurance
  • FSA/HSA
  • Ability to have your fingerprint on an innovative platform
  • End-to-end ownership of your projects

Interested in this job?