Sr. Software Engineer, Data Orchestration (Airflow) - Slack

Slack is a business communication platform handling 65PB+ of storage and processing ~900B records daily.
$184,000 - $253,000
Data
Senior Software Engineer
Remote
1,000 - 5,000 Employees
5+ years of experience
Enterprise SaaS

Description For Sr. Software Engineer, Data Orchestration (Airflow) - Slack

Slack is seeking a Senior Software Engineer to join their Data Orchestration Team within the Data Engineering organization. The team is responsible for building and operating the platform that ingests data into their Data Warehouse and maintains low-latency, high-throughput analytics with OLAP DB. The infrastructure currently handles 65PB+ of storage, processes ~900B records daily, manages 400+ ETL pipelines, and 1300+ Active Airflow DAGs.

As a Senior Software Engineer, you'll be instrumental in building scalable services and tools to help partners implement, deploy, and analyze data assets efficiently. The role involves developing and maintaining real-time analytics solutions, optimizing data workflows, and ensuring high data quality and reliability. You'll work with modern OLAP solutions and be responsible for improving the end-to-end workflow for data users at Slack.

The position requires strong technical expertise in data engineering, experience with cloud infrastructure (preferably AWS), and proficiency in languages like Python, Java/Scala. You'll be working with technologies such as Airflow, Docker, Kubernetes, and various big data frameworks. The role also emphasizes leadership aspects, including mentoring other engineers and leading technical initiatives end-to-end.

This is an excellent opportunity for a seasoned engineer who is passionate about data-driven decisions and wants to make a significant impact at Slack. The role offers the flexibility of remote work and competitive compensation, making it an attractive position for those looking to work with big data at scale while contributing to a platform used by millions of users globally.

Last updated 10 days ago

Responsibilities For Sr. Software Engineer, Data Orchestration (Airflow) - Slack

  • Develop and maintain real-time analytics/low latency data access layer built on top of modern OLAP solutions
  • Optimize end-to-end workflow for data users at Slack
  • Improve data quality and reliability of pipelines through monitoring and failure detection
  • Collaborate with cross-functional partners and lead technical initiatives
  • Coach and mentor other engineers across the org
  • Write, review, and provide feedback on technical design proposals

Requirements For Sr. Software Engineer, Data Orchestration (Airflow) - Slack

Python
Java
Scala
Kubernetes
  • 5+ years of experience in data engineering or data infrastructure
  • Experience in real-time analytics with OLAP stores (Apache Pinot or Apache Druid)
  • Experience operating Airflow and workflow management systems
  • Experience working with cloud infrastructure (AWS preferred)
  • Experience building and maintaining large scale ETL pipelines
  • Skilled at crafting robust distributed microservices with Docker, Kubernetes, AWS ECS/EKS
  • Strong dedication to code quality, automation and operational excellence
  • Proficient in Python, Java/Scala, Chef, Terraform
  • Excellent written and verbal communication skills
  • Ability to context switch and handle oncall rotation

Interested in this job?

Jobs Related To Slack Sr. Software Engineer, Data Orchestration (Airflow) - Slack

Sr. Software Engineer, Search Infrastructure & Data - Slack

Senior Software Engineer position at Slack focusing on search infrastructure and data engineering, building scalable search systems and data pipelines.

Senior Data Engineer, AWS Fintech

Senior Data Engineer role at AWS Fintech building large-scale financial data systems and processing billions of transactions daily.

Sr Data Engineer, Partner Experience

Senior Data Engineer position at Amazon Prime Video, building and maintaining data infrastructure for partner experience products using AWS services and big data technologies.

Senior Robotics Business Intelligence Engineer

Senior Robotics Business Intelligence Engineer role at Amazon Robotics, focusing on data analysis and insights for warehouse automation technology.

Senior Engineer (Data Engineer)

Senior Data Engineer position at Velotio Technologies - Build scalable data infrastructure and ETL pipelines using modern big data technologies in a remote work environment.