Senior Software Engineer, Search Infrastructure Data Engineering - Slack

At Slack, we are dedicated to revolutionizing the way people find and interact with information. Our mission is to build cutting-edge search technologies that deliver relevant and personalized results to our users. We are looking for a talented and passionate Search Infrastructure Data Engineer to join our team and help us achieve this goal.

The Core Infrastructure organization at Slack is responsible for designing, developing, and maintaining the information retrieval infrastructure that supports our Search, ML, and many other product experiences. This stack is at the heart of what makes Slack such a compelling store of company knowledge, allowing our customers to find and discover messages, channels, people, files, and other business units within their workspaces. We are also heavily invested in generative AI, helping build products that leverage the full wealth of knowledge in our company.

As a Search Infrastructure Data Engineer, you will work across the Search Infra and ML Infra teams to support their data engineering needs. You will be responsible for designing, building, and maintaining the data infrastructure and pipelines that power our search and recommendation systems. You will work closely with data scientists, machine learning/ai engineers, and software developers to ensure that our search algorithms are efficient, scalable, and deliver high-quality results.

Key Responsibilities:

• Design and develop scalable and resilient information retrieval infrastructure to power search and other products.
• Build and integrate scalable backend systems, platforms, and tools that power our data warehouse and help our partners implement, deploy, and analyze data assets.
• Develop and maintain ETL processes to ensure data quality and consistency.
• Collaborate with data scientists and machine learning engineers to deploy machine learning models for semantic retrieval in our own kubernetes-based deployment system, working with tools like Chef and Hashicorp Terraform.
• Optimize data storage and retrieval to support real-time search queries and recommendations.
• Monitor and troubleshoot data pipelines in production.
• Work with the Search and ML Infrastructure teams to maintain and improve various data pipelines.
• Mentor other engineers and deeply review code.
• Improve engineering standards, tooling, and processes.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
• 5+ years of relevant technical experience, including significant experience in data engineering, with a focus on search.
• Experience with search technologies such as Elasticsearch, Solr, or Lucene.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with big data technologies such as Airflow, EMR, Hadoop, Hive, Spark, and Kafka.
• Strong knowledge of SQL and NoSQL databases.
• Experience with cloud platforms (e.g., AWS, GCP, Azure) and containerization (e.g., Docker, Kubernetes).
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills.

Preferred Qualifications:

• Knowledge of natural language processing (NLP) techniques and tools.
• Experience with A/B testing and experimentation frameworks.
• Familiarity with data visualization tools and techniques.
• Experience with vector-based retrieval systems like Vespa, Milvus, or Solr.
• Experience with ML model serving frameworks/toolkits like Kubeflow, MLflow, Sagemaker, and AWS Bedrock.

What We Offer:

• Competitive salary and benefits package.
• Opportunity to work on cutting-edge search technologies.
• Collaborative and inclusive work environment.
• Professional development and growth opportunities.

Slack has a positive, diverse, and supportive culture—we look for people who are curious, inventive, and work to be a little better every single day. In our work together we seek to be smart, humble, hardworking and, above all, collaborative.