Senior Software Engineer

If you are looking for the opportunity to build highly scalable services and rich data insights on Azure's largest distributed platforms, then the Security Platform team is the right place for you to come and take on your next challenge. Our team owns multiple data pipelines that process tens of terabytes of data per day and stores multiple petabytes of data. Our challenge is to do this efficiently, reliably, and with low latency.

The Security Platform LT team needs to understand how the Security platform is used, to make data-driven decisions and build product features based on customer usage and feedback. We also ensure the data is compliant with data privacy regulations and other privacy regulations. Our job is to create data pipelines to produce meaningful insights about their level of compliance, and to create understandable actions that they can take to improve.

We are looking for engineers who have a track record of success, who deliver results, and who (preferably) have a background in large-scale data processing systems in a distributed service environment. Everyone on the team owns significant areas of the system, directly contributes to the vision, direction, and implementation of new features, and are key to the success of our products. You will collaborate on your designs across our partner engineering teams and work closely with our PMs to design, build, and maintain a growing set of features that will serve the needs of our customers.

Key Responsibilities:

• Requirements and Modeling: Collaborate with stakeholders, document data requirements, evaluate feasibility, negotiate data agreements, and propose data metrics for project-relevant features.
• Data Model Design and Specification: Contribute to data model design, draft design specification documents, improve data schemas, and balance analytical requirements and compute/storage consumption for data.
• Data Extraction and Validation: Implement code to extract raw data from upstream sources, perform code reviews, implement orchestration techniques, validate data quality and reliability, and record data source control and versioning.
• Data Transformation and Merging: Apply standard operations to transform raw data into compatible forms, use software and tools to wrangle and analyze data, and merge data into distributed systems or products.
• Data Tool Design and Maintenance: Design and maintain data tools/platforms used to transform, manage, and access data, write efficient code to test and validate data platforms, and implement sustainable design patterns.
• Data Monitoring and Troubleshooting: Follow performance monitoring protocols, build visualizations and aggregations to monitor data quality and pipeline/report health, troubleshoot and fix problems or anomalies, and document and share insights.
• Data Compliance and Governance: Follow data modeling and handling procedures to comply with laws and policies, tag and classify data, govern data accessibility and traceability, and contribute to the data glossary.

Join Microsoft's Security Platform team and contribute to building highly scalable services and rich data insights on Azure's largest distributed platforms.