Overview

Core AI is at the forefront of Microsoft’s mission to redefine how software is built and experienced. We build core tools that have defined software engineering for years—VS Code, GitHub, Visual Studio, AI Foundry—and we are infusing AI deeply into all parts of this portfolio.

Within Core AI, the Cloud Mine team serves as the data backbone of Microsoft's engineering intelligence. Our mission is to make Microsoft’s engineering teams the most productive, secure, and agile in the industry by building the authoritative engineering data platform that powers decisions at scale. We mine, validate, and serve trusted datasets derived from engineering artifacts—pull requests, work items, build pipelines, security signals, AI telemetry, and more—used daily in shiproom operations, security and compliance enforcement, organization-wide productivity programs, and AI-powered insights across Windows, Azure, Microsoft 365, Xbox, and other major product lines. Our data directly shapes how Microsoft ships software.

Responsibilities:

As a Senior Software Engineer on the Cloud Mine team, you will lead the design and delivery of critical data mining and data pipeline infrastructure. This is a data-centric role: you will build and operate large-scale data ingestion, transformation, and validation systems—not web UIs or front-end experiences. Your day-to-day responsibilities center on ensuring that massive volumes of engineering data are collected, processed, validated, and made available with high correctness and freshness.

You will architect and implement data pipelines in C# running on Azure, build automated data quality and validation frameworks, design schema evolution and lineage tracking, and integrate new engineering signal sources into the platform. You will set technical direction for your area, mentor other engineers, and drive cross-team alignment on data contracts and quality standards.

Success in this role requires deep expertise in data engineering and a passion for data correctness. You think in terms of data flows, schemas, validation rules, and pipeline reliability. You thrive in backend and data systems, care deeply about data quality at scale, and bring the judgment and autonomy expected of a senior engineer.

At Microsoft, our mission is to empower every person and every organization on the planet to achieve more. As employees, we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Qualifications:

Required Qualifications:

• Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.

Other Requirements:

• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years

Preferred Qualifications:

• Demonstrated experience building and operating production data pipelines, ETL/ELT systems, or data processing frameworks at scale.

• Strong proficiency in C# or a comparable strongly-typed backend language, with the ability to write performant, maintainable data-processing code.

• Hands-on experience with data validation, data quality practices, or data governance (e.g., schema enforcement, anomaly detection, correctness monitoring).

• Experience with Azure data and cloud services (e.g., Azure Data Lake, Azure Data Explorer/Kusto, Cosmos DB, Event Hubs, Azure Storage, or similar).

• Track record of operating production systems: monitoring, incident response, performance analysis, and driving reliability improvements.

• Experience with Microsoft-internal big data systems (e.g., Cosmos/SCOPE, Azure Data Explorer/Kusto) or equivalent large-scale analytics platforms.

• Deep familiarity with data pipeline patterns: idempotency, backfill, schema evolution, data lineage, and late-arriving data handling.

• Experience building data quality frameworks with automated validation, data profiling, and regression detection.

• Familiarity with engineering systems signals (source control, CI/CD, work tracking, security scanning) and deriving metrics and insights from them.

• Experience setting technical direction, mentoring engineers, and driving cross-team alignment on data standards and contracts.

• Background in data mining, data analysis, or working with large heterogeneous datasets to extract structured, actionable information.

Software Engineering IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $158,400 - $258,000 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.