Azure Databricks Platform Engineer

Role: Azure Databricks Platform Engineer

Role Description:

Support the Platform Team in designing, implementing, and scaling Azure and Databricks-based data and platform solutions. The role combines hands-on engineering with platform ownership, automation, team collaboration, and continuous improvement.

Key Responsibilities:

Design, build, and maintain scalable Azure services and Azure Databricks solutions.

Develop and optimize ETL/ELT pipelines using Python, Py Spark, and SQL.

Support GitHub, CI/CD, IAM, security, monitoring, and platform configurations.

Drive platform automation, standardization, reusability, and optimization.

Explore AI, LLMs, and emerging technologies to build and scale solutions and present relevant ideas to the team.

Collaborate with cross-functional teams and guide team members on technical topics.

Required Skills:

5+ years of experience in Data Engineering, Platform Engineering, or a similar role.

Strong hands-on experience with Azure Services and Azure Databricks.

Experience in solution/component design and development, including participation in or leading technical design reviews.

Strong proficiency in Python, Py Spark, and SQL.

Strong understanding of ETL/ELT, data pipelines, data modeling, and data architecture.

Good knowledge of Git, GitHub, CI/CD, and automated deployment practices.

Experience with Python package management and release processes, including building and deploying Wheel packages.

Hands-on experience with Pytest and automated unit, integration, and regression testing.

Experience with IAM, cloud security, Key Vault, Storage, and monitoring.

Understanding of Data Governance, Data Quality, and Databricks Unity Catalog.

Knowledge of AI and LLMs and their application in building and scaling solutions.

Strong problem-solving, communication, collaboration, stakeholder management, and team coordination skills.

Good to Have:

• Experience with Azure Web Apps / App Services.
• Knowledge of Infrastructure as Code (IaC) and Declarative Asset Bundles.
• Exposure to streaming technologies, MLOps, and Power BI/Fabric.