Azure Data Engineer

Job Description

• Data Modelling & Architecture

• Strong understanding of semantic models used in enterprise reporting and analytics.

• Expertise in dimensional modelling, including star and snowflake schema design.

• Practical experience implementing Slowly Changing Dimensions (SCD) Type 1 and Type 2.

• Ability to translate business requirements into scalable data models.

• Advanced Spark SQL

• Proficiency in Spark SQL, including complex joins, aggregations, and transformations.

• Strong command over window functions, ranking functions, and analytical queries.

• Ability to troubleshoot performance issues and optimize Spark SQL queries.

• Py Spark Development

• Working knowledge of Py Spark for data ingestion, transformation, and workflow orchestration.

• Ability to write and understand basic pyspark functions

• Basic understanding of Spark architecture, including executors, partitions, and caching.

• Data Engineering Skills

• Experience building ETL/ELT pipelines using Azure Databricks and load to Synapse.

• Familiarity with Delta Lake, data versioning, and ACID transactions.

• Independence & Collaboration

• Ability to work independently with minimal oversight.

• Strong communication skills to collaborate with cross-functional teams and business stakeholders.

• Proactive in identifying issues and proposing data-driven solutions.