IT Delivery

Key Responsibilities:

• Lead the design and development of scalable data pipelines and ETL/ELT workflows using Databricks on Azure.

• Architect and implement data lakehouse solutions, optimizing storage, compute, and query performance for large datasets.

• Develop, optimize, and maintain Databricks notebooks, jobs, and workflows for batch and near real-time data processing.

• Define and enforce coding standards, best practices, and reusable frameworks for Databricks-based data engineering.

• Collaborate with data architects, analysts, and business teams to understand requirements and translate them into technical designs.

• Implement robust data quality checks, validation rules, and monitoring to ensure reliability and trust in data products.

• Optimize Spark jobs for performance and cost efficiency, including cluster sizing, caching strategies, and partitioning.

• Ensure security, governance, and compliance of data solutions, including access control and data lineage within Databricks and Azure.

• Troubleshoot complex data issues, perform root cause analysis, and drive long-term fixes and improvements.

• Mentor junior engineers, provide technical guidance, conduct code reviews, and support knowledge sharing within the team.

• Contribute to roadmap planning, effort estimation, and delivery management for data engineering initiatives.

Technology->Cloud Platform->Databricks AI Services,Technology->Data Engineering->Databricks

• Knowledge of more than one technology
• Basics of Architecture and Design fundamentals
• Knowledge of Testing tools
• Knowledge of agile methodologies
• Understanding of Project life cycle activities on development and maintenance projects
• Understanding of one or more Estimation methodologies, Knowledge of Quality processes
• Basics of business domain to understand the business requirements
• Analytical abilities, Strong Technical Skills, Good communication skills
• Good understanding of the technology and domain
• Ability to demonstrate a sound understanding of software quality assurance principles, SOLID design principles and modelling methods
• Awareness of latest technologies and trends
• Excellent problem solving, analytical and debugging skills

Education: MCA,MSc,MTech,Bachelor of Engineering,BCA,BSc,BTech

Preferred skills: Technology->Data Engineering->Databricks,Technology->AI Engineering->Databricks AI Engineering Services