Senior Apache Spark Technical Lead - Scala, Python

Job Summary

• Hands on knowledge of SPARK, scala, Java etc.
• Hands on knowledge of RDBMS like MS-SQL/Oracle
• Knowledge of CI/CD tools like u Deploy

Responsibilities:

• Development & Optimization: Develop, test, and deploy production-grade Spark applications in Scala, ensuring optimal performance, scalability, and resource utilization.
• Technical Leadership: Provide technical leadership and guidance to a team of data engineers, fostering a culture of technical excellence, collaboration, and continuous improvement.
• Code Review & Best Practices: Conduct rigorous code reviews, establish coding standards, and enforce best practices for Spark Scala development, data governance, and data quality.
• Performance Tuning: Identify and resolve performance bottlenecks in Spark applications through advanced tuning techniques (e.g., shuffling, partitioning, memory management, caching).

Key Responsibilities

• Hands on knowledge of SPARK, , scala, Java etc.
• Hands on knowledge of RDBMS like MS-SQL/Oracle
• Knowledge of CI/CD tools like u Deploy

Responsibilities:

• Development & Optimization: Develop, test, and deploy production-grade Spark applications in Scala, ensuring optimal performance, scalability, and resource utilization.
• Technical Leadership: Provide technical leadership and guidance to a team of data engineers, fostering a culture of technical excellence, collaboration, and continuous improvement.
• Code Review & Best Practices: Conduct rigorous code reviews, establish coding standards, and enforce best practices for Spark Scala development, data governance, and data quality.
• Performance Tuning: Identify and resolve performance bottlenecks in Spark applications through advanced tuning techniques (e.g., shuffling, partitioning, memory management, caching).

Skill Requirements

Scala/Java with Spark

Python, SQL and Agile

Other Requirements

1.Relevant certifications in apache spark, scala, or python are a plus