Senior Apache Spark Technical Lead - Scala, Python

Job Summary

• Expert-level proficiency in Python and SQL.

• Hands-on experience with Spark for big data processing.

• Databricks

• Strong experience with AWS/Azure cloud services (e.g., S3, EC2, Redshift, EMR).

• Expertise in data visualization and reporting with Tableau or Power BI.

• Streaming & Data: Kafka, Pandas, ETL Pipelines

• Generative AI: Large Language Models (LLM), Lang Chain, Prompt Engineering, AI Integrations

• Strong hands-on experience in SQL for complex transformations, data validation, and performance tuning

Key Responsibilities

null

Skill Requirements

null

Other Requirements

1.Relevant certifications in apache spark, scala, or python are a plus