Senior PySpark Developer

We are looking for an experienced Senior Py Spark Developer with 8+ years of experience in Data Engineering and Big Data technologies. The ideal candidate should possess strong expertise in Py Spark, Python, Spark SQL, ETL development, Data Warehousing, and Cloud platforms (AWS/Azure/GCP). The candidate will be responsible for designing, developing, and optimizing large-scale data processing solutions.

Key Responsibilities:

Design and develop scalable data pipelines using Py Spark and Apache Spark.
Build and optimize ETL/ELT frameworks for processing large volumes of structured and unstructured data.
Develop high-performance Spark applications using Spark SQL, Data Frames, and RDDs.
Work closely with data architects and business teams to define data requirements.
Implement data quality, validation, and governance processes.
Optimize Spark jobs for performance, memory management, and scalability.
Integrate data from various sources including databases, APIs, cloud storage, and streaming platforms.
Support production deployments and resolve performance bottlenecks.
Mentor junior developers and participate in code reviews.

Mandatory Skills:

8+ years of IT experience.
Strong hands-on experience in Py Spark and Python.
Expertise in Apache Spark, Spark SQL, Data Frames, and ETL Development.
Strong SQL and database concepts.
Experience with Hadoop Ecosystem (Hive, HDFS, YARN).
Experience in Data Warehousing and Data Modeling.
Hands-on experience with Unix/Linux.

Education: MTech,Bachelor of Engineering,BTech

• Preferred skills: Technology->Big Data
• Data Processing->Py Spark