Python, Spark Scala Developer

Join a fast-paced engineering team where you’ll build and optimize data-driven solutions using Python, Apache Spark, and Scala. In this role, you’ll collaborate closely with data engineers, analysts, and platform teams to deliver reliable pipelines and scalable processing jobs that turn raw data into trusted insights. You’ll contribute across the development lifecycle—design, implementation, testing, deployment, and performance tuning—while learning from peers and sharing your own expertise. If you enjoy solving complex data challenges, improving runtime efficiency, and writing clean, maintainable code, this is a great opportunity to grow your impact in a collaborative, quality-focused environment.

Data Engineering & Development:

• Develop and maintain batch/stream data processing jobs using Spark with Scala and/or Python.

• Build reusable components and utilities to standardize ingestion, transformation, and validation patterns.

• Implement efficient transformations, joins, aggregations, and partitioning strategies for large-scale datasets.

Performance, Quality & Reliability

• Tune Spark jobs for performance (e.g., caching, shuffles, partition sizing) and troubleshoot failures or bottlenecks.

• Write unit/integration tests and ensure code quality through reviews, refactoring, and documentation.

• Monitor job runs, analyze logs/metrics, and drive continuous improvements in stability and runtime.

Data & SQL:

• Use SQL to validate data, reconcile outputs, and support downstream reporting/analytics needs.

• Collaborate with stakeholders to clarify requirements and translate them into robust data processing logic.

• Bachelor’s degree in Computer Science, Engineering, or a related field.

• 2–5 years of hands-on experience building solutions with Python.

• 2–5 years of experience working with Apache Spark for large-scale data processing.

• 2–5 years of experience developing with Scala (or strong ability to work across Scala and Python codebases).

• Working knowledge of SQL for querying, validation, and troubleshooting datasets.

• Strong problem-solving skills and ability to collaborate effectively in a team environment.

Education: Bachelor of Engineering

• Preferred skills: Technology->Analytics
• Packages->Python
• Big Data,Technology->Big Data
• Data Processing->Spark,Technology->Functional Programming->Scala