About the Role

Spark is our main batch big data analytics query engine at Uber. It accounts for the majority of our data compute resources, powers most of our major ingestion, ETL, data science, and machine learning workloads. Spark at Uber is used by over 2.5K weekly active users, who run 3M+ applications every week.
As part of the Spark team, you will design, implement, optimize, and manage large scale distributed computing infrastructure that will impact multiple business use cases at Uber scale. You will also have the opportunity to collaborate with the open source community for Spark and related domains.

What you'll do

1. Work with Uber data science and engineering teams to improve Spark applications and use-cases at Uber.
2. Leverage knowledge of spark internals to dramatically help improve customer's Spark job performance.
3. Design and implement distributed algorithms for large scale batch data processing.
4. Work with multiple partner teams within and outside of Uber and build cross-functional solutions in a collaborative work environment.
5. Work on Spark internals so as to continue improving Spark's robustness and performance for Uber's unique use cases.
6. Work with the community to upstream Uber's contributions to open source and also keep our internal fork up to date

What you'll need

1. Bachelor's degree in Computer Science or related field.
2. 10+ years of experience building large scale distributed software systems.
3. Solid understanding of Java for backend / systems software development.
4. BTech/ MS / PhD in Computer Science or related field.
5. Experience managing production systems with a strong availability SLA.
6. Experience working with Apache Spark or similar analytics technologies.
7. Experience working with large scale distributed systems, HDFS / Yarn.
8. Experience working with SQL Compiler, SQL Plan / Runtime Optimization.

Uber's mission is to reimagine the way the world moves for the better. Here, bold ideas create real-world impact, challenges drive growth, and speed fuelds progress. What moves us, moves the world - let's move it forward, together.
Offices continue to be central to collaboration and Uber's cultural identity. Unless formally approved to work fully remotely, Uber expects employees to spend at least half of their work time in their assigned office. For certain roles, such as those based at green-light hubs, employees are expected to be in-office for 100% of their time. Please speak with your recruiter to better understand in-office expectations for this role.
Accommodations may be available based on religious and/or medical conditions, or as required by applicable law. To request an accommodation, please reach out to accommodations@uber.com.