Software Engineer, Metrics, AI Evaluation

What to Expect

The AI Evaluation team is the main line of defense in ensuring product safety. Working alongside our AI team, you will design metrics for open and closed loop simulation testing on telemetry collected from the fleet. These results help drive key decisions about end-to-end model architecture, data integrity, and exported model performance. Your work will directly impact L4 Unsupervised robotaxi operation, FSD v14+ customer satisfaction, and Optimus success. A strong candidate will have a computer science, ML or robotics engineering background, strong attention to detail, self-driven attitude, and passion to make an impact.

What You’ll Do

• Design metrics to evaluate model predictions and vehicle motion across every scenario a self -driving robotaxi could ever encounter
• Work with AI researchers to evaluate & compare end-to-end FSD models
• Own projects from inception to completion and present projects to org leadership
• Build tooling and automation to validate open-loop and closed-loop simulations
• Measure the correlation between real-world driving and simulated rollouts, and drive down the gaps end to end
• Mine data sourced from the vast Customer fleet to surface high-value cases and turn them into reusable, high-signal evaluation scenarios
• De-noise eval results to surface high value signal that guides model iterations and signoff decisions

What You’ll Bring

• Degree in Computer Science, Physics or Robotics Engineering, or equivalent in experience and evidence of exceptional ability
• Strong Python coding abilities
• Exposure to Num Py and Pandas for data manipulation
• Experience with SIL, HIL, open-loop, and closed-loop robotics eval a plus
• Experience with computer vision systems, ADAS features, path-planning, real-time embedded systems, sensor calibration, C++ and/or CAN is a plus