Software Engineer, Performance Modeling, SI Hardware

What to Expect

The Tesla AI Hardware team is at the forefront of revolutionizing artificial intelligence through cutting-edge hardware innovation. Comprising brilliant engineers and visionaries, the team designs and develops advanced AI inference chips tailored to accelerate Tesla’s machine learning capabilities. A key part of this effort is Dojo, Tesla's custom supercomputer system. The work of Tesla's AI Hardware team powers the neural networks behind Full Self-Driving (FSD), and Tesla humanoid robot, Optimus, pushing the boundaries of computational efficiency and performance. By creating custom silicon and optimized architectures, the team ensures Tesla remains a leader in AI-driven automotive and energy solutions, shaping a future where intelligent machines enhance human life.

As a key member of the Tesla AI hardware team, the AI/ML Modeling Engineer will drive the modeling, and optimization of next-generation tensor compute hardware. This role requires a deep expertise in computer architecture, and proven proficiency in performance modeling and optimization. The ideal candidate has a thorough understanding of AI workloads, and thrives in fast-paced environments, delivering elegant, high-performance solutions that push the boundaries of AI hardware efficiency.

What You’ll Do

• Collaborate closely with system architects, micro architects, ML model designers, and compiler engineers to create highly efficient hardware designs on accelerated timelines
• Develop and validate sophisticated performance models to evaluate architecture and microarchitecture tradeoffs, guiding informed decision-making
• Define comprehensive workload suites and key performance metrics to benchmark and evaluate hardware capabilities
• Analyze and assess implementations of machine learning algorithms on Tesla AI hardware, identifying bottlenecks and opportunities for improvement
• Participate in hardware/software co-design efforts to align future hardware capabilities with evolving algorithmic demands
• Contribute to code reviews, testing, and debugging processes to maintain exceptional code quality and reliability
• Stay abreast of cutting-edge advancements in AI workloads, domain-specific languages, computer architecture, and simulation methodologies to inform strategic directions

What You’ll Bring

• Degree in Engineering, Computer Science, or a related field, or equivalent practical experience demonstrating exceptional ability
• In-depth knowledge of Large Language Models (LLMs), transformer architectures, including their training, inference processes, and optimization techniques
• Strong expertise in CPU, GPU, and/or ML accelerator microarchitectures, with hands-on experience in performance characterization
• Proficiency with deep learning frameworks such as Py Torch, JAX, Pallas, or similar tools
• Robust programming skills in C/C++, Python, or other relevant languages
• Solid understanding of physical design constraints, including power, performance, and area (PPA) tradeoffs in hardware development
• Exceptional problem-solving abilities, capable of dissecting complex issues and devising innovative, practical solutions
• Excellent communication and interpersonal skills, fostering effective collaboration across teams, including leadership and engineering stakeholders
• Demonstrated experience in performance analysis, including the use of simulation frameworks and profiling tools