ByteDance
ByteDance

Senior Software Engineer, AI Infrastructure - Developer Tooling

RoleEngineering
LevelSenior
LocationSan Jose, Canada, United States
WorkOn-site
TypeRegular
PostedToday
Apply now

About the role

About the Team:

We're building the AI-powered backbone of our developer tools. You'll work on three core systems: a retrieval infrastructure powering our AI products, a coding agent that assists users in writing code, and the evaluation frameworks that measure their effectiveness.

Responsibilities:

  • Retrieval Systems (RAG Infrastructure)

  • Design and scale retrieval pipelines including vector search, BM25, and hybrid retrieval strategies

  • Build and optimize embedding pipelines, chunking strategies, and re-ranking systems

  • Develop query understanding and rewriting components to improve retrieval relevance

  • Manage vector database infrastructure at production scale

  • Coding Agent Development

  • Build agent architectures that support multi-step code generation, refactoring, debugging, and diagnostics

  • Implement tool-use patterns (function calling, code execution sandboxing, file system interaction)

  • Develop context management strategies for long-form code understanding

  • Evaluation & Benchmarking

  • Design and maintain evaluation frameworks for retrieval quality and agent task completion

  • Build custom benchmark suites for real-world coding task assessment

  • Create reproducible testing infrastructure with automated regression detection

Requirements:

  • Minimum Qualifications

  • Bachelor's degree in CS, EE, or related field (or equivalent experience)

  • 4+ years of software engineering experience

  • Strong proficiency in Python (primary) and TypeScript/JavaScript

  • Experience with at least one systems-level language (C++, Rust, Go)

  • Practical experience integrating LLMs into applications (prompt engineering, context management, output parsing)

  • Understanding of agent patterns: tool use, multi-turn reasoning, error recovery

  • Familiarity with code-specific LLM tasks (generation, summarization, analysis)

  • Preferred Qualifications

  • Master's or Ph.D. in Computer Science, Machine Learning, or related field

  • Contributions to open-source developer tooling, retrieval systems, or coding assistants

  • Experience with AST parsing, code analysis tools, or language servers

  • Familiarity with rendering pipelines or cross-platform framework architecture

  • Experience deploying and optimizing ML models in production (latency, cost, reliability)

Required skills

Software engineering

System design

Troubleshooting

Can my resume get me an interview?

Get your resume score and fixes with refresh.cv.

Create a resume

About ByteDance

San Jose

Headquarters