Performance & Debug Software Engineer – AI Inference

Skills

About the Role

You will improve end-to-end LLM inference performance on custom AI accelerators, including FPGA-based Atlas systems and Asimov ASIC-based Titan systems. You will profile and tune the whole stack, from hardware-accelerated attention to runtime and host software, build benchmarks and workload traces, develop distributed inference and KV cache capabilities, work with test engineers on regression tests and release gates, and influence hardware and software architecture where measurements reveal limits.

Requirements

• Strong C++ and Python skills, with hands-on experience profiling and optimizing complex systems that exercise Linux VFIO interfaces
• Deep understanding of memory hierarchy, concurrency, data movement, batching, scheduling, and performance measurement
• Hands-on experience using profilers, counters, traces, and statistical methods
• Demonstrated ability to debug across application, runtime, driver, firmware, and hardware boundaries
• Experience bringing up software on new silicon and analyzing performance in customer production deployments
• Enthusiasm for AI-accelerated software development, including using coding agents and orchestrating subagents
• Authorized to work in the U.S.

Responsibilities

• Profile and optimize latency, throughput, utilization, and efficiency across attention, accelerator configuration, device runtime, host software, and on-device execution
• Use performance counters, traces, telemetry, and simulation to isolate regressions and validate fixes
• Develop and optimize distributed inference capabilities, including KV cache management and data movement across heterogeneous hardware
• Build representative benchmarks and workload traces that expose bottlenecks
• Partner with test engineers to turn performance findings into regression tests and release gates
• Influence hardware and software architecture and interfaces where measured performance reveals structural limitations

Benefits

• Fully company-paid medical, dental, and vision insurance for you and dependents
• Company-paid life and disability coverage with voluntary options
• Supplemental hospital, critical illness, and accident coverage
• Unlimited paid time off
• 13 paid company holidays
• Remote-first culture with company-provided computer and home office setup
• Equity
• 401(k) with company matching from day one
• H-1B visa transfer facilitation for eligible candidates