Software Engineer AI Model Enablement and Inference

Skills

About the Role

You will analyze model architectures, implement and optimize TCL kernels, and integrate models into Furiosa-LLM. You will validate correctness on NPUs, automate benchmarking and regression testing, evaluate inference-framework techniques, and provide technical guidance on model and inference trade-offs.

Requirements

• Deep understanding of transformer-based LLMs, including attention variants, mixture-of-experts architectures, and KV-cache behavior
• Strong Python skills and experience reading, modifying, and debugging model implementations in PyTorch or a comparable framework
• Experience implementing, debugging, and optimizing tensor operations or accelerator kernels
• Understanding of LLM inference performance, including prefill, decode, batching, and latency-throughput trade-offs
• Ability to read and reason about Rust or C++ code
• Technical communication and cross-team collaboration skills

Responsibilities

• Analyze model architectures, algorithms, and reference implementations to identify implementation requirements and trade-offs
• Design, implement, and optimize model-specific TCL kernels
• Integrate models and TCL kernels into Furiosa-LLM
• Build reusable analysis and integration tools and automate validation and benchmarking
• Validate kernel and model correctness on NPUs and build regression tests
• Evaluate and apply relevant techniques from vLLM and SGLang
• Study state-of-the-art models and provide technical guidance on model capabilities and inference requirements