Research Engineer Latency and Throughput

Skills

About the Role

You will research and prototype LLM inference techniques that reduce latency and increase throughput. You will form hypotheses, run controlled experiments, validate performance and quality outcomes, build research tooling, and collaborate on integrating proven methods into a vLLM-based serving stack and model-training workflows.

Requirements

• Knowledge of machine learning and LLM training techniques, including architecture, tokenization, fine-tuning, and evaluation
• Experience with Python and PyTorch, or JAX with strong Python proficiency
• Knowledge of CUDA and GPUs
• Research experience through a thesis, publication, research project, or laboratory or industry work
• Experience forming hypotheses, designing controlled comparisons, and drawing conclusions from results
• Knowledge of state-of-the-art LLM inference
• Inference or serving experience is beneficial
• Experience with vLLM, production LLM deployment, or serving systems is beneficial

Responsibilities

• Research and prototype inference techniques including speculative decoding, quantization, KV compression, watermarking, and model architectures
• Design and run experiments to validate or refute ideas and demonstrate latency or throughput gains without unacceptable quality loss
• Collaborate to integrate validated techniques into the vLLM-based serving stack
• Partner on co-training, retraining, and auxiliary-model training when needed
• Build tooling and validation platforms for AI-driven inference research

Benefits

• Healthcare coverage
• Parental leave
• Retirement plans
• Relocation support
• Wellness programs
• Meal allowances
• Transportation allowances
• Location-specific perks
• Accommodation and travel for the first week of onboarding for remote senior hires