Senior LLM Inference Performance and Evaluation Engineer

Skills

About the Role

You build benchmark pipelines for LLM inference performance, create model launch gates, maintain representative workloads, compare model and runtime options, automate regression detection, and collaborate with runtime engineers and SRE to identify bottlenecks and define service objectives and alert thresholds.

Requirements

• 5+ years of experience in ML infrastructure, performance engineering, model evaluation, QA automation, or backend testing for production systems.
• Experience with TTFT, TPOT/ITL, request latency, token throughput, concurrency, and GPU-utilization metrics.
• Strong Python skills.
• Go experience preferred.
• Familiarity with OpenAI and Anthropic APIs, vLLM, Dynamo, SGLang, Triton-style servers, and Kubernetes test environments.
• Ability to design statistically meaningful tests and communicate tradeoffs.
• Experience building dashboards, reports, and release gates.

Responsibilities

• Build benchmark pipelines for latency, throughput, concurrency, error rate, and GPU utilization.
• Create model launch gates for API compatibility, streaming, tool calling, reasoning, multimodal behavior, and long-context cases.
• Maintain synthetic, replayed, and customer-like workloads.
• Compare model, runtime, and provider options and recommend routing, fallback, pricing, and capacity decisions.
• Automate regression detection in CI/CD and staging.
• Identify bottlenecks and verify performance improvements.
• Convert benchmark results into SLOs and alert thresholds.
• Build dashboards, reports, and release gates.

Benefits

• Inclusive and respectful work environment.
• Opportunity to contribute directly to the future of the digital asset industry.
• Involvement in new projects and process development.
• Personal accountability, autonomy, fast growth, and learning opportunities.
• Training and mentoring opportunities.
• Attractive welfare benefits.