Senior Engineering Manager, AI
Skills
About the Role
You will lead the AI Platform team that builds and operates the core AI Gateway, the high-throughput highway for all Large Language Model traffic across the entire AlphaSense ecosystem. You will manage, mentor, and grow a team of up to 10 senior and staff software engineers distributed across North America, fostering a culture of technical excellence, predictable delivery, and career growth. You will own the strategy and architecture of an enterprise AI Gateway processing up to 1 trillion LLM tokens per week with projected 10x growth, ensuring high availability, sub-millisecond routing, caching, queueing, and robust fallback mechanisms. You will strategically guide the integration, evaluation, and management of 6+ major LLM vendors and 50+ distinct models, optimizing for latency, cost, and reliability. You will partner closely with product management, AI research teams, and executive engineering leads to align the gateway roadmap with core product features and organizational goals. You will define and champion software engineering best practices, architectural standards, and token-based cost-tracking telemetry across the AI Platform organization.
Requirements
- Proven track record of managing and growing high-performing teams of senior and staff engineers
- Experience designing, building, and operating high-throughput, low-latency distributed systems including routing, caching, queueing, and rate limiting
- Ability to engage in high-level architectural discussions and provide technical direction in architectural reviews
- Exceptional communication and partnership skills to align engineering, product, and executive stakeholders
- Active embrace of AI tools in developer workflows
- Experience in the applied AI space or with frameworks such as LangChain, LangGraph, LiteLLM, or OpenRouter (nice to have)
- Experience with orchestrators and proxies like Temporal or Envoy (nice to have)
- Experience with Python, Java, and highly distributed caching/queueing systems
Responsibilities
- Manage, mentor, and grow a team of up to 10 senior and staff software engineers
- Foster a high-performance culture of technical excellence, predictable delivery, and career growth
- Own the strategy and architecture of the enterprise AI Gateway processing up to 1 trillion LLM tokens per week
- Ensure high availability, sub-millisecond routing, caching, queueing, and robust fallback mechanisms
- Guide the integration, evaluation, and management of 6+ major LLM vendors and 50+ distinct models
- Optimize for latency, cost, and reliability across different LLM providers
- Partner with product management, AI research teams, and executive engineering leads to align the gateway roadmap with product features and organizational goals
- Define and champion software engineering best practices, architectural standards, and token-based cost-tracking telemetry
Benefits
- Equity
- Generous benefits program
