Staff Software Engineer AI Search

Databricks is a data and AI platform that lets organizations build analytics, AI agents, and applications on a unified, governed lakehouse.

160 Spear Street, Suite 1300, San Francisco, CA 94105, United States
About Databricks

Data engineers, analysts, and AI teams use Databricks to process large datasets, build reliable pipelines, and train models on a single governed platform. Users can run SQL analytics, serve ML predictions in real time, and deploy AI agents grounded in enterprise data. Its open lakehouse architecture provides consistent security and governance across analytical and operational workloads.

View jobs by Databricks

Skills

Candidate Availability

Required and preferred rules are kept separate and reflect the wording in the original posting.

About the Role

You will design and evolve a search platform spanning vector, keyword, and hybrid retrieval. You will solve indexing, storage, query execution, and ranking challenges, shape APIs and developer experiences, and lead production engineering efforts focused on performance, correctness, and reliability.

Requirements

  • 10+ years of experience building and operating large-scale distributed systems
  • Vector search
  • Keyword search
  • Hybrid retrieval
  • Indexing
  • Ranking
  • Retrieval infrastructure
  • Relevance optimization
  • Storage systems
  • Recommendation engines
  • Database internals
  • Algorithms
  • Data structures
  • System design
  • Architecture leadership for performance-sensitive systems

Responsibilities

  • Drive the architecture and evolution of the search product
  • Deliver scalable, low-latency, high-quality retrieval across multimodal datasets
  • Solve indexing, storage, query execution, and ranking challenges
  • Shape APIs and developer experiences for AI-powered search applications
  • Lead engineering efforts from design through production rollout
  • Contribute to core search, discovery, and recommendation systems
  • Mentor engineers through design reviews and technical discussions
  • Influence long-term technical direction for search and data systems

Benefits

  • Annual performance bonus
  • Equity