Data and AI Engineer Equities Technology
Skills
About the Role
You will design, build, and maintain scalable data and AI-enabled platforms for equities. You will develop data pipelines, ETL/ELT frameworks, AI services, RAG workflows, and Azure-native solutions. You will productionize AI proofs of concept, implement CI/CD, monitor systems, resolve defects, document technical designs, and collaborate on requirements and delivery.
Requirements
- Bachelor’s degree in information technology or a related field
- 4–6+ years of hands-on experience with Databricks, Spark, Azure Data Factory, Azure Synapse, Python, ADLS, and Azure Functions
- Experience designing and managing Synapse and Azure Data Factory pipelines, activities, and linked services
- Ability to build full and incremental data loads from Azure and on-premises data sources
- Experience designing reusable ETL/ELT frameworks and orchestrating pipelines across ADF, Synapse, and Databricks
- Experience implementing LLM-enabled or RAG-based production solutions
- Proficiency with REST APIs, data gateways, and third-party integrations
- Strong SQL skills, data modeling experience, analytical storage knowledge, and performance-tuning experience
- Experience with Azure DevOps and YAML-based CI/CD pipelines
- Familiarity with Azure Key Vault, automation runbooks, Logic Apps, and cloud security best practices
- Experience with Azure AI Services, including OpenAI and embeddings, is preferred
- Familiarity with Microsoft Fabric is preferred
- Proficiency in ASP.NET, .NET Core, or C# is preferred
- Financial services, capital markets, or regulated-environment experience is preferred
Responsibilities
- Design, build, and maintain scalable data pipelines using Databricks, Azure Data Factory, and Azure Synapse
- Implement ETL/ELT workflows for structured and unstructured financial data
- Ensure data quality, lineage, governance, security, and observability across pipelines and storage layers
- Design and optimize data models and analytical schemas
- Build reusable ingestion and transformation frameworks for analytics and AI workloads
- Build and deploy AI-enabled services, agents, and workflows for equity research, trading, sales, and client service
- Implement LLM-based and agentic patterns, including Retrieval-Augmented Generation
- Integrate AI capabilities through APIs, batch jobs, and event-driven workflows
- Productionize AI and data science proofs of concept into secure, scalable, monitored services
- Optimize prompts, embeddings, orchestration logic, and inference workflows
- Establish monitoring, alerting, lifecycle management, and runbooks for AI solutions
- Support migration of legacy data and application solutions to Azure-native architectures
- Implement CI/CD pipelines using Azure DevOps and YAML
- Use Azure services to build secure, reliable, maintainable solutions
- Develop operational dashboards for pipeline health, SLAs, performance, and cloud spend
- Define functional and technical requirements with stakeholders
- Participate in Agile ceremonies, sprint planning, and retrospectives
- Lead testing, analyze issues, and resolve defects
- Document technical designs, integrations, playbooks, and runbooks
- Monitor industry trends and recommend aligned technology adoption
Benefits
- Discretionary annual bonus and/or commission-based incentives
- Medical, dental, and vision coverage
- Employer-paid short-term and long-term disability insurance
- Employer-paid life insurance
- 401(k)
- Profit sharing
- Paid time off
- Maven family and fertility benefit
- Parental leave, including adoption, surrogacy, and foster placement
- Hybrid work schedule