Inference Intern
Etched is an AI-hardware company building rack-scale frontier inference clusters.
Funding history
About Etched
Etched co-designs chips, racks, software, and manufacturing systems for efficient inference of frontier AI models, targeting throughput, latency, cost, and power efficiency across prefill and decode workloads.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will support model porting and build programming abstractions and test capabilities for rapid iteration. You will help develop runtime capabilities for multi-node inference, intra-node execution, state management, and error handling. You will optimize communication layers, profile and debug bottlenecks, co-design hardware instructions and model operations, and implement high-performance Model Toolkit components during a 12-week internship.
Requirements
- Progress toward a Bachelor's, Master's, or PhD degree in computer science, computer engineering, applied mathematics, or a related field
- Proficiency in Python and C++
- Understanding of performance-sensitive or complex distributed software systems
- Experience porting applications to non-standard accelerator hardware or platforms
- Knowledge of transformer model architectures or inference-serving stacks
Responsibilities
- Support porting state-of-the-art models and build programming abstractions and testing capabilities
- Assist with runtime development for multi-node inference, intra-node execution, state management, and error handling
- Optimize routing and communication layers using collectives
- Use profiling and debugging tools to identify bottlenecks and correctness issues
- Co-design hardware instructions and model architecture operations to maximize performance
- Implement high-performance software components for the Model Toolkit
Benefits
- Housing support for eligible relocating interns
- Daily office lunch and dinner