AI Researcher - Pre-training
German enterprise AI company building and deploying specialized large language models for sovereign, mission-critical use cases.
About Aleph Alpha GmbH
Aleph Alpha designs, builds, and deploys specialized large language models with enterprises and public institutions, emphasizing European infrastructure, data sovereignty, compliance, explainability, and domain-specific performance.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will design and run experiments on training recipes, optimizers, schedules, initialization, and PyTorch-based model architectures. You will evaluate convergence, stability, and efficiency; analyze proxy experiments and scaling behavior; monitor and debug large-scale distributed training runs; reproduce relevant research; and collaborate across compute, data, evaluation, and post-training work to inform pre-training decisions.
Requirements
- Proficiency in Python and hands-on PyTorch experience.
- Strong machine-learning foundation demonstrated through research, open-source contributions, or substantial projects.
- Solid mathematical background in linear algebra, probability, and optimization.
- Understanding of transformers and experience pre-training LLMs from scratch.
- Ability to design careful experiments and communicate evidence and results clearly.
- Ability to learn unfamiliar technical areas quickly.
- Ability to write clean, maintainable code and produce reproducible experiments.
- Clear communication and responsiveness to feedback.
Responsibilities
- Design and run ablations on training recipes and implement PyTorch-based architectural changes.
- Evaluate convergence, stability, and training efficiency.
- Run and analyze small-scale proxy experiments and contribute to scaling methods.
- Monitor large-scale training runs and investigate convergence and distributed training failures.
- Follow research literature, reproduce promising results, and propose ideas for testing.
- Collaborate with compute performance, data, evaluation, and post-training functions.