PhD Position “Foundation Models & Efficient LLM Training” – DS-NLP Lab @HSG (m/f/d)

Über den Job

We offer
fully funded PhD position in a dynamic and fast-growing research area
Access to modern compute infrastructure
Support for publishing at top-tier conferences
A collaborative and international research environment
Full supervision and mentoring throughout your PhD journey

• You will work on your doctoral thesis in the area of foundation model training and efficient learning methods
• You design and conduct research on training dynamics of large language models, including scaling behavior, optimization, and generalization
• You incorporate linguistic insights into the development and analysis of language models, with a particular focus on token weighting strategies and their effects on training dynamics
• You engage in teaching activities (approx. 2 SWS per semester), including tutorials, exercises, and supervision of Bachelor's and Master's theses and projects
• You support the chair in organizational and research-related tasks

• You hold a master's degree in computer science, machine learning, computational linguistics, or a related field
• Desirable: prior research experience or publications in NLP, ML, or related areas
• You have strong programming skills in Python
• Desirable: experience with PyTorch or TensorFlow/Keras
• You have experience or strong interest in:
• Training large language models (transformers)
• Efficient training techniques (e.g., LoRA, quantization, sparsity, distillation)
• Scaling laws, training dynamics, or evaluation of generative models
• Dataset construction and large-scale data processing
• Desirable:
• Background or interest in linguistics or computational linguistics, particularly in relation to tokenization and token weighting strategies
• You are able to work independently, structure your research, and manage your time effectively
• You are motivated to contribute to teaching and collaborative research

"A place where knowledge is created" - As one of Europe's leading universities of economics and business administration, the University of St.Gallen (HSG), Switzerland, is committed to the education of over 10'000 students. The HSG is one of the largest employers in the region and provides an attractive and innovative environment for more than 3'500 researchers, educators and professional staff.