Welcome to the Advanced Machine Learning on HPC Clusters workshop. This intensive, 6-7 hour session is designed to bridge the gap between high-level machine learning theory and the practical realities of large-scale computation. As datasets grow and architectures like Transformers become the standard, the need for High-Performance Computing (HPC) has never been greater. This workshop will guide you through the transition from local development to running sophisticated models on professional-grade HPC services.
By the end of this course, you will be able to:
Implement Transformer Architectures: Master the "Attention" mechanism and understand how Vision Transformers (ViT) have revolutionised fields beyond text. Optimise Model Training: Apply advanced regularisation and data augmentation techniques like Mixup and CutMix to improve model generalisation. Scale to HPC and Parallel: Navigate Slurm-based environments to orchestrate multi-node jobs and leverage massive parallel computing power.
Summary (30 min) - Basics and HPC setup Transformers (90 min) - Vision Transformers and Attention network Regularisation (60 min) - Training and adapting models Data Augmentation (60 min) - Mixup and CutMix, creating new training samples HPC Scaling and Parallel (60 min) - Slurm-based HPC services
Python programming Familiarity with Jupyter notebooks Understanding of machine learning concepts Access to HPC cluster with GPU resources Recommend to have familiarities with the contents from workshops of: Intro to Bash Shell, Intro to Python, ML01, and ML02.