About the Course: Diffusion Models — Image and Audio Generation

Topics covered

• •Understand what generative models are and where diffusion models fit among them.
• •Break down the forward process (adding noise) and the reverse process (step-by-step denoising).
• •Master the basic components of the 🤗 Diffusers library: pipelines, the UNet model, schedulers.

Topics covered

• •Master the full training loop for a diffusion model trained from scratch on your own data.
• •Understand when training from scratch is impractical and why fine-tuning a pretrained model is an efficient alternative.
• •Learn to fine-tune a pretrained model on a new dataset using Diffusers.

Topics covered

• •Understand how guidance (control at sampling time) differs from conditioning (a condition set during training).
• •Learn to steer an unconditional model with an arbitrary loss function, including CLIP guidance from text.
• •Break down the ways to feed a condition into a UNet: extra channels, adding embeddings, cross-attention.

Topics covered

• •Understand what latent diffusion is and why a VAE is needed.
• •Break down the components of Stable Diffusion: VAE, the CLIP text encoder, a UNet with cross-attention, the scheduler.
• •Master classifier-free guidance (CFG) and the effect of the guidance scale.

Topics covered

• •Understand how distillation speeds up diffusion model sampling.
• •Master the main approaches to image editing, including DDIM inversion.
• •Break down how diffusion is applied to video and audio (spectrograms).