Generative AI Foundations: Algorithms and Architectures
Ashkan Jasour |
Last modified 2025-04
“Generative AI Foundations: Algorithms and Architectures” offers a comprehensive and technical guide to modern generative modeling. It introduces fundamental principles, key algorithms—such as flow models, diffusion models, VAEs, GANs, and autoregressive models—and the neural architectures—such as CNNs, U-Nets, Transformers, and multimodal frameworks—that power state-of-the-art generative AI systems. The course balances mathematical depth with conceptual clarity, presenting precise formulations of modeling goals, corresponding training objectives, and the architectures that realize them.
📄 View Full Course Slides
💻 View Course Codes
Generative AI - Algorithms
- Flow Models
- Ordinary Differential Equation (ODE)-based Flow Models
- Denoising Diffusion Models (DDMs)
- Stochastic Differential Equation (SDE)-based Denoising Diffusion Models
- Autoencoders and Variational Autoencoders (VAEs)
- Latent Space Diffusion Models
- Autoregressive Models
- Generative Adversarial Networks (GANs)
Generative AI - Architectures
- Multilayer Perceptrons (MLPs)
- Training and Loss Functions Types
- Backpropagation Algorithm
- Stochastic Gradient Descent (SGD) and Adam Optimizer
- Common Training Issues, Regularization in Deep Learning, and Scaling Laws for Deep Learning
- Convolutional Neural Networks (CNNs)
- PixelCNN
- U-Net Denoising Model
- Recurrent Neural Networks (RNNs)
- LSTM (Long Short-Term Memory)
- GRU (Gated Recurrent Unit)
- Transformers: Self-Attention, Multi-Head Attention, and Cross-Attention
- Diffusion Transformers (DiTs)
- Vision Transformers (ViTs)
- Attention-Based U-Nets
- Multimodal Models
- Foundation Models
Appendices
- Key Differential Equations in Generative AI
- Fine-tuning Large Language Models
- Deep Reinforcement Learning - Key Concepts and Summary (PG, VPG, PPO, DDPG, TD3, SAC)
- Reinforcement Learning from Human Feedback (RLHF) and Imitation Learning
- Adversarial Training, Robustness in Language Models, and Language Models Evaluation
- Python Libraries for Generative AI