Everything you need to optimize model training and inference workloads with GPUs, CUDA, and PyTorch

Authored by Chris Fregly, AI product and performance engineering leader, formerly AWS, Databricks, and Netflix

AI systems are layered and fast‑moving. Chris breaks the complexity down into a reference that will set the standard for years.” — Chris Lattner, CEO at Modular

“CUDA kernels, distributed training, compilers, disaggregated inference — finally in one place. An encyclopedia of ML systems.” — Mark Saroufim, PyTorch engineer at Meta and founder of GPU Mode Community

“Squeezing the most performance out of your AI system is what separates the good from the great. This is the missing manual.” — Sebastian Raschka, ML/AI researcher and thought leader

20 chapters covering every layer of AI systems’ performance:

• CUDA kernel optimization: occupancy tuning, warp efficiency, memory access patterns
• PyTorch at scale: torch.compile, Triton, distributed training, FSDP
• Inference engineering: disaggregated prefill/decode, KV cache, speculative decoding, MoE routing
• DeepSeek architecture deep dives: Fire-Flyer, FlashMLA, inline PTX, and more
• 175+ item optimization checklist