Categories
Distributed Training
Inference and Serving
Software Architecture
Deep-Learning Mechanisms
GPU Systems and Performance Engineering
Tensor Parallelism
Transformer Architecture
Research Methodology
2026
Ray mental model: tasks, actors, objects, scheduling, placement
Tensor Parallelism II: GQA Attention, One Head Group at a Time
Tensor Parallelism I: Rebuilding a Megatron SwiGLU Block Across Two Ranks
Book review: Circe by Madeline Miller
