Categories
2 pages
GPU Systems and Performance Engineering
Megatron Tensor Parallelism: Rebuilding a SwiGLU Feed-Forward Block Across Two Ranks
Tensor-Parallel GQA Attention, One Head Group at a Time
