Categories
2 pages
Inference and Serving
Ray mental model: tasks, actors, objects, scheduling, placement
Tensor Parallelism II: GQA Attention, One Head Group at a Time
