Synaptic Radio

Principal ML engineer focused on foundation models, agentic systems and scientific AI. I write about the math, systems and engineering judgement behind reliable ML workflows.

Home
Archives
Categories
Tags
Resume
Search
1. Dark Mode

Search

Categories

Distributed Training GPU Systems and Performance Engineering Inference and Serving Tensor Parallelism Data Quality and Data Systems Deep-Learning Mechanisms Evaluation and Experimentation Multimodal Learning Reliability and Observability Software Architecture

Tags

PyTorch Megatron-LM Tensor Parallelism Checkpointing Coordinate Encoding Distributed Training Evaluation Fault-Tolerance Grouped-Query Attention Kv-Cache

Section

20 pages

Tags

Kv-Cache

Llm-Inference

Paged-Attention

Systems

VLLM

Distributed Training

Grouped-Query Attention

Megatron-LM

PyTorch

SwiGLU

1 2

© 2026 Anshuman Sahoo