Learning path

Serving Performance

A short path through the runtime decisions that turn model capability into production latency and throughput.

Advanced3 lessons17 min

Path outcome

What this path gives you

You can separate raw model speed from batching, memory layout, decode strategy, and hardware specialization.

The Memory Trick Behind Fast LLM Serving

Bigger GPUs help, but fast LLM serving is mostly a scheduling and KV cache problem. PagedAttention, block allocation, batching, and prefix caching decide how much useful work fits in memory.

AI Engineering / 6 min read

How Speculative Decoding Makes LLMs Feel Faster

Speculative decoding speeds up LLM inference by letting a cheap draft model guess several tokens, then asking the expensive target model to verify those guesses in one parallel pass.

AI Engineering / 6 min read

17,000 Tokens per Second, Burned Into Silicon

If language models can be compiled directly into silicon, inference stops looking like software running on a GPU and starts looking like the computer itself.

AI Engineering / 5 min read