AI Infrastructure: LLM-D, vLLM and GPUs

This is a no-nonsense introduction to AI infrastructure for absolute beginners. No machine learning background needed. The course focuses on building the mental model of what actually runs behind every AI product you've used: the GPUs, the model servers, and the systems around them, so you can reason about it instead of memorizing buzzwords.

The course builds one thing, end to end: a setup that serves a large language model, the kind of thing behind ChatGPT, to a lot of people at the same time. It starts with a question typed into a chat box and works its way down to the hardware answering it. Every lesson follows the same shape. Something breaks or hits a wall, whether the model doesn't fit on the GPU, the answer stalls before it starts, or the load balancer sends you to a server that has never seen you before. Then comes the piece that fixes it, and the new problem that piece creates. Nothing shows up before you've felt the pain it solves.

Across a dozen lessons you'll build the full foundation: what a model actually is, what it takes to load one onto a GPU, vLLM and what a model server really does, tokens and why a model writes one word at a time, the pause before the answer and the stream after it, the model's scratchpad and why reusing saved work is the whole game, batching many users onto one GPU and the memory ceiling that stops you, throughput versus how fast a single answer feels, slicing a model too big for any single GPU across several of them, why the load balancer we've trusted for decades breaks on this traffic, and llm-d, the open project from Red Hat, Google, IBM, and NVIDIA, which routes each request to the server that already holds your work, splits prompt-reading from answer-writing, and runs as real pods on a Kubernetes cluster. Everything is in plain layman's language; you don't need a math degree to follow it.

There's also a free hands-on lab if you want to feel some of this yourself: you start with a model server that answers one request at a time while users sit waiting, and by the end you've built a real vLLM server handling many of them at once. It comes with plenty of hints and solutions, so it's fine if you're not feeling confident yet.

We close on where this leaves you. This was infrastructure work from start to finish, servers, memory, routing, and clusters, which means you don't throw away the foundation you already have. You add the AI layer on top of it.