FixTheRobot Explore Robotics / Large Language Models

Learning track

Learn Large Language Models from Scratch

Understand how large language models work by building a GPT-style model step by step. Learn tokenization, attention, Transformers, pretraining, fine-tuning, evaluation, and modern LLM engineering through theory and hands-on implementation.

12 of 12 modules are complete. 65 of 65 theory lessons and 35 of 35 labs are open (12 of the labs are PyTorch notebooks). Items marked coming soon are not yet available. Start with the first lesson.

What you will learn

By the end of the track you can explain how an autoregressive language model generates text, build tokenizers and training data, implement attention and a GPT-style Transformer in PyTorch, pretrain a small model, control text generation, fine-tune for classification and instructions, adapt models efficiently with LoRA, evaluate models honestly, and build applications around them with retrieval and tools. The method is understand, implement, train, evaluate, improve, apply: understanding through implementation, not through calling an API.

Understand, implement, train, evaluate, improve, applySix boxes in a row: understand, implement, train, evaluate, improve, apply, with an arrow from improve back to implement.UnderstandImplementTrainEvaluateImproveApplyiterate
The teaching loop of the track. Every module follows it at a different scale.

Who it is for

Students and engineers who can read Python and want to understand LLMs from the inside, including robotics learners who will meet language models in the Physical AI and Vision-Language-Action tracks. You do not need prior machine-learning experience.

Prerequisites

  • Python: functions, lists, dictionaries and classes. The browser labs run Python in your browser; no installation is needed.
  • Maths: vectors and matrices (a dot product, a matrix product), the idea of a derivative, and basic probability. Each lesson recalls what it uses.
  • For the notebook labs: a computer or hosted notebook where you can install PyTorch. A GPU is optional; every notebook has a CPU-friendly path with a tiny model.

What you will build

A tokenizer and data pipeline, scaled dot-product and multi-head causal attention, a GPT-style Transformer, a training loop, samplers, a classifier and an instruction-following model, a LoRA experiment, an evaluation harness, a small retrieval application, and in the capstone a complete, evaluated LLM system with a written report. These are small educational models; they do not match the capabilities of production LLMs, and the track says so wherever it matters.

How the track works

KindWhere it runsWhat it is for
Theory lessonYour browserIntuition, mathematics with tensor shapes, worked examples, common mistakes and three knowledge-check questions.
Browser labYour browser (Python via Brython)Pure-Python exercises on the mechanics: tokenizers, attention arithmetic, sampling, metrics, memory estimates. Graded automatically on hidden cases. These are educational simulations, not trained models.
PyTorch notebook labYour computer or a hosted notebookReal PyTorch implementation and training of small models, with self-checks inside the notebook. The site cannot run these for you.

Primary reference and further resources

The curriculum follows Sebastian Raschka’s book Build a Large Language Model (From Scratch): see the book companion site, the official code repository and the publisher page. The explanations, examples and exercises on this site are original; each lesson links to the matching book chapter and code. Modules 8 to 11 add optional modern material from research papers and official documentation, clearly marked as going beyond the book’s core path.

Connection to Physical AI and VLA

This track covers language models themselves. How a language model is connected to a robot (grounding, skills, safety layers, vision-language-action policies) is taught in the Physical AI and VLA tracks and is linked, not repeated, here. Module 11 shows why a model must never bypass a robot’s safety layer, and how tool calling and structured outputs connect to validated robot functions.

Module 1

Understanding Large Language Models

What a language model is, how next-token prediction becomes text generation, how training differs from inference, what scale means, and the plan for building a GPT-style model.

You will be able to: explain autoregressive generation as repeated next-token prediction; distinguish training, inference, pretraining and fine-tuning; read parameter counts, token counts and context windows; state what LLMs can and cannot do and describe the build plan.

All 7 items available

Theory (5)

  1. LM1-T1What a Language Model IsTheory
  2. LM1-T2Autoregressive GenerationTheory
  3. LM1-T3Training Versus InferenceTheory
  4. LM1-T4Parameters, Tokens and Context WindowsTheory
  5. LM1-T5What LLMs Can and Cannot Do, and the Build PlanTheory

Labs (2)

  1. LM1-L1Explore Next-Token Probabilities and DecodingBrowser lab
  2. LM1-L2Trace Text to Generated OutputBrowser lab

Module 2

Text Data and Tokenization

Normalising text, building vocabularies, byte-pair encoding, special tokens, input-target pairs, sliding windows, batching and data splits.

You will be able to: turn raw text into token IDs and back; build a vocabulary and explain subword tokenization with byte-pair encoding; construct input-target pairs for next-token prediction; batch data correctly and avoid leakage between training and validation sets.

All 9 items available

Theory (6)

  1. LM2-T1Text Normalization and PreprocessingTheory
  2. LM2-T2Characters, Words, Subwords and Token IDsTheory
  3. LM2-T3Byte-Pair EncodingTheory
  4. LM2-T4Special Tokens and Unknown TextTheory
  5. LM2-T5Input-Target Pairs and Sliding WindowsTheory
  6. LM2-T6Batching, Data Loaders and Data SplitsTheory

Labs (3)

  1. LM2-L1Build a Simple TokenizerBrowser lab
  2. LM2-L2Implement Byte-Pair EncodingBrowser lab
  3. LM2-L3Diagnose Sliding Windows and BatchingBrowser lab

Module 3

Embeddings and the Mathematics of Attention

Token embeddings, positional information, queries, keys and values, scaled dot-product attention, causal masking, multi-head attention and tensor shapes.

You will be able to: explain why token IDs become vectors and how position is added; compute attention scores and weights step by step; apply a causal mask and explain what it prevents; implement multi-head causal self-attention in PyTorch and track its shapes.

All 9 items available

Theory (6)

  1. LM3-T1From Token IDs to Embedding VectorsTheory
  2. LM3-T2Positional InformationTheory
  3. LM3-T3Queries, Keys, Values and Scaled Dot-Product AttentionTheory
  4. LM3-T4Causal MaskingTheory
  5. LM3-T5Multi-Head AttentionTheory
  6. LM3-T6Tensor Shapes and Matrix Operations in AttentionTheory

Labs (3)

  1. LM3-L1Compute Attention Step by StepBrowser lab
  2. LM3-L2Implement Causal MaskingBrowser lab
  3. LM3-L3Implement Causal Multi-Head Attention in PyTorchPyTorch notebook

Module 4

Build a GPT Model from Scratch

The Transformer architecture, layer normalization, residual connections, feed-forward layers, the Transformer block, the output head, parameter counts and tensor shapes through the forward pass.

You will be able to: describe the GPT architecture and the role of every component; implement layer normalization, a feed-forward block and a Transformer block; assemble a GPT model and run a forward pass; count parameters and explain the tensor shape at each step.

All 8 items available

Theory (5)

  1. LM4-T1The GPT ArchitectureTheory
  2. LM4-T2Layer Normalization and Residual ConnectionsTheory
  3. LM4-T3Feed-Forward Layers and GELUTheory
  4. LM4-T4The Transformer Block and Assembling GPTTheory
  5. LM4-T5Output Logits, Parameter Counts and ShapesTheory

Labs (3)

  1. LM4-L1Count GPT Parameters and Tensor ShapesBrowser lab
  2. LM4-L2Layer Normalization and Residual NumericsBrowser lab
  3. LM4-L3Assemble a GPT Model in PyTorchPyTorch notebook

Module 5

Pretraining and Text Generation

The next-token objective, cross-entropy, gradient-based optimization, training loops, overfitting, learning rates, and decoding with temperature, top-k and top-p.

You will be able to: explain cross-entropy loss and perplexity for next-token prediction; describe backpropagation and AdamW at the level needed to run training; write a training loop with validation and checkpoints and diagnose a bad run; generate text with temperature, top-k and top-p and compare them.

All 9 items available

Theory (6)

  1. LM5-T1The Next-Token Objective and Cross-EntropyTheory
  2. LM5-T2Backpropagation and Gradient-Based OptimizationTheory
  3. LM5-T3Training Loops, Validation and CheckpointingTheory
  4. LM5-T4Overfitting, Underfitting and Data QualityTheory
  5. LM5-T5Temperature and SamplingTheory
  6. LM5-T6Top-k, Top-p and Stopping CriteriaTheory

Labs (3)

  1. LM5-L1Cross-Entropy and PerplexityBrowser lab
  2. LM5-L2Compare Sampling StrategiesBrowser lab
  3. LM5-L3Pretrain a Tiny GPT and Diagnose the RunPyTorch notebook

Module 6

Fine-Tuning for Classification

Adapting a pretrained language model to a labelled task with a classification head, frozen and trainable layers, metrics and failure analysis.

You will be able to: explain how supervised fine-tuning differs from pretraining; attach a classification head and choose the sequence representation; decide which parameters to freeze and which to train; evaluate a classifier and inspect its failures and data quality.

All 7 items available

Theory (5)

  1. LM6-T1Pretraining Versus Supervised Fine-TuningTheory
  2. LM6-T2Classification Heads and Sequence RepresentationsTheory
  3. LM6-T3Frozen and Trainable ParametersTheory
  4. LM6-T4Dataset Preparation, Labels and MetricsTheory
  5. LM6-T5Generalization and Failure AnalysisTheory

Labs (2)

  1. LM6-L1Classification Metrics and Failure CasesBrowser lab
  2. LM6-L2Fine-Tune a Language Model for ClassificationPyTorch notebook

Module 7

Instruction Fine-Tuning

Why pretraining does not produce instruction following, instruction datasets, prompt templates, special tokens, loss masking and evaluating responses.

You will be able to: explain why a base model does not follow instructions; format instruction-response data with a template and special tokens; mask the loss to the response tokens and justify it; evaluate instruction following against criteria and recognise contamination.

All 8 items available

Theory (5)

  1. LM7-T1Why Base Models Do Not Follow InstructionsTheory
  2. LM7-T2Instruction Datasets and Prompt FormatsTheory
  3. LM7-T3Special Tokens and Conversation TemplatesTheory
  4. LM7-T4Masking the LossTheory
  5. LM7-T5Evaluating Instruction Following and Data ContaminationTheory

Labs (3)

  1. LM7-L1Format Instruction Data with TemplatesBrowser lab
  2. LM7-L2Implement Loss MaskingBrowser lab
  3. LM7-L3Instruction-Tune a Tiny Model and Evaluate ItPyTorch notebook

Module 8

Efficient Fine-Tuning and Model Adaptation

Full versus parameter-efficient fine-tuning, LoRA, memory and compute estimates, quantization, and choosing a base model.

You will be able to: compare full fine-tuning with parameter-efficient methods; derive the LoRA update and count its trainable parameters; estimate training and inference memory; reason about quantization trade-offs and checkpoint management.

All 8 items available

Theory (5)

  1. LM8-T1Full Versus Parameter-Efficient Fine-TuningTheory
  2. LM8-T2Low-Rank Adaptation (LoRA)Theory
  3. LM8-T3Memory and Compute for TrainingTheory
  4. LM8-T4Quantization Concepts and Trade-OffsTheory
  5. LM8-T5Choosing a Base Model and Managing CheckpointsTheory

Labs (3)

  1. LM8-L1Estimate Trainable Parameters and MemoryBrowser lab
  2. LM8-L2LoRA Low-Rank Update NumericsBrowser lab
  3. LM8-L3LoRA Fine-Tuning ExperimentPyTorch notebook

Module 9

Evaluation, Inference and Reliability

Perplexity and its limits, task evaluation, sampling variability, hallucination, prompt sensitivity, reproducibility, latency, memory and responsible evaluation.

You will be able to: choose and compute metrics that match a task; measure the variability of a result and report uncertainty; identify hallucination and prompt-sensitivity failures; build a small repeatable evaluation harness and measure inference cost.

All 9 items available

Theory (6)

  1. LM9-T1Perplexity and Its LimitsTheory
  2. LM9-T2Task Evaluation and Test SetsTheory
  3. LM9-T3Sampling Variability and ReproducibilityTheory
  4. LM9-T4Hallucination and Prompt SensitivityTheory
  5. LM9-T5Inference Latency and MemoryTheory
  6. LM9-T6Responsible Evaluation and Model LimitationsTheory

Labs (3)

  1. LM9-L1Compare Two CheckpointsBrowser lab
  2. LM9-L2Build an Evaluation HarnessBrowser lab
  3. LM9-L3Evaluate a Tiny GPT End to EndPyTorch notebook

Module 10

Modern LLM Architecture and Inference

How modern decoder-only models differ from a minimal GPT: rotary embeddings, multi-query and grouped-query attention, gated feed-forward layers, mixture of experts, KV caching and inference efficiency.

You will be able to: list what changed between a minimal GPT and modern open models and why; explain rotary position embeddings, MQA/GQA and SwiGLU; estimate KV-cache memory and the cost of prefill versus decode; reason about architecture trade-offs for latency, memory and quality.

All 9 items available

Theory (6)

  1. LM10-T1Decoder-Only Models Then and NowTheory
  2. LM10-T2Rotary Position EmbeddingsTheory
  3. LM10-T3Multi-Query and Grouped-Query AttentionTheory
  4. LM10-T4SwiGLU, RMSNorm and Mixture of ExpertsTheory
  5. LM10-T5KV Caching, Prefill and DecodeTheory
  6. LM10-T6Inference Efficiency and Architecture Trade-OffsTheory

Labs (3)

  1. LM10-L1KV-Cache Memory and BehaviourBrowser lab
  2. LM10-L2Compare Attention VariantsBrowser lab
  3. LM10-L3Implement a KV Cache and Rotary Embeddings in PyTorchPyTorch notebook

Module 11

Building LLM Applications

Base versus instruction-tuned models, prompting and structured outputs, embeddings and retrieval, RAG, tool calling, application evaluation and security.

You will be able to: choose between base and instruction-tuned models and design prompts; build retrieval with embeddings, chunking and grounding; validate structured outputs and tool calls before acting; identify cost, latency and security risks including prompt injection.

All 9 items available

Theory (6)

  1. LM11-T1Base Versus Instruction-Tuned Models and Prompt DesignTheory
  2. LM11-T2Embeddings and Semantic SimilarityTheory
  3. LM11-T3Retrieval-Augmented GenerationTheory
  4. LM11-T4Tool Calling and Schema ValidationTheory
  5. LM11-T5Application Evaluation, Latency, Cost and ReliabilityTheory
  6. LM11-T6Security, Prompt Injection and Robot SafetyTheory

Labs (3)

  1. LM11-L1Chunking and RetrievalBrowser lab
  2. LM11-L2Validate Structured Outputs and Tool CallsBrowser lab
  3. LM11-L3Build a Retrieval Question-Answering PrototypePyTorch notebook

Module 12

Capstone: Build and Evaluate Your Own LLM System

A staged project: prepare data, build the tokenizer and pipeline, train or adapt a model, evaluate reproducibly, build a small application and write the report.

You will be able to: scope an LLM project to the compute available; run and document a reproducible experiment; evaluate honestly, including failure cases; write a report that states architecture, data, training, results and limits.

All 8 items available

Theory (4)

  1. LM12-T1Capstone Brief and Scoping for Limited ComputeTheory
  2. LM12-T2Experiment Design and ReproducibilityTheory
  3. LM12-T3Evaluating and Presenting Your SystemTheory
  4. LM12-T4Writing the Report, Limitations and Next StepsTheory

Labs (4)

  1. LM12-L1Capstone Milestone 1: Dataset, Tokenizer and PipelinePyTorch notebook
  2. LM12-L2Capstone Milestone 2: Model and Training ExperimentPyTorch notebook
  3. LM12-L3Capstone Milestone 3: Evaluation and ApplicationPyTorch notebook
  4. LM12-L4Capstone Milestone 4: Report ReviewBrowser lab