Robot Learning · Reinforcement Learning Foundations · M4-L1 · Intermediate
Solve an MDP by Value Iteration
Experiment log · Scout at Meridian General Hospital
Experiment log, Meridian General Hospital, east ward. Scout must carry a medication tray from the nurses' station to the pharmacy. The ward is a small grid: some cells are walls, one is the pharmacy, and the open stairwell has no railing. After the night shift cleaned the floors, they are slippery: about one move in five turns sideways.
The map and the slip are known, so Scout does not need to learn anything by trial and error. You will turn the transition table into the optimal value of every cell with value iteration, then read the best route off those values. On the way you will meet the two bugs that sink most first attempts: a terminal cell that leaks future reward, and a route chosen by looking only where Scout means to go.
Your task
Lab complete: post-mortem
You have passed every stage. Now connect it back to the theory and move on:
Output
What Python can I use?
Plain Python with math and heapq. Your code runs in a background worker: an endless loop is stopped after a few seconds and the page stays usable. It runs in your browser and is never sent anywhere.