RL-401: Markov Decision Processes and Dynamic Programming

Course Description

Mathematical foundations of sequential decision-making. MDPs: states, actions, rewards, transition dynamics, discount factor. Bellman equations: optimality conditions for value and Q functions. Dynamic programming: policy evaluation, policy iteration, value iteration. Finite vs. infinite horizon problems. Partial observability (POMDPs). Grid world and inventory control case studies. Students implement all DP algorithms from scratch.