From Markov decision processes and policy optimization to reasoning models, agents, and multimodal systems
Train on ModelScope · Run the CartPole Notebook · ModelScope Script
Learning Paths · Hands-On Examples · Book Features · About This Book · News · Contents · Book Structure · Experiment Code · Quick Start · Contributing
Classical & deep RL (Chapters 1–12). Start with CartPole: choose an action, observe the next state, and collect a reward. Build MDPs and value functions around that interaction, then implement DQN, policy gradients, Actor-Critic, and PPO in games and control tasks. Offline RL, imitation learning, exploration, and world models extend these foundations.
Modern RL & Agentic AI (Chapters 13–26). Apply the policy foundations to existing language models through RLHF, DPO, GRPO, and verifiable rewards. Agentic AI extends a model from answering once to calling tools, browsing, writing code, and completing multi-step tasks; the Agentic RL chapters train these decisions using feedback from entire trajectories. The later chapters cover visual and embodied tasks, training systems, and evaluation.
📣 Announcement
We sincerely thank everyone for your support of this tutorial! A new version is coming soon. Many sections are still being organized and refined, so we appreciate your patience. Suggestions and feedback are always welcome!
Note: This course was created with AI assistance and has not yet been fully reviewed. It may contain factual mistakes or code that does not run as expected. Issues and pull requests are very welcome.
- [2026-08-19] 🎮 Online Classic RL Environments and Scripts: Over the past two weeks, we have added and refined a collection of online reinforcement learning environments, training scripts, and companion notebooks. Learners can now run classic reinforcement learning experiments online, inspect training logs and evaluation results, and study the algorithms more conveniently. We also fixed many previously reported bugs in the course content, links, and experiment code.