Data Science Texts

Discover what you don't know, and attack your weaknesses!

Reinforcement Learning

Strongly Recommended Prerequisites

Recommended Prerequisites

Reinforcement learning is similar to supervised learning, but it has two major differences that are perhaps best illustrated by a canonical problem in reinforcement learning: chess. If you wanted to formulate chess as a supervised learning problem, you would collect a large set of board positions and the best possible move from each board position, and then you would train your learner based on that data. The problem with this approach is that the best move will generally not be known except in very simple situations. The other major difference is that the board positions a real automated chess player will encounter are determined by the choices of that player, rather than, for instance, a set of chess positions from human games. Thus, reinforcement learning differs from supervised learning in that there aren't labels for every input and the inputs depend on the learning process.

Recommended Books

  1. Reinforcement Learning: An Introduction

    Richard S. Sutton and Andrew G. Barto

    (Image takes you to Amazon.)

    This is a great introduction to reinforcement learning. It's very applied, so it should be approachable for all data scientists with a modest probability background. The authors provide Python code through their website for many of the figures which you can experiment with. The new edition contains up-to-date examples of reinforcement learning that have been prominent in the news. If you're curious about reinforcement learning, this is both the canonical text and a great place to get started.

  2. Decision Making Under Uncertainty

    Mykel J. Kochenderfer

    (Image takes you to Amazon.)

    This is a great book if you want to learn about probabilistic decision making in general. It's gentler than the top recommendation, and reinforcement learning is not its exclusive focus. Note that there are exercises and solutions to the exercises available at the link above.