Reinforcement Learning

Returns, policies, the Bellman equation and the deep Q-network that approximates it.

Module 02.9 · Intermediate
Free

8h · 10 lessons · 1 challenge

Before this: Recommender Systems

About this module

Reinforcement learning replaces the labelled example with a reward, and almost everything follows from that one change. This module builds the formalism: the return and its discount, policies, the state-action value function, and the Bellman equation that ties them together.

Then the practical half: stochastic environments, continuous state spaces, deep Q-learning, and the exploration problem that every one of them has.

After this module you can

10 lessons

Lesson 1
1The reinforcement learning formalism
Kind
Concept
Length
45 min
Lesson 2
2The return, and what the discount factor buys
Kind
Concept
Length
45 min
Lesson 3
3Policies and the state-action value function
Kind
Concept
Length
50 min
Lesson 4
4The Bellman equation
Kind
Concept
Length
50 min
Lesson 5
5Random environments and expected return
Kind
Concept
Length
40 min
Lesson 6
6Continuous state spaces
Kind
Concept
Length
45 min
Lesson 7
7Deep Q-learning and the epsilon-greedy policy
Kind
Concept
Length
50 min
Lesson 8
8Bandit Arena: explore against exploit
Kind
Interactive
Length
30 min
Lesson 9
9Train an agent, and watch what breaks
Kind
Lab
Length
90 min
Lesson 10
10Checkpoint: formalize the environment
Kind
Checkpoint
Length
25 min

The challenge: Bandit Arena

Ten arms, a thousand pulls and no second chance. Beat the optimal policy, or find out how far off you were.

Worth 400 XP.

← Recommender Systems

End of the track

Related modules

All modules