Structuring Machine Learning Projects

One number to optimize, a human yardstick to beat, and an error analysis that says what to do next.

Module 03.4 · Advanced
Free

9h · 12 lessons · 1 challenge

Before this: Optimization Algorithms & Tuning

About this module

This module is about the decisions around the model rather than the model. It opens where the source course does: orthogonalization, and choosing a single number to optimize so that two candidates can actually be compared.

Then human-level performance as the yardstick that turns error into avoidable bias, error analysis done by counting, what to do when training and deployment data differ, and when transfer or multi-task learning is the cheaper answer.

After this module you can

12 lessons

Lesson 1
1Orthogonalization: one knob, one effect
Kind
Concept
Length
40 min
Lesson 2
2A single number evaluation metric
Kind
Concept
Length
40 min
Lesson 3
3Satisficing and optimizing metrics
Kind
Concept
Length
40 min
Lesson 4
4Train, dev and test distributions and sizes
Kind
Concept
Length
45 min
Lesson 5
5Human-level performance and avoidable bias
Kind
Concept
Length
50 min
Lesson 6
6Error analysis, and cleaning mislabelled data
Kind
Concept
Length
50 min
Lesson 7
7Mismatched training and dev/test data
Kind
Concept
Length
50 min
Lesson 8
8Transfer learning and multi-task learning
Kind
Concept
Length
50 min
Lesson 9
9End-to-end deep learning: when it works
Kind
Concept
Length
45 min
Lesson 10
10Ceiling Check: how much is that fix worth?
Kind
Interactive
Length
30 min
Lesson 11
11Error analysis on a failing production model
Kind
Lab
Length
80 min
Lesson 12
12Checkpoint: rank the interventions
Kind
Checkpoint
Length
25 min

The challenge: Ceiling Check

Six proposed fixes and one error analysis. Say what each is worth before you are told, then see the ceiling it actually removes.

Worth 450 XP.

Related modules

All modules
Convolutional Networks
11h · 14 lessons
Word Embeddings
9h · 12 lessons