How to Create a Neural Network (and Train it to Identify Doodles)

Exploring how neural networks learn by programming one from scratch in C#, and then attempting to teach it to recognize various doodles and images.

Source code: https://github.com/SebLague/Ne....ural-Network-Experim
Demo: https://sebastian.itch.io/neur....al-network-experimen

If you'd like to support me in creating more videos (and get early access to new stuff), you can join my patreon here: https://www.patreon.com/SebastianLague

The data is from:
Digits http://yann.lecun.com/exdb/mnist/
Fashion https://github.com/zalandoresearch/fashion-mnist
Doodles https://github.com/googlecreat....ivelab/quickdraw-dat
Cifar10 https://www.cs.toronto.edu/~kriz/cifar.html

Chapters
0:00 Introduction
2:39 The decision boundary
3:49 Weights
5:42 Biases
6:45 Hidden layers
7:45 Programming the network
9:57 Activation functions
12:42 Cost
15:07 Gradient descent example
18:22 The cost landscape
19:55 Programming gradient descent
21:10 It's learning! (slowly)
23:21 Calculus example
27:34 The chain rule
29:50 Some partial derivatives
33:14 Backpropagation
39:25 Digit recognition
43:56 Drawing our own digits
47:37 Fashion
48:25 Doodles
52:00 The final challenge

Music:
Cosmic Waves - Michael FK
Amber - The Stolen Orchestra
Beyond the Horizon - Sounds Like Sander
Air - Assaf Ayalon
Purest Form - Sounds Like Sander
Hear Wide Open - Sounds Like Sander
Universal Wonder - Moments
Roman P - Moments
All In Good Time - Shimmer
It Will Come Back - The Stolen Orchestra
Frontier - Shimmer
New Moon - Cloud Wave
Sunflower - Cody Martin
Inner Peace - Moments
Enchanted - Cody Martin
Just Around The Corner - Shimmer