Top videos

Proximal Policy Optimization (PPO) - How to train Large Language Models

Generative AI
5 Views · 5 months ago

Reinforcement Learning with Human Feedback (RLHF) is a method used for training Large Language Models (LLMs). In the heart of RLHF lies a very powerful reinforcement learning method called Proximal Policy Optimization. Learn about it in this simple video!

This is the first one in a series of 3 videos dedicated to the reinforcement learning methods used for training LLMs.

Full Playlist: https://www.youtube.com/playli....st?list=PLs8w1Cdi-zv

Video 0 (Optional): Introduction to deep reinforcement learning https://www.youtube.com/watch?v=SgC6AZss478
Video 1 (This one): Proximal Policy Optimization
Video 2: Reinforcement Learning with Human Feedback https://www.youtube.com/watch?v=Z_JUqJBpVOk
Video 3 (Coming soon!): Deterministic Policy Optimization

00:00 Introduction
01:25 Gridworld
03:10 States and Action
04:01 Values
07:30 Policy
09:39 Neural Networks
16:14 Training the value neural network (Gain)
22:50 Training the policy neural network (Surrogate Objective Function)
33:38 Clipping the surrogate objective function
36:49 Summary

Get the Grokking Machine Learning book!
https://manning.com/books/grok....king-machine-learnin
Discount code (40%): serranoyt
(Use the discount code on checkout)

Latent Dirichlet Allocation (Part 1 of 2)

Generative AI
5 Views · 5 months ago

Latent Dirichlet Allocation is a powerful machine learning technique used to sort documents by topic. Learn all about it in this video!

This is part 1 of a 2 video series.
Video 2: https://www.youtube.com/watch?v=BaM1uiCpj_E

For information on my book "Grokking Machine Learning": https://www.manning.com/books/....grokking-machine-lea
Discount code (40%): serranoyt

0:00 Introduction
0:11 The problem
3:47 Machine that generates documents
5:13 Blueprint for the LDA machine
5:49 Probability of a document
9:22 Quiz: Which one for topics?
11:06 A distribution of distributions
11:30 More topics? More dimensions
12:20 In More dimensions
12:55 Quiz: Where to put the topics?
14:05 Two Dirichlet distributions
14:22 Latent Dirichlet Allocation
22:05 Best settings on the machine
22:37 The winning arrangements
25:28 Series of two videos
25:48 Acknowledgements
26:30 Thank you!

Why do we divide by n-1 to estimate the variance? A visual tour through Bessel correction

Generative AI
5 Views · 5 months ago

Correction: At 30:42 I write "X = Y". They're not equal, what I meant to say is "X and Y are identically distributed".

The variance is a measure of how spread out a distribution is. In order to estimate the variance, one takes a sample of n points from the distribution, and calculate the average square deviation from the mean.
However, this doesn't give a good estimate of the variance of the distribution. The best estimate, however, is obtained when dividing by n-1 instead of n.
WHY!?!?!?!?!?!?!?
In this video, we dig deeper into why the variance calculation should be divided by n-1 instead of by n. For this, we use an alternate definition of the variance, which doesn't use the mean in its calculation.

*[0:00] Introduction and Bessel's Correction*
- Introducing Bessel's Correction and why we divide by $ n-1 $ instead of $ n $ to estimate variance.

*[0:12] Introduction to Variance Calculation*
- Explaining the premise of calculating variance and introducing the concept of estimating variance using a sample instead of the entire population.

*[1:01] Definition of Variance*
- Defining variance as a measure of how much values deviate from the mean and outlining the basic steps of variance calculation.

*[1:52] Introduction to Bessel's Correction*
- Discussing why we divide by $ n-1 $ when calculating variance and introducing Bessel's Correction.

*[2:35] Challenges of Bessel's Correction*
- Sharing personal challenges in understanding the rationale behind Bessel's Correction and discussing my research process on the topic.

*[3:20] Alternative Definition of Variance*
- Presenting an alternative definition of variance to aid in understanding Bessel's Correction and expressing curiosity about its presence in the literature.

*[4:45] Quick Recap of Mean and Variance*
- Briefly revisiting the concepts of mean and variance, demonstrating how they are calculated with examples, and explaining how variance reflects different distributions.

*[7:05] Sample Mean and Variance Estimation*
- Explaining the challenges of estimating the mean and variance of a distribution using a sample and discussing why sample variance is not a good estimate.

*[8:49] Bessel's Correction and Why $ n-1 $ is Used*
- Explaining how Bessel's Correction provides a better estimate of variance and why we divide by $ n-1 $ instead of $ n $. Emphasizing the importance of making a correct variance estimate.

*[10:51] Why Better Estimation Matters?*
- Discussing why the original estimate is poor and why making a better estimate is crucial. Explaining the significance of sample mean as a good estimate.

*[13:02] Issues with Variance Estimation*
- Illustrating the problems with variance estimation and demonstrating with examples why using the correct mean is essential for accurate estimates. Explaining the accuracy of estimates made using $ n-1 $.

*[15:04] Introduction to Correcting the Estimate*
- Discussing the underestimated variance and the need for correction in estimation.

*[15:57] Adjusting the Variance Formula*
- Explaining the adjustment in the variance formula by changing the denominator from $ n $ to $ n - 1 $.

*[16:22] Calculation Illustration*
- Demonstrating the calculation process of variance with the adjusted formula using examples.

*[16:57] Better Estimate with Bessel's Correction*
- Discussing how the corrected estimate provides a more accurate variance estimation.

*[18:24] New Method for Variance Calculation*
- Introducing a new method for calculating variance without explicitly calculating the mean.

*[20:06] Understanding the Relation between Variance and Variance*
- Explaining the relationship between variance and variance, and how they are related mathematically.

*[21:52] Demonstrating a Bad Calculation*
- Illustrating a flawed method for calculating variance and explaining the need for correction.

*[23:37] The Role of Bessel's Correction*
- Explaining why removing unnecessary zeros in variance calculation leads to better estimates, equivalent to Bessel's Correction.

*[25:08] Summary of Estimation Methods*
- Summarizing the difference between the flawed and corrected estimation methods for variance.

*[26:02] Importance of Bessel's Correction*
- Emphasizing the significance of Bessel's Correction for accurate variance estimation, especially with smaller sample sizes.

*[30:19] Mathematical Proof of Variance Relationship*
- Providing two proofs of the relationship between variance and variance, highlighting their equivalence.

*[35:24] Acknowledgments and Conclusion*

Thanks @mkan543 for the summary!

Decision trees - A friendly introduction

Generative AI
5 Views · 5 months ago

A video about decision trees, and how to train them on a simple example.

Accompanying blog post: https://medium.com/@luis.serra....no/splitting-data-by

For a code implementation, check out this repo:
https://github.com/luisguiserr....ano/manning/tree/mas

Helper videos:
- Gini index: https://www.youtube.com/watch?v=u4IxOk2ijSs
- Entropy and information gain: https://www.youtube.com/watch?v=9r7FIXEAGvs
- Machine learning error and metrics: https://www.youtube.com/watch?v=aDW44NPhNw0

Grokking Machine Learning book:
www.manning.com/books/grokking-machine-learning
40% discount code: serranoyt

This Tiny Model is Insane... (7m Parameters)

Generative AI
5 Views · 5 months ago

Build your first app today with Mocha: https://www.getmocha.com?utm_source=matthew_berman

Download Humanities Last Prompt Engineering Guide (free) 👇🏼
https://bit.ly/4kFhajz

Download The Matthew Berman Vibe Coding Playbook (free) 👇🏼
https://bit.ly/3I2J0YQ

Join My Newsletter for Regular AI Updates 👇🏼
https://forwardfuture.ai

Discover The Best AI Tools👇🏼
https://tools.forwardfuture.ai

My Links 🔗
👉🏻 X: https://x.com/matthewberman
👉🏻 Forward Future X: https://x.com/forward_future_
👉🏻 Instagram: https://www.instagram.com/matthewberman_ai
👉🏻 Discord: https://discord.gg/xxysSXBxFW
👉🏻 TikTok: https://www.tiktok.com/@matthewberman_ai

Media/Sponsorship Inquiries ✅
https://bit.ly/44TC45V

Links:
https://arxiv.org/html/2510.04871v1

What is Small Language Model? | Need for SLMs | GPTs in Laptop | Offline Language Models

Generative AI
5 Views · 5 months ago

This video covers about Small Language Models with a cast study

⏱ Chapter Timestamps
====================
00:00 - Intro
00:28 - What is Small Language Model
02:31 - Need for SLMs?
03:00 - Case Study: Offline Applications
05:06 - SLMs in the market

Join this channel by contributing to the community:
https://www.youtube.com/channe....l/UCB12jjYsYv-eipCvB

📌 Related Links
=============
🔗CoPilot+PCs - https://www.youtube.com/watch?v=5JmkWJNng2I

📌 Related Playlist
================
🔗 AI Primer Playlist - https://www.youtube.com/playli....st?list=PLTyWtrsGknY
🔗Spring Boot Primer - https://www.youtube.com/playli....st?list=PLTyWtrsGknY
🔗Spring Cloud Primer - https://www.youtube.com/playli....st?list=PLTyWtrsGknY
🔗Spring Microservices Primer - https://www.youtube.com/playli....st?list=PLTyWtrsGknY
🔗Spring JPA Primer - https://www.youtube.com/playli....st?list=PLTyWtrsGknY
🔗Java 8 Streams - https://www.youtube.com/playli....st?list=PLTyWtrsGknY
🔗Spring Security Primer - https://www.youtube.com/playli....st?list=PLTyWtrsGknY

💪 Join TechPrimers Slack Community: https://bit.ly/JoinTechPrimers
📟 Telegram: https://t.me/TechPrimers
🧮 TechPrimer HindSight (Blog): https://medium.com/TechPrimers
☁️ Website: http://techprimers.com
💪 Slack Community: https://techprimers.slack.com
🐦 Twitter: https://twitter.com/TechPrimers
📱 Facebook: http://fb.me/TechPrimers
💻 GitHub: https://github.com/TechPrimers or https://techprimers.github.io/

🎬 Video Editing: FCP

---------------------------------------------------------------
🔥 Disclaimer/Policy:
The content/views/opinions posted here are solely mine and the code samples created by me are open sourced.
You are free to use the code samples in Github after forking and you can modify it for your own use.
All the videos posted here are copyrighted. You cannot re-distribute videos on this channel in other channels or platforms.
#SmallLanguageModel #LLMs #RAG