Implement and Train VLMs (Vision Language Models) From Scratch - PyTorch

In this video, we will build a Vision Language Model (VLM) from scratch, showing how a multimodal model combines computer vision and natural language processing for vision QA. GitHub below ↓

Want to support the channel? Hit that like button and subscribe!

GitHub Link of the Code
https://github.com/uygarkurt/VLM-PyTorch

Dataset
https://huggingface.co/dataset....s/uygarkurt/simple-i

Implement and Train ViT From Scratch for Image Recognition - PyTorch
https://youtu.be/Vonyoz6Yt9c

Implement Llama 3 From Scratch - PyTorch
https://youtu.be/lrWY4O5kUTY

Fine-Tune Visual Language Models (VLMs) - HuggingFace, PyTorch, LoRA, Quantization, TRL
https://youtu.be/3ypHZayanBI

ViT (Vision Transformer) - An Image Is Worth 16x16 Words (Paper Explained)
https://youtu.be/8phM16htKbU

What should I implement next? Let me know in the comments!

00:00 VLM Explanation
04:42 Downloading Dataset
06:39 Imports
10:09 Hyperparameters
12:37 Data Processing
16:03 ViT Definition
19:05 VLM Implementation
41:40 Sample Inference
45:20 Data Loaders
52:46 Training Loop
57:50 Inference

References
https://github.com/AviSoori1x/seemore

Buy me a coffee! ☕️
https://ko-fi.com/uygarkurt