The GPT3 model from OpenAI is a new AI system that is surprising the world by its ability. This is a gentle and visual look at how it works under the hood -- including how the model is trained, and how it calculates its predictions.


Introduction & GPT-3 Demos (0:00)
GPT-3 Inputs and Outputs (2:06)
Training the GPT-3 model (2:48)
The scale of GPT-3 and its 175 billion parameters (6:37)
The order of GPT-3 token processing (7:58)
"Deep" learning: looking inside a layer stack (9:00)
Input prompts and priming examples (11:00)
Fine-tuning: the best is yet to come (11:56)


Twitter: https://twitter.com/JayAlammar
Blog: https://jalammar.github.io/
Mailing List: https://jayalammar.substack.com/

More videos by Jay:
Jay's Visual Intro to AI
https://www.youtube.com/watch?v=mSTCzNgDJy4

Making Money from AI by Predicting Sales - Jay's Intro to AI Part 2
https://www.youtube.com/watch?v=V4-lXSs3jrk

Learn how to fine-tune GPT-3 without code in less than 10 minutes.
Create custom natural language models trained specifically for a use case you choose.

Introduction to Langchain Javascript Documentation. How to Create GPT-3 GPT-4 Chatbots that can contextually reference your data (txt, JSON, webpages, PDF) with embeddings . Brief Introduction into embeddings, vectorstorage options such as Pinecone, Chroma.

Consulting 1hr Call: https://cal.com/starmorphai/co....nsultingcall?duratio
Starmorph Website: https://Starmorph.com
NextJS Trained Bot: https://code.chat
Langchain JS Docs: https://js.langchain.com/docs/
Langchain Python Docs: https://langchain.readthedocs.io/en/latest/
OpenAI Embeddings Docs: https://platform.openai.com/do....cs/guides/embeddings
Zahid Open Source Langchain: https://github.com/zahidkhawaj....a/langchain-chat-nex
Instagram: https://instagram.com/starmorphdesign

This video will teach you how to use openai's gpt-2 to create a custom model for text generation. This allows you to generate text based on more specific information which can render better results when using the AI to create text.

If you missed the first video explaining how to launch gpt-2 on your local machine, here is the link: https://www.youtube.com/watch?v=UH3mCaB7Quc&t=168s

For more learning materials visit Alizarin9.com and subscribe to the channel to stay up to date with our content.

Want to build your own ChatGPT or learn the advanced ways to use Chat GPT? This video will be a more advance tutorial about OpenAI and its AI models, how to do prompts, generate content, and finally build an application based on its API.

⭐ New - Teach Me OpenAI Digital Book ⭐
https://enhanceui.gumroad.com/l/teach-me-openai
I've put together a easy to understand book with ~60 pages about OpenAI and GPT with how to get started to integrate it into projects, code examples, and much more!

Try the OpenAI Template - Starter Kit I've made
https://enhanceui.gumroad.com/....l/chatgpt-starter-te
This is new and ready for those looking to kickstart their own app or website using the OpenAI GPT models. I've built it on MERN it should save you over 100 hours if you want to build or test your own ideas! This also supports the channel if you have found these videos of benefit!

New - ChatGPT Starter Kit
https://enhanceui.gumroad.com/....l/chatgpt-template-s

00:00 - Introduction
00:55 - OpenAI Template Starter Kit
01:40 - Part 1. Prompt Engineering for ChatGPT & AI
02:29 - Part 1. Examples of Chat GPT Prompts
03:58 - Part 1. OpenAI Playground and Promps
04:53 - Part 1. Example 1 AI Q&A Prompt
06:45 - Part 1. Example 2 AI Airport Prompt
08:00 - Part 1. Example 3 AI Code Prompt
09:42 - Part 2. OpenAI Models and Uses
10:35 - Part 2. GPT3 Models Davinci Curie etc
11:54 - Part 2. Coding Models Codex, Crushman
12:41 - Part 2. OpenAI API Pricing and Tokens
13:44 - Part 2. Comparing AI Model and Outputs
15:18 - Part 2. Prompt variables for Temp, Length etc
16:55 - Part 2. OpenAI Fine-tuned models
17:26 - Part 2. Summary and Recap
17:47 - Part 3. Technical Chat GPT Introduction
18:00 - Part 3. Dependencies (NodeJS, CRA, API Keys, etc)
18:50 - Part 3. Create React Application
19:30 - Part 3. Express Web API
21:09 - Part 3. Frontend React JS Component setup
22:46 - Part 3. Backend Express OpenAI API setup
26:10 - Part 3. Fetch data from Client to Server to API (OpenAI)
29:44 - Part 3. Intro to Advanced Uses API
30:10 - Part 3. Example 1: Advanced API ChatGPT like Chatbot
35:21 - Part 3. Example 2: Advanced Terminal Chat GPT bot
36:41 - Conclusion

This video focused on the more advanced and technical aspects of GPT3 and its models. If you want a more beginner video on ChatGPT, I've you can access the following ones below:

1. What is ChatGPT
https://youtu.be/40Kp_fa8vIw

2. ChatGPT Tutorial for Beginners
https://youtu.be/JTxsNm9IdYU

3. How to use ChatGPT
https://youtu.be/3aV5-q8vRz8

#chatgpt #ai #openai

Discord: https://www.editorx.com/editor-x/codex

Real-time coding and exploring nanoGPT with me!


See detailed model explanation in Andrej Karpathy's legendary video (best GPT explanation on the internet): https://www.youtube.com/watch?v=kCc8FmEb1nY


🌼 About me 🌼
Sophia Yang is a Senior Data Scientist working at a tech company.


🔔 SUBSCRIBE to my channel: https://www.youtube.com/c/Soph....iaYangDS?sub_confirm


⭐ Stay in touch ⭐
📚 DS/ML Book Club: http://dsbookclub.github.io/
▶ YouTube: https://youtube.com/SophiaYangDS
✍️ Medium: https://sophiamyang.medium.com
🐦 Twitter: https://twitter.com/sophiamyang
🤝 Linkedin: https://www.linkedin.com/in/sophiamyang/
📧 Email: sophiayang.ds@gmail.com
💚 #datascience

Patreon: https://www.patreon.com/daveshap (now includes Discord!)
GATO Framework: https://www.gatoframework.org/

GitHub: https://github.com/daveshap
LinkedIn: https://www.linkedin.com/in/dave-shap-automator/
Twitter: https://twitter.com/d_shap_autom8r

Relevant Subreddits:
Artificial Sentience: https://www.reddit.com/r/ArtificialSentience/
Heuristic Imperatives: https://www.reddit.com/r/HeuristicImperatives/
Autonomous AI: https://www.reddit.com/r/Autonomous_AI/

DISCLAIMER: This video is not medical, financial, or legal advice. This is just my personal story and research findings. Always consult a licensed professional.

I work to better myself and the rest of humanity.

Learn how to train Chat GPT to write in your own style so it can produce on-brand blog posts, social media posts, video scripts and more.

✅ Join my Free Chat GPT Community: https://jasonwest.com/chatgptgroup

🔥 Ready to Master ChatGPT? Get the ultimate ChatGPT Course at: https://ChatGPTSkool.com

Tools
***************************
👉 Try Fast Funnels, perfect for split testing free for 14 days: https://FastFunnels.com/register

👉 Ultimate directory of chat GPT detectors: https://chatgptdetectors.com/

👉 Try chatGPT for free here: https://beta.openai.com/signup

SOLUTION TO BEAT AI DETECTORS
**********************************
👉 Guaranteed method: https://jasonwest.com/undetectable

DOWNLOAD
*************
⤵️ 10 Valuable Tips to Improve Your Funnels - https://jasonwest.com/funnel-tips

👉 Interested in starting your own 6 Figure Website and Funnel Agency? Learn more about my White Label Funnel Agency at: https://whitelabelfunnelagency.com/startagency/

*Affiliate Disclaimer:
Note this description may contain affiliate links that allow you to find the items mentioned in this video and support the channel at zero cost to you. Thank you for your support! 🙏🏻

#chatgpt #openai #jasonwest

In this video I'll be showing you how to create ChatGPT style custom knowledge chatbots using LlamaIndex/GPTIndex. You can create useful chatbot agents for your business by loading in your own data, eg for customer service chatbots or a range of other uses that LlamaIndex/GPTIndex make possible through their range of dataloaders on LlamaHub.

Important Links 🔗
👉🏻 My code: https://github.com/wombyz/custom-knowledge-chatbot
👉🏻 LlamaIndex: https://gpt-index.readthedocs.....io/en/latest/index.h

My AI Consulting Services 👋🏼
Want to speak with me about your next big project? Book here:
https://calendly.com/liamottley/consulting-call

Join My Newsletter! 🗞️
Get your fix of the latest AI news direct to your inbox:
https://morningsideai.beehiiv.com/

My AI Entrepreneurship Community 📈
Find top shelf co-founders for your next AI venture:
http://discord.gg/dAa78Zj7yj

My Links 🔗
👉🏻 Subscribe: https://www.youtube.com/channe....l/UCui4jxDaMb53Gdh-A
👉🏻 Twitter: https://twitter.com/liamottley_
👉🏻 LinkedIn: https://www.linkedin.com/in/liamottley/
👉🏻 Instagram: https://instagram.com/liamottley

Timestamps:
0:00 - Intro
0:27 - Why custom knowledge?
1:17 - Example (ChatPDF)
2:08 - LlamaIndex
5:30 - LlamaHub loaders
10:15 - Chatbot Example

About Me 👋🏼
Hi! My name is Liam Ottley and I’m an AI developer and entrepreneur, founder of my own AI development company: Morningside AI (https://morningside.ai). You’ve found my YouTube channel where I help other entrepreneur’s wrap their heads around AI with tutorials on how to get started with building AI apps!

With developments in AI moving so fast, I’m constantly working on my own projects and those of clients and sharing my learnings with my viewers 2-3x per week.

I don’t have anything to sell you, my only hope is that one day some of you will learn enough from my videos to see an exciting opportunity in the AI space and want to work with my development company to make it a reality.

Tutorial about building an AI with a custom knowledge base using OpenAI API, GPTIndex, and Langchain.

The technique was first described by Dan Shipper https://www.lennysnewsletter.c....om/p/i-built-a-lenny

Source code: https://colab.research.google.....com/drive/1PQXcM_jhN

High-converting accessible e-commerce template from Tech Foundation https://techfoundation.gumroad.....com/l/e-commerce_ki
Designed following 100+ insights from large-scale usability testing and research provided by such organizations as Baymard University and NNGroup.

Free UI design course: https://www.youtube.com/watch?v=1so9nqyjbYY&list=PLq_MSzhDeWSLK9SsQgQXzlQpgKg1iiZ01

#UXDesign #UX #Figma #UXUI

We build a Generatively Pretrained Transformer (GPT), following the paper "Attention is All You Need" and OpenAI's GPT-2 / GPT-3. We talk about connections to ChatGPT, which has taken the world by storm. We watch GitHub Copilot, itself a GPT, help us write a GPT (meta :D!) . I recommend people watch the earlier makemore videos to get comfortable with the autoregressive language modeling framework and basics of tensors and PyTorch nn, which we take for granted in this video.

Links:
- Google colab for the video: https://colab.research.google.....com/drive/1JMLa53HDu
- GitHub repo for the video: https://github.com/karpathy/ng-video-lecture
- Playlist of the whole Zero to Hero series so far: https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ
- nanoGPT repo: https://github.com/karpathy/nanoGPT
- my website: https://karpathy.ai
- my twitter: https://twitter.com/karpathy
- our Discord channel: https://discord.gg/3zy8kqD9Cp

Supplementary links:
- Attention is All You Need paper: https://arxiv.org/abs/1706.03762
- OpenAI GPT-3 paper: https://arxiv.org/abs/2005.14165
- OpenAI ChatGPT blog post: https://openai.com/blog/chatgpt/
- The GPU I'm training the model on is from Lambda GPU Cloud, I think the best and easiest way to spin up an on-demand GPU instance in the cloud that you can ssh to: https://lambdalabs.com . If you prefer to work in notebooks, I think the easiest path today is Google Colab.

Suggested exercises:
- EX1: The n-dimensional tensor mastery challenge: Combine the `Head` and `MultiHeadAttention` into one class that processes all the heads in parallel, treating the heads as another batch dimension (answer is in nanoGPT).
- EX2: Train the GPT on your own dataset of choice! What other data could be fun to blabber on about? (A fun advanced suggestion if you like: train a GPT to do addition of two numbers, i.e. a+b=c. You may find it helpful to predict the digits of c in reverse order, as the typical addition algorithm (that you're hoping it learns) would proceed right to left too. You may want to modify the data loader to simply serve random problems and skip the generation of train.bin, val.bin. You may want to mask out the loss at the input positions of a+b that just specify the problem using y=-1 in the targets (see CrossEntropyLoss ignore_index). Does your Transformer learn to add? Once you have this, swole doge project: build a calculator clone in GPT, for all of +-*/. Not an easy problem. You may need Chain of Thought traces.)
- EX3: Find a dataset that is very large, so large that you can't see a gap between train and val loss. Pretrain the transformer on this data, then initialize with that model and finetune it on tiny shakespeare with a smaller number of steps and lower learning rate. Can you obtain a lower validation loss by the use of pretraining?
- EX4: Read some transformer papers and implement one additional feature or change that people seem to use. Does it improve the performance of your GPT?

Chapters:
00:00:00 intro: ChatGPT, Transformers, nanoGPT, Shakespeare
baseline language modeling, code setup
00:07:52 reading and exploring the data
00:09:28 tokenization, train/val split
00:14:27 data loader: batches of chunks of data
00:22:11 simplest baseline: bigram language model, loss, generation
00:34:53 training the bigram model
00:38:00 port our code to a script
Building the "self-attention"
00:42:13 version 1: averaging past context with for loops, the weakest form of aggregation
00:47:11 the trick in self-attention: matrix multiply as weighted aggregation
00:51:54 version 2: using matrix multiply
00:54:42 version 3: adding softmax
00:58:26 minor code cleanup
01:00:18 positional encoding
01:02:00 THE CRUX OF THE VIDEO: version 4: self-attention
01:11:38 note 1: attention as communication
01:12:46 note 2: attention has no notion of space, operates over sets
01:13:40 note 3: there is no communication across batch dimension
01:14:14 note 4: encoder blocks vs. decoder blocks
01:15:39 note 5: attention vs. self-attention vs. cross-attention
01:16:56 note 6: "scaled" self-attention. why divide by sqrt(head_size)
Building the Transformer
01:19:11 inserting a single self-attention block to our network
01:21:59 multi-headed self-attention
01:24:25 feedforward layers of transformer block
01:26:48 residual connections
01:32:51 layernorm (and its relationship to our previous batchnorm)
01:37:49 scaling up the model! creating a few variables. adding dropout
Notes on Transformer
01:42:39 encoder vs. decoder vs. both (?) Transformers
01:46:22 super quick walkthrough of nanoGPT, batched multi-headed self-attention
01:48:53 back to ChatGPT, GPT-3, pretraining vs. finetuning, RLHF
01:54:32 conclusions

Corrections:
00:57:00 Oops "tokens from the _future_ cannot communicate", not "past". Sorry! :)
01:20:05 Oops I should be using the head_size for the normalization, not C

This short tutorial explains the training objectives used to develop ChatGPT, the new chatbot language model from OpenAI.

Timestamps:
0:00 - Non-intro
0:24 - Training overview
1:33 - Generative pretraining (the raw language model)
4:18 - The alignment problem
6:26 - Supervised fine-tuning
7:19 - Limitations of supervision: distributional shift
8:50 - Reward learning based on preferences
10:39 - Reinforcement learning from human feedback
13:02 - Room for improvement

ChatGPT: https://openai.com/blog/chatgpt

Relevant papers for learning more:
InstructGPT: Ouyang et al., 2022 - https://arxiv.org/abs/2203.02155
GPT-3: Brown et al., 2020 - https://arxiv.org/abs/2005.14165
PaLM: Chowdhery et al., 2022 - https://arxiv.org/abs/2204.02311
Efficient reductions for imitation learning: Ross & Bagnell, 2010 - https://proceedings.mlr.press/v9/ross10a.html
Deep reinforcement learning from human preferences: Christiano et al., 2017 - https://arxiv.org/abs/1706.03741
Learning to summarize from human feedback: Stiennon et al., 2020 - https://arxiv.org/abs/2009.01325
Scaling laws for reward model overoptimization: Gao et al., 2022 - https://arxiv.org/abs/2210.10760
Proximal policy optimization algorithms: Schulman et al., 2017 - https://arxiv.org/abs/1707.06347

Special thanks to Elmira Amirloo for feedback on this video.

Links:
YouTube: https://www.youtube.com/ariseffai
Twitter: https://twitter.com/ari_seff
Homepage: https://www.ariseff.com

If you'd like to help support the channel (completely optional), you can donate a cup of coffee via the following:
Venmo: https://venmo.com/ariseff
PayPal: https://www.paypal.me/ariseff

#chatgpt #artificialintelligence #openai
Super simple guide on How to Fine Tune ChatGPT, in a Beginners Guide to Building Businesses w/ GPT-3. Knowing how to Fine Tune GPT-3 is one of the most important skills you can have as an entrepreneur to build your own ChatGPT site or get into the AI industry and build a valuable business business.

While limited in scope, this video gives you a clear breakdown of the stages in fine tuning these models.

Kaggle Dataset:
https://www.kaggle.com/dataset....s/thedevastator/unlo

Prompts & Code HERE:
https://docs.google.com/docume....nt/d/1jJbk61grxYZV2q

My AI Consulting Services 👋🏼
Want to speak with me about your next big project? Book here:
https://calendly.com/liamottley/consulting-call

Join My Newsletter! 🗞️
Get your fix of the latest AI news direct to your inbox:
https://morningsideai.beehiiv.com/

My AI Entrepreneurship Community 📈
Find top shelf co-founders for your next AI venture:
http://discord.gg/dAa78Zj7yj

My Links 🔗
👉🏻 Subscribe: https://www.youtube.com/channe....l/UCui4jxDaMb53Gdh-A
👉🏻 Twitter: https://twitter.com/liamottley_
👉🏻 LinkedIn: https://www.linkedin.com/in/liamottley/
👉🏻 Instagram: https://instagram.com/liamottley

Timestamps:
0:00 - Intro
1:07 - Generating Pairs
7:11 - Fine Tuning w/ API
10:18 - Running Our App
12:38 - Outro

About Me 👋🏼
Hi! My name is Liam Ottley and I’m an AI developer and entrepreneur, founder of my own AI development company: Morningside AI (https://morningside.ai). You’ve found my YouTube channel where I help other entrepreneur’s wrap their heads around AI with tutorials on how to get started with building AI apps!

With developments in AI moving so fast, I’m constantly working on my own projects and those of clients and sharing my learnings with my viewers 2-3x per week.

I don’t have anything to sell you, my only hope is that one day some of you will learn enough from my videos to see an exciting opportunity in the AI space and want to work with my development company to make it a reality.

In this video, we're going to go over how to fine-tune a GPT-3 model.

Become a member here:
https://www.youtube.com/c/AllAboutAI/join

Join the newsletter here:
https://www.allabtai.com/newsletter/

We'll start by creating a prompt, then we'll generate some synthetic data to use for training. Next, we'll augment that data to make it even more useful for training.

Finally, we'll prepare a JSON file for uploading to OpenAI GPT-3. Once our model is fine-tuned, we'll test it out to see how it performs.


00:00 Intro
00:26 Fine Tune GPT-3 Model Plan
03:07 Creating a prompt for fine-tuning GPT-3
08:22 Syntetic Data for fine-tuning GPT-3
13:13 Augmenting the Data for fine-tuning
15:46 Preparing a json file for fine-tuning
19:06 Uploading json file to OpenAI GPT-3
20:29 Testing our fine-tuned GPT-3 model
26:20 Recap of GPT-3 fine tuning

I show you how to train ChatGPT on your own custom data to create your own customisable GPT-4 powered chatbot you can use for your businesses Website or employees. This is the easiest no-code method I have found, and I've tried many different types of chatbot software!

👉 Custom Data Chatbot software: https://jasonwest.com/customgpt

👉 Sitemap generator to feed the required data into the Chatbot: https://adorosario-customgpt-s....treaml-build-sitemap

✅ Get free prompts in my free Chat GPT Community: https://jasonwest.com/chatgptgroup

🔥 Ready to Master ChatGPT? Get the No1 ChatGPT Course at: https://ChatGPTSkool.com

Tools & Links
***************************
👉 Ultimate directory of chat GPT detectors: https://ChatGPTDetectors.com/

👉 Try Fast Funnels website and funnel building software with built-in AI free for 14 days: https://FastFunnels.com/register

SOLUTION TO BEAT AI CONTENT DETECTORS
************************************************
👉 Guaranteed method: https://jasonwest.com/undetectable

DOWNLOAD
*************
⤵️ 10 Valuable Tips to Improve Your Funnels - https://jasonwest.com/funnel-tips

👉 Interested in starting your own 6 Figure Website and Funnel Agency like mine? Learn more about my White Label Funnel Agency at: https://whitelabelfunnelagency.com/startagency/

*Affiliate Disclaimer:
Note this description may contain affiliate links that allow you to find the items mentioned in this video and support the channel at zero cost to you. Thank you for your support! 🙏🏻

#chatgpt #chatbot #jasonwest

A quick walkthrough of training a fine-tuned model on gpt-3 using the openai cli.

In this video I train a fine-tuned gpt-3 model on Radiohead lyrics so that it will generate a Radiohead song form the first line.

Documentation: https://beta.openai.com/docs/guides/fine-tuning