Learning
Transformers
BERT
GPT
GPT-2
GPT-3
Attention is all you need
Deep Learning
NLP
Writing blog posts and emails can be tough at the best of times.
TBH, some days just writing anything can be a struggle
I mean, right now, I'm struggling to write this description 😅
There's got to be a better way right?
Well, there is. Using the amazing AI power of GPT2 and Python you can generate your own blog posts using a technique called Text Generation. This can be extended out to a whole heap of different use cases, it could be used to write emails, poems, code. You name it, you could probably do it.
In this case, we're focused on blog posts though. You'll be able to pass through a simple sentence and have a whole chunk of text output that you can then use on your blog!
In this video, you'll learn how to:
1. Setting up Hugging Face Transformers to use GPT2-Large
2. Loading the GPT2 Model and Tokenizer
3. Encoding text into token format
4. Generating text using the GPT2 Model
5. Decoding output to generate blog posts
Get the code: https://github.com/nicknochnac....k/Generating-Blog-Po
Chapters:
0:00 - Start
3:34 - Installing Hugging Face Transformers with Python
4:03 - Importing GPT2
5:23 - Loading the GPT2-Large Model and Tokenizer
8:39 - Tokenizing Sentences for AI Text Generation
10:57 - Generating Text using GPT2-Large
11:50 - Decoding Generated Text
14:13 - Outputting Results to .txt files
16:11 - Generating Longer Blog Posts
Oh, and don't forget to connect with me!
LinkedIn: https://www.linkedin.com/in/nicholasrenotte
Facebook: https://www.facebook.com/nickrenotte/
GitHub: https://github.com/nicknochnack
Patreon: https://www.patreon.com/nicholasrenotte
Join the Discussion on Discord: https://discord.gg/mtTTwYkB29
Happy coding!
Nick
P.s. Let me know how you go and drop a comment if you need a hand!
Basic mathematics from a language model? Rob Miles on GPT3, where it seems like size does matter!
More from Rob Miles: http://bit.ly/Rob_Miles_YouTube
https://www.facebook.com/computerphile
https://twitter.com/computer_phile
This video was filmed and edited by Sean Riley.
Computer Science at the University of Nottingham: https://bit.ly/nottscomputer
Computerphile is a sister project to Brady Haran's Numberphile. More at http://www.bradyharan.com
GPT-3 has demonstrated remarkable results in human-like text generation for a wide range of contexts. Check out the video to understand this transformer-based model under the hood.
Training some some short tests of the newly-minted generative python transformer model :D
Neural Networks from Scratch book: https://nnfs.io
Channel membership: https://www.youtube.com/channe....l/UCfzlCWGWYyIQ0aLC5
Discord: https://discord.gg/sentdex
Reddit: https://www.reddit.com/r/sentdex/
Support the content: https://pythonprogramming.net/support-donate/
Twitter: https://twitter.com/sentdex
Instagram: https://instagram.com/sentdex
Facebook: https://www.facebook.com/pythonprogramming.net/
Twitch: https://www.twitch.tv/sentdex
ERRATA:
In the "original transformer" (slide 51), in the source attention, the key and value come from the encoder, and the query comes from the decoder.
In this lecture we look at the details of some famous transformer models. How were they trained, and what could they do after they were trained.
slides: https://dlvu.github.io/slides/dlvu.lecture12.pdf
course website: https://dlvu.github.io
Lecturer: Peter Bloem
In this video, we are going to implement the GPT2 model from scratch. We are only going to focus on the inference and not on the training logic. We will cover concepts like self attention, decoder blocks and generating new tokens.
Paper: https://openai.com/blog/better-language-models/
Code minGPT: https://github.com/karpathy/minGPT
Code transformers: https://github.com/huggingface..../transformers/blob/0
Code from the video: https://github.com/jankrepl/mi....ldlyoverfitted/tree/
00:00 Intro
01:32 Overview: Main goal [slides]
02:06 Overview: Forward pass [slides]
03:39 Overview: GPT module (part 1) [slides]
04:28 Overview: GPT module (part 2) [slides]
05:25 Overview: Decoder block [slides]
06:10 Overview: Masked self attention [slides]
07:52 Decoder module [code]
13:40 GPT module [code]
18:19 Copying a tensor [code]
19:26 Copying a Decoder module [code]
21:04 Copying a GPT module [code]
22:13 Checking if copying works [code]
26:01 Generating token strategies [demo]
29:10 Generating a token function [code]
32:34 Script (copying + generating) [code]
35:59 Results: Running the script [demo]
40:50 Outro
If you have any video suggestions or you just wanna chat feel free to join the discord server: https://discord.gg/a8Va9tZsG5
Twitter: https://twitter.com/moverfitted
Credits logo animation
Title: Conjungation · Author: Uncle Milk · Source: https://soundcloud.com/unclemilk · License: https://creativecommons.org/licenses/... · Download (9MB): https://auboutdufil.com/?id=600
Learn more about Transformers → http://ibm.biz/ML-Transformers
Learn more about AI → http://ibm.biz/more-about-ai
Check out IBM Watson → http://ibm.biz/more-about-watson
Transformers? In this case, we're talking about a machine learning model, and in this video Martin Keen explains what transformers are, what they're good for, and maybe ... what they're not so good at for.
Download a free AI ebook → http://ibm.biz/ai-ebook-free
Read about the Journey to AI → http://ibm.biz/ai-journey-blog
Get started for free on IBM Cloud → http://ibm.biz/Bdf7QA
Subscribe to see more videos like this in the future → http://ibm.biz/subscribe-now
#AI #Software #ITModernization
Slides: https://sebastianraschka.com/p....df/lecture-notes/sta
-------
This video is part of my Introduction of Deep Learning course.
Next video: https://youtu.be/_BFp4kjSB-I
The complete playlist: https://www.youtube.com/playli....st?list=PLTKMiZHVd_2
A handy overview page with links to the materials: https://sebastianraschka.com/b....log/2021/dl-course.h
-------
If you want to be notified about future videos, please consider subscribing to my channel: https://youtube.com/c/SebastianRaschka
This video explains the original GPT model, "Improving Language Understanding by Generative Pre-Training". I think the key takeaways are understanding that they use a new unlabeled text dataset that requires the pre-training language modeling to incorporate longer range context, the way that they format input representations for supervised fine-tuning, and the different NLP tasks this is evaluated on!
Paper Links:
GPT: https://s3-us-west-2.amazonaws.....com/openai-assets/r
DeepMind "A new model and dataset for long range memory": https://deepmind.com/blog/arti....cle/A_new_model_and_
SQuAD: https://rajpurkar.github.io/SQuAD-explorer/explore/v2.0/dev/Oxygen.html?model=BiDAF%20+%20Self%20Attention%20+%20ELMo%20(single%20model)%20(Allen%20Institute%20for%20Artificial%20Intelligence%20[modified%20by%20Stanford])&version=v2.0
MultiNLI: https://www.nyu.edu/projects/bowman/multinli/
RACE: https://arxiv.org/pdf/1704.04683.pdf
Quora Question Pairs: https://www.quora.com/q/quorad....ata/First-Quora-Data
CoLA: https://arxiv.org/pdf/1805.12471.pdf
Thanks for watching! Please Subscribe!
Plausible text generation has been around for a couple of years, but how does it work - and what's next? Rob Miles on Language Models and Transformers.
More from Rob Miles: http://bit.ly/Rob_Miles_YouTube
AI YouTube Comments: https://youtu.be/XyMdpcAPnZc
Thanks to Nottingham Hackspace for providing the filming location: http://bit.ly/notthack
https://www.facebook.com/computerphile
https://twitter.com/computer_phile
This video was filmed and edited by Sean Riley.
Computer Science at the University of Nottingham: https://bit.ly/nottscomputer
Computerphile is a sister project to Brady Haran's Numberphile. More at http://www.bradyharan.com
Slides: https://sebastianraschka.com/p....df/lecture-notes/sta
-------
This video is part of my Introduction of Deep Learning course.
Next video: https://youtu.be/LOCzBgSV4tQ
The complete playlist: https://www.youtube.com/playli....st?list=PLTKMiZHVd_2
A handy overview page with links to the materials: https://sebastianraschka.com/b....log/2021/dl-course.h
-------
If you want to be notified about future videos, please consider subscribing to my channel: https://youtube.com/c/SebastianRaschka
Dale’s Blog → https://goo.gle/3xOeWoK
Classify text with BERT → https://goo.gle/3AUB431
Over the past five years, Transformers, a neural network architecture, have completely transformed state-of-the-art natural language processing. Want to translate text with machine learning? Curious how an ML model could write a poem or an op ed? Transformers can do it all. In this episode of Making with ML, Dale Markowitz explains what transformers are, how they work, and why they’re so impactful. Watch to learn how you can start using transformers in your app!
Chapters:
0:00 - Intro
0:51 - What are transformers?
3:18 - How do transformers work?
7:41 - How are transformers used?
8:35 - Getting started with transformers
Watch more episodes of Making with Machine Learning → https://goo.gle/2YysJRY
Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech
#MakingwithMachineLearning #MakingwithML
product: Cloud - General; fullname: Dale Markowitz; re_ty: Publish;
First Principles of Computer Vision is a lecture series presented by Shree Nayar who is faculty in the Computer Science Department, School of Engineering and Applied Sciences, Columbia University. Computer Vision is the enterprise of building machines that “see.” This series focuses on the physical and mathematical underpinnings of vision and has been designed for students, practitioners and enthusiasts who have no prior knowledge of computer vision.
Artificial Neural Networks 3D simulation.
Subscribe to this YouTube channel or connect on:
Web: https://www.cybercontrols.org/
LinkedIn: https://www.linkedin.com/in/de....nis-dmitriev-b6a9599
Support on Patreon: https://www.patreon.com/deep_robotics
Support on PayPal, user: [email protected]
MarI/O is a program made of neural networks and genetic algorithms that kicks butt at Super Mario World.
Source Code: http://pastebin.com/ZZmSNaHX
"NEAT" Paper: http://nn.cs.utexas.edu/downlo....ads/papers/stanley.e
Some relevant Wikipedia links:
https://en.wikipedia.org/wiki/Neuroevolution
https://en.wikipedia.org/wiki/....Evolutionary_algorit
https://en.wikipedia.org/wiki/....Artificial_neural_ne
BizHawk Emulator: http://tasvideos.org/BizHawk.html
SethBling Twitter: http://twitter.com/sethbling
SethBling Twitch: http://twitch.tv/sethbling
SethBling Facebook: http://facebook.com/sethbling
SethBling Website: http://sethbling.com
SethBling Shirts: http://sethbling.spreadshirt.com
Suggest Ideas: http://reddit.com/r/SethBlingSuggestions
Music at the end is Cipher by Kevin MacLeod
