Training a new tokenizer
0
0
3,577 Views
Published on 06/02/23 / In
How-to & Learning
Did you ever wonder how to create a BERT or GPT2 tokenizer in your own language or on your own corpus? This video will teach you how to do this with any tokenizer of the 🤗 Transformers library.
This video is part of the Hugging Face course: http://huggingface.co/course
Open in colab to run the code samples:
https://colab.research.google.....com/github/huggingfa
Related videos:
- Building a new tokenizer: https://youtu.be/MR8tZm5ViWU
Don't have a Hugging Face account? Join now: http://huggingface.co/join
Have a question? Checkout the forums: https://discuss.huggingface.co/c/course/20
Subscribe to our newsletter: https://huggingface.curated.co/
Show more
0 Comments
sort Sort By