Dutch GPT-2 Efficient Transformers

Thanks to ML6 for virtually hosting us tonight! For those who would like to attend live, have a look at https://www.meetup.com/ai-camp....us-berlin/events/282

Talk 1

Title: How we trained our own Dutch GPT-2 using Transformers

Speaker: Thomas Vrancken

Abstract:
Text Generation and the GPT series of Transformer models have been a hot topic since the public got to know the astounding power of it. The latest GPT-3 can mimic a human conversation to a close to scary level.
At ML6 we trained and open sourced our own Dutch GPT-2 model using Huggingface’s Transformers library. This talk addresses the questions:
How do you do that? What kind of data do you need and how to access enough compute power to actually train the model?

Bio:
Thomas Vrancken is an ML Engineer at ML6 with a background in strategy consulting and research. Thomas is passionate about NLP, data science and making real world impacts with creative Machine Learning applications. Staying versatile is his credo, joining a maximum of events with interesting talks is a means of achieving it.

Talk 2

Title: Efficient Transformers

Speaker: Mats Uytterhoeven

Abstract:
In recent years we’ve seen an exponential increase in the size of pre-trained transformer based models and although they push the state-of-the-art to ever greater heights, they also become increasingly cumbersome to work with. This has prompted researchers around the world to try and find more efficient alternatives to the classic transformer architecture and has spawned an interesting new research direction. In this talk, we will have a look at some of the interesting ideas in this area and what the future may hold for these transformer based models.

Bio:
Mats Uytterhoeven is an ML Engineer at ML6 interested in a broad range of topics. His main focus is on NLP and unsupervised learning problems. When he's not hacking on machine learning code, he likes playing tennis, reading (non-fiction), and traveling. He believes machine learning can have a positive impact on people's lives and loves working on projects that can make a difference.