Inspired by Karpathy's lecture on how to build a GPT from scratch: https://www.youtube.com/watch?v=kCc8FmEb1nY, this is the implementation of a transformer architecture from the Attention is All You Need paper: https://arxiv.org/abs/1706.03762 applied to the corpus of the Brasilian novelist Machado de Assis found in the NLTK library.
In the notebook example.ipynb you can import data from the NLTK library, instantiate a transformer form the train.py file and use helper functions from train.py and data folders to train a GPT to generate text in the style of Machado de Assis