This repository includes a number of deep learning models written by myself using the PyTorch deep learning framework.
The models are included in the modules folder, whereas saved trained models are in the models folder.
The script folder contains some readily available scripts to run to ensure the models work as expected.
This repository is currently a work in progress and more things will be added as I build them.
The currently available models are:
✅ Transformer: the transformer model (including the Encoder, the Decoder and all respective
sublayers) as described in the paper
"Attention is All You Need", Dec 2017, written
entirely in pytorch with no additional deep learning libraries.
:radio_button: Includes transformer training function for Machine
Translation tasks
✅ BERT for Question Answering Fine Tuning: full code to fine tune
a base BERT model for question answering tasks using the well-known
SQuAD v1.1 dataset, starting from a
transformers.BertForQuestionAnswering model (i.e. a general, non task-specific pre-trained BERT model).
The code includes all the data pre-processing steps, the training (fine-tuning) and validation of the model, as well
as the scores calculations based on the predictions on a test set. The code provides both CPU and cuda (GPU) support,
and a final chat-bot interface to answer questions given a context and the trained model.