Building BERT with PyTorch from scratch

This is the repository containing the code for a tutorial

Building BERT with PyTorch from scratch

Installation

After you clone the repository and setup virtual environment, install dependencies

pip install -r requirements.txt

Installation on Mac M1

You may experience difficulties installing tensorboard. Tensorboard requires grpcio that should be installed with extra environment variables. Read more in StackOverflow.

So, your installation line for Mac M1 should look like

export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

pip install -r requirements.txt