Skip to content
This repository has been archived by the owner on Nov 8, 2022. It is now read-only.

New Transformer models, Quantized BERT, Distillation from BERT, Neural Taggers, PyTorch backend, CLI, procedures,

Compare
Choose a tag to compare
@peteriz peteriz released this 28 Aug 13:58
· 195 commits to master since this release

General features

  • Added PyTorch as a backend
  • New nlp_architect CLI for training/running/process models and scripts
  • New nlp_architect.procedures package for creating procedures/scripts for doing scripted work with NLP architect.
  • S3 URL caching of pre-trained models
  • Many additions/refactors/base classes reflecting new API (WIP)

Transformer models

  • Integrated pytorch-transformers library into NLP Architect's model package.
  • Created a base transformer class for training transformer-based models
  • Added sequence/token classification models

Optimization related models

  • Added a base transformer model for training quantized (8bit) BERT models
  • Added distillation process from Transformer models into CNN/LSTM based token classification models

NLP related additions

  • Added GLUE benchmark tasks
  • Added CNN-LSTM and ID-CNN token classification models (pytorch backend)

Documentation

  • Updated documentation website to reflect 0.5 release
  • Updated website with logo and matching style