N2D2 1.3.0

💡 Highlights

We are delighted to annonce the new release of N2D2, this version introduces the addition of the SAT method and the STM32 export to the open-source version of N2D2 as well as some bug fixes in the Python API.
We would like to thank everyone who contributed to this release!

• The quantization method SAT is now open source: The SAT method is now available with both the INI and Python API. To get started on this feature, you can follow tutorials available in the documentation;
• Various improvement for the Python API: We have provided various improvement to the Python API, focusing on better handling transformations, interoperability with Keras and Pytorch and better handling of the exports;
• Our STM32 export is now available for everyone: You can now export your model for the STM32 boards. More details on the documentation
• Pruning module added: we have added a pruning module in order to use this method in N2D2 to compress models. This addition completes the list of compression techniques already present in the framework with the quantization

The rate of the update on N2D2 is decreasing as we are working on an exciting complete refont of the software. 💪

More news coming soon ! 🤗

🐛 Bug Fixes

Python API

• Fixed infinite loop error in summary method after using DeepNetCell.remove();
• Fixed Torch to N2D2 conversion in the case of a view tensor;
• Fixed pip install method for machine with one processor;
• Fixed an error with the way we saved BatchNorm statistics. Now, statistics are not saved, instead, we temporaly update the BatchNorm propagate to avoid BatchNorm fuse and statitics updates.
• Fixed a bug with the try of quantizing the Reshape layer during PTQ.
• Fixed extraction process for gz files in the installation dataset script

Core C++

• Fixed Mapping and Group handling on CPU for customized Mapping;
• Fixed a bug which could occur during the compilation on 32-bit processors
• Added a Scheduler class to implement local scheduling
• Fixed a bug during CPP export generation

⚙️ Refactor

Python API

• Torch Interrop now uses ONNX simplifier;
• Added function n2d2.deepnet.associate_provider_to_deepnet to reduce code redundancy in export.py and quantization.py;
• Fixed typo in Pool and Scaling is_exportable_to method;
• Added option to disable memory optimizations for the C++ export.

Exports

• Changed the path of the calibration folder (now generated into the statistics folder)

🚀 New features

Python API

• Torch and Keras Interrop wrap method now allow to set the opset version used to generate the ONNX;
• Export now check if dataprovider contains data, otherwise prints a warning;
• Added analysis tools to DeepNetCell;
• Added set_provider to DeepNetCell;
• Updated data_augmentation.py example with a data_path parameter;
• Added a method in Database to download data (not available for every database);
• Updated converter.from_N2D2 to support conversion of C++ transformation to python;
• Added DataProvider.get_transformations method;
• Added DataProvider.normalize_stimuli to normalize stimuli between [0,1];
• Added Reshape transformation to the Python API.
• Added SATCell object to quantize your layers with the SAT method;
• Added PruneCell object to prune the weights in the associated layer;

Exports

• Added the STM32 export (available versions for STM32H7 and STM32L4)

N2D2 1.2.0

💡 Highlights

We are delighted to annonce the new release of N2D2 with exciting new features and with a little tidying up in the codebase 🧹
We would like to thank everyone who contributed to this release!

• Improvement and bug fixes in the Python API: the API continues to become more and more interesting to use. We integrated multiple features such as a summary function and fixed some bugs in this release. Don't hesitate to give us your feedback about it !
• Change the name of the interoperabilities: you can now use our interop with the new names pytorch_to_n2d2 and keras_to_n2d2.
• Fixed bugs in the generation of the N2D2 exports and improvements of the user experience: some bugs have been fixed in the export code as such as in the structure of the exports. Test them to tell what you think of the improvements.

Moreover, more features are already planned to be added in the next releases like other QAT techniques (the SAT technique for example).
So don't go too far !

🐛 Bug Fixes

Python API

• Fixed a bug when N2D2 is not compiled with lib json cpp, the python API would fail to load as some Cpp class are not defined.
• When generating a network from ONNX, the first cell is linked to a dummy provider. When calibrating the network the first cell still pointed to this dummy provider and not to the new one.
• Fixed an issue in LSQ quantization when we re-initialize weights and diff weights.
• Fixed a bug where it was not possible to export a float32 model.
• Fixed a bug where the ONNX import would fail to fuse the bias into a fully connected/conv layer.
• Interface instances can now be used as an input for a cell without coming from another cell themselves.
• Updated torch -> n2d2 tensor conversion to handle views.

C++

• Added the former CPP export and the current one is moved to CPP_Quantization
• The Memory Manager has been fixed for the Float32 export
• DeepNet::removeCell now updates the cell pointed by the Target if the removed cell is the last one
• Fixed the import ONNX to concatenate Mul and Add layers into a BatchNorm;
• Fixed ONNX import padding for opset version < 11 (see onnx changelog).
  • Versions < 11 have pads as a parameter of the node.
  • Versions >= 11 have pads as an input of the node.

⚙️ Refactor

Python API

• Global variable is now an enum;
• n2d2.global_variables.cuda_compiled has been renamed to cuda_available
• Introduced the use of single method dispatch for code readability (code specific to the datatype of the first parameter of a method).
• Conv, Fc and Deconv layer use a Linear activation by default instead of None to fix error during post training quantization.
• Added protobuff as a dependency of keras interop and removed dependencies to the onnx package in requirements.txt.
• Renamed pytorch_interoperability -> pytorch_to_n2d2 and keras_interoperability -> keras_to_n2d2

C++

• Refactored quantification and export functions to remove the use of DeepNet referenced by cells.
• Renamed N2D2 namespace to N2D2_Export in the CPP export, to avoid conflict with N2D2 when using both N2D2 and the CPP export.
• Renamed the ONNX constant to EXPORT_ONNX in the CPP export, to avoid conflict with the ONNX constant in N2D2.
• The new path of the binaries generated by the exports are located in ./bin/run_export
• Multiple files in the CPP export have been improved like the Makefile or cpp_utils.cpp.

🚀 Improvements

Python API

• Added decorator to check inputs type in the PythonAPI.
• Added default model ResNetv1-[18,34,50,101,152].
• Improved export test by compiling and running the exports.
• Added an option to allow the user to choose the opset version used for PyTorch interoperability.
• Added a summary method, a new display system to synthesize neural network.
  The implemented function draws a table with all the NN layers.
  Per layer, multiple information are displayed such as output dimensions, number of parameters of the layer, computing demand (in MAC), some extra information depending on the layer type, and if the current layer is trainable or not.

Here an example of what summary can do with a small network.

--------------------------------------------------------------------------------------------------------------
Layer (type)           Output Shape          Param #     MAC #       Connected to    Extra                Grad
==============================================================================================================
Image1 (input)         (1, 3, 256, 256)            0           0k                                         -

Conv_0 (Conv 3x3)      (1, 64, 256, 256)       1,792     112,657k    Image1          Act: ReLu            True

Pool_0 (Pool)          (1, 64, 129, 129)           0           0k    Conv_0          size: 2x2, pad: 1    -

Conv_1 (PointWise)     (1, 128, 131, 131)      8,320     136,323k    Pool_0          Act: ReLu            True

Fc_0 (Fc)              (1, 100, 1, 1)              0     219,660k    Conv_1                               True

Fc_1 (Fc)              (1, 10, 1, 1)               0           1k    Fc_0                                 True

Softmax_0 (Softmax)    (1, 10, 1, 1)               0           0k    Fc_1                                 -

Features (output)      (1, 10, 1, 1)               0           0k    Softmax_0                            -
==============================================================================================================
Total params:  10,112
Total computing:  468,642,326 MAC

• Added list_exportable_cell function to the PythonAPI export module.
  Usage example :

>>> import n2d2 
>>> n2d2.export.list_exportable_cell("CPP_TensorRT")
+----------------+-----------+
| Cell Name      | Available |
+================+===========+
| Activation     | Yes       |
+----------------+-----------+
| BatchNorm2d    | Yes       |
+----------------+-----------+
| Conv           | Yes       |
+----------------+-----------+
| Deconv         | Yes       |
+----------------+-----------+
| Dropout        | No        |
+----------------+-----------+
| ElemWise       | Yes       |
+----------------+-----------+
| Fc             | Yes       |
+----------------+-----------+
| Padding        | Yes       |
+----------------+-----------+
| Pool           | Yes       |
+----------------+-----------+
| Reshape        | Yes       |
+----------------+-----------+
| Resize         | Yes       |
+----------------+-----------+
| Scaling        | Yes       |
+----------------+-----------+
| Softmax        | Yes       |
+----------------+-----------+
| Transformation | No        |
+----------------+-----------+
| Transpose      | Yes       |
+----------------+-----------+

• Added the list of exportable cells to the documentation.
• Created a decorator to template documentation.

C++

• Added a broadcasting possibility for ElementWise cell ( (1 * 1) Prod (N * N) = (N * N))
• Added a CPU implementation for the LSQ quantization method.

N2D2 1.1.0

💡 Highlights

We are delighted to annonce the new release of N2D2 with exciting new features and with a little tidying up in the codebase 🧹
We would like to thank everyone who contributed to this release!

• Improvement of the Python API: our main objective is to improve the user experience with the Python API so we work to make your experience more seamless and close to your needs. Don't hesitate to give us your feedback about it !
• Learned Stepsize Quantization module implemented in N2D2: this module implements the first QAT method developed in the framework. You can now quantize your network with a precision lower than 8-bit with this QAT method with the Python API or the INI one.
• Documentation improvement: we added multiple examples to make N2D2 easier to learn.

Moreover, more features are already planned to be added in the upcoming releases like other QAT techniques (the SAT technique for example). So don't go too far !

🐛 Bug Fixes and Other Changes

Cmake

Compile N2D2 without CUDA

You can now compile N2D2 without enabling CUDA, even if the library is installed on your device.
To activate this feature, export the environment variable N2D2_NO_CUDA=1 before compiling N2D2.

Resolved warnings about the Python binding for extra-modules

The CMake warning happens when N2D2 is adding the Binding functions to the compilation. The warning doesn't occur when N2D2 is used in an extra module but happens when N2D2 is not used in an extra module.
The resolution is to include a variable in the extra module CMakeLists.txt of the extra module to import the N2D2 Python binding and performs a test to know if the variable is defined in the CMakeLists.txt of N2D2.

So in the development of the upcoming extra modules using N2D2, don't forget to include at the beginning of your CMakeLists.txt.

set(N2D2_EXTRAMODULE 1)

C++

Removed PYBIND directive

The PYBIND directive has been removed from all files since it is not necessary to add it because pybind is mandatory to compile and use N2D2.

Moved MultiDevice Peer Access function to CudaUtils

To enable Peer Access among devices while the Multi-GPU is used, the function is called from CudaUtils now and not from DeepNet::initialize. This fixed a problem when the deepNet was initialized more than once.

Python API

Default_model not set up anymore

The wrap function of the Keras Interoperability doesn't automatically set the model to Frame_CUDA anymore.
You need to change the global variable default_model in your Python script.

Refactoring/splitting files into classes

The following python modules have been splitted to improve the code readibility

• transformation
• database
• filler
• activation
• cell

Fixed incomplete batch in the Pytorch Interoperability

Before, an error was raised if the batch provided by Pytorch was incomplete.
Now N2D2 can handle different batch sizes with the same model.

Change the deepNet name convention (from hashes to incremental indexes)

deepnet_234213 -> deepnet_0

The provider will no longer load data to check data size

Before the provider needed to load data from the dataset in order to provide the data size which was not necessary.

Avoid BatchNorm fuse with tf2onnx framework

The library tf2onnx removes automatically the BatchNorm layer during an ONNX export (fuse it the previous convolutional layer). It is an incompatible behavior with the Keras Interoperability. Therefore, we added a context to temporaly remove this optimization in order to keep the BatchNorm in N2D2.
More details in python/keras_interoperability/keras_interface.py.

Fixed setattr check if N2D2 object is created

Before, setting an attribute on an object which is initialized on call raises an error N2D2 object is not initialized.
Now, the set attribut method checks if N2D2 object is created. Otherwise, it fills tha attribut dictionary

Exports

C++ export

• Fixed signed outputs display (sign bit badly converted)
• Fixed output results from the convCellDWPropagate

🚀 Improvements

C++

QAT module added

We added the Quantization Aware-Training module to the framework in order to use QAT techniques.
The first technique is LSQ and is implemented to be used with Activation cells and with Conv/Fc cells.
You can find the functions in Quantizer/QAT. This module is organized as follows:

• Activation: methods to quantize activations
• Cell: methods to quantize the cells (essentially the parameters)
• Kernel: kernels to quantize on CPU and on CUDA

Moreover, several examples are proposed to use LSQ in python/examples.

if isinstance(cell, n2d2.cells.Conv):
        cell.quantizer = LSQCell(
                range = q_range,
                solver=n2d2.solver.SGD(
                        learning_rate_policy = "CosineDecay",
                        learning_rate=lr,
                        momentum=mom,
                        decay=decay,
                        max_iterations=max_iter
        )) 

Warning: LSQ is available for CUDA experiments so far. A CPU implementation is coming soon !

Separation calibration and export

In order to propose Post-Training Quantization techniques without necessarily exporting the model, we decided to separate the PTQ function and the generation of the export. So if you want to use the PTQ and the export, add those lines in your script:

n2d2.quantizer.PTQ(n2d2_net_test.get_deepnet_cell(), 8, provider=provider, act_scaling_mode="SINGLE_SHIFT")

n2d2.export.export_c(n2d2_net_test.get_deepnet_cell(), export_folder_name="test_export") 

Python API

Added tiny ML models to N2D2 tests

We decided to use tiny ml networks to test our framework, interoperabilities and exports.

Decorator for export docstring

We added a decorator to add docstring to the export functions in order to improve the comprehension of our functions.

@n2d2.utils.add_docstring(export_doc_string)
def export_c(deepnet_cell: n2d2.cells.DeepNetCell,
             provider: n2d2.provider.Provider=None,
             **kwargs) -> None:
    """Generate a C export of the neural network.

Added Linter for the Python API

In order to improve the Python API code quality, we added a linter in our Continious Integration tests.

📖 Documentation modifications

SAT QAT method example added

We added multiple examples of the SAT quantization technique with the Python API.

Improve examples with the StimuliProvider

We improved our examples in the Python API:

• Creation of a StimuliProvider
• Add a transformation and OnTheFly transformation
• Creation of N2D2 graph
• How to access layers and parameters
• ONNX import

N2D2 1.0.0

Beginning of the releases

N2D2 has been on Github since 2017 and we are grateful for the support and the time that have been given by our contributors and the users to develop and enrich N2D2.

But this is not the time for the adventure to end, quite the contrary! 🚀

We set up a release system to keep you regularly informed of the latest additions of features, bug fixes, changes, ...

If you have any question or problem with the framework, feel free to open an issue on Github.
We will answer it as soon as possible 😉

How to use N2D2

Currently to use N2D2, you can work with INI files to design, build, simulate and export your network.
We are also working to propose the same experience with our Python API with the same features and even more. You can test right now the Python API of N2D2 and tell us your feedback about it.

You can find some explanations on our documentation about the installation of the Python API and how use it. We are open to any kind of relevant improvement of the framework and everyone can express his/her opinion.