Another Dynet Windows CUDA installation writeup (Sep '19)

Other instruction-sets for installing DyNet/CUDA on Windows did not work on my setup, I suspect that changes to the setup code might also be at play.

Rather than perform a thorough debug of the build/install logic to pinpoint exactly where things are going wrong, I took more of a brute-force approach, especially with the 'Installation' stage.

The writeup below captures this.

Environment

• Physics: Thinkpad Carbon X1 G6 with Razor eGPU (with an RTX2070)
• OS: Windows 10 Enterprise (10.0.18362)
• Visual Studio: Community 2015 with updates (installed with en_visual_studio_community_2015_with_update_3_x86_x64_web_installer_8922963)
• CUDA: v10.0 (installed with cuda_10.0.130_win10_network.exe installer)
• CUDNN: for CUDA v10.0 (downloaded cudnn-10.0-windows10-x64-v7.6.3.30.zip and copied files to CUDA folders per instructions)
• Python: Anaconda3 2019.07
• CMAKE: v3.15.3 (installed with cmake-3.15.3-win64-x64 installer)
• GIT BASH: when installing choose the option that adds all Linux utilities to PATH

Preparations

Running from a cmd session I create a new conda environment and activate it. I then install numpy and cython with standard conda commands:

conda create --name myenv python=3.7
conda activate myenv
pip install cython numpy

I clone this repository and then make two subtle changes to the setup.py file that can be seen here. The following sequence, run from the same cmd session (using the same conda environment), accomplishes this:

mkdir dynet-build
cd dynet-build
git clone https://github.com/clab/dynet.git
wget https://github.com/yyellin/dynet/commits/master/setup.py
mv -f setup.py dynet\
cd dynet

Build

Build and install must be treated separately; and build will not work without setting a bucket load of environment parameters (most of my toil was in getting these assignments lined up properly)

set BACKEND=cuda
set CMAKE=C:\Program Files\CMake\bin\cmake.exe
set CC=C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\cl.exe
set CXX=C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\cl.exe
set MAKE=C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe
set MAKE_FLAGS=dynet.sln /p:Configuration=Release
set CUDA_TOOLKIT_ROOT_DIR=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\
set CUDA_NVCC_EXECUTABLE=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin\nvcc.exe
set CMAKE_GENERATOR=Visual Studio 14
set CMAKE_GENERATOR_PLATFORM=x64

python setup.py build

Do not be concerned by the error that get spits out almost immediately, it seems harmless.

cl : Command line warning D9002 : ignoring unknown option '--version'
cl : Command line error D8003 : missing source filename

On my setup this takes just under 14.5 minutes and completes with the following output:

6727 Warning(s)
0 Error(s)

Time Elapsed 00:14:24.72
running build_py
INFO:root:Building Python files...
creating build\lib.win-amd64-3.7
copying dynet.py -> build\lib.win-amd64-3.7
copying dynet_viz.py -> build\lib.win-amd64-3.7
copying dynet_config.py -> build\lib.win-amd64-3.7
running build_ext

The final running build_ext doesn't do anything due to the second change in the setup.py file. (Python extensions are built as part of the main Make sequence).

Installation

All my attempts to use the given setup.py machinery to get the dynet package installed were fruitless. So I took a shortcut and built a mini setup environment as follows (within the same cmd session).

mkdir -p for_local_install\dynet
cp build\py3.7-64bit\python\_dynet.cp37-win_amd64.pyd for_local_install\dynet\
cp build\py3.7-64bit\python\dynet.py for_local_install\dynet\
cp build\py3.7-64bit\python\dynet_config.py for_local_install\dynet\
cp build\py3.7-64bit\python\dynet_viz.py for_local_install\dynet\

Next create a seup.py file in the for_local_install\dynet\ directory with the following content:

from setuptools import setup, find_packages

setup(
    name='dyNET',
    version='2.1',
    description='The Dynamic Neural Network Toolkit',
    py_modules=["dynet", "dynet_viz", "dynet_config"],
    install_requires=["cython", "numpy"],
    data_files=[('lib\site-packages', ['_dynet.cp37-win_amd64.pyd'])],
    include_package_data=True
)

And finally, finally, in same session:

pip install for_local_install\dynet