Experimental implementation for paper Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training.
- CUDA 11.7
- linux-tools-aws
- python3-dev
- libunwind-dev
Test on Ubuntu 20.04, Python 3.9, PyTorch 1.13.0, CUDA 11.7.1
Import daydream, start trace at the beginning of your code and end trace at the end of your code. For an example, see train.py.
- Trace six activities, including Memory Copy, Memory Set, Kernel Execution, CUDA Driver and CUDA Runtime.
- Packaged as PyThon extension.
- Add trace visualization and analysis tools.
- Modify DNN framework for kernel to layer mapping.
- Construct dependency graph and expose graph trasformation APIs for users to manipulate the graph.