practical_structural_pruning.md

Practical Structural Pruning

Welcome to the resource list for structural pruning, which is continually being updated. If you have any useful resources, papers, or repositories related to structural pruning and would like to share them with other users, please feel free to open a new pull request or issue.

0. Projects/Papers that use Torch-Pruning

Title	URL
LLaMA-Pruning: Structural Pruning for LLaMA	project
Performance-aware Approximation of Global Channel Pruning for Multitask CNNs	project, TPAMI'23
LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning	project, NeurIPS'22
mobile-yolov5-pruning-distillation	project
Pruning-for-YOLOX	project
YOLOX-pruning	project
Vision Model Compression (ResNet-18, VGG-16)	project
LAASP: Loss-aware automatic selection of structured pruning criteria for deep neural network acceleration	project
Magnitude and Similarity based Variable Rate Filter Pruning for Efficient Convolution Neural Networks	project
Resnet50-pruning	projects
yolov5-pruning	project
Model-Pruning (MobileNetv1/v2/v3, VGG16, and EfficientNet-b0)	project
Melanoma detection CNN using PyTorch	project
KamalEngine (Model Reuse)	project
Modified-Yolov4Tiny-RaspberryPi	project
model_pruning (ResNet18+CIFAR-10)	project

1. Benchmark & Survey & Awesome List

Title	Authors	Venue	Code
Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning	Huan Wang	Preprint	pytorch
Structured Pruning for Deep Convolutional Neural Networks	Yang He	Preprint	Awesome List

2. General

Title	Authors	Venue	Code
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks	Zhonghui You	NeurIPS'19	pytorch
Neural Pruning via Growing Regularization	Huan Wang	ICLR'21	pytorch
Group Fisher Pruning for Practical Network Compression	Liyang Liu	ICML'21	pytorch
Only Train Once: A One-Shot Neural Network Training And Pruning Framework	Tianyi Chen	NeurIPS'21	pytorch
OTOV2: AUTOMATIC, GENERIC, USER-FRIENDLY	Tianyi Chen	ICLR'23	pytorch
DepGraph: Towards Any Structural Pruning	Gongfan Fang	CVPR'23	pytorch

3. YOLO

Type	Title	Authors	Venue	Code
YOLOv5	Performance-aware Approximation of Global Channel Pruning for Multitask CNNs	Hancheng Ye	TPAMI'23	pytorch
YOLOv7	Torch-Pruning: benchmarks/prunability/yolov7	-	-	pytorch

4. GANs

Title	Authors	Venue	Code
GAN Compression: Efficient Architectures for Interactive Conditional GANs	Muyang Li	CVPR'20	pytorch

5. Vision Transformers

Title	Authors	Venue	Code
Vision Transformer Pruning	Mingjian Zhu	Preprint	N/A
SAViT: Structure-Aware Vision Transformer Pruning via Collaborative Optimization	Zheng Chuanyang	NeurIPS'22	N/A
Width & Depth Pruning for Vision Transformers	Fang Yu	AAAI'22	N/A
Chasing Sparsity in Vision Transformers: An End-to-End Exploration	Tianlong Chen	NeurIPS'21	pytorch
CP-ViT: Cascade Vision Transformer Pruning via Progressive Sparsity Prediction	Zhuoran Song	Preprint	pytorch
DepGraph: Towards Any Structural Pruning	Gongfan Fang	CVPR'23	pytorch

6. NLP

Title	Authors	Venue	Code
Structured Pruning Learns Compact and Accurate Models	Mengzhou Xia	ACL'22	pytorch

7. GNNs

Title	Authors	Venue	Code
DepGraph: Towards Any Structural Pruning	Gongfan Fang	CVPR'23	pytorch

8. Misc

Title	Source
Issue - Does the tool support to prune the yolov5/v7/v8?	Torch-Pruning
Issue - How can we transfer schedular and optimizer from original model to pruned model	Torch-Pruning
Discussion - How to get the index of the pruned channel(s) when using High-level Pruners?	Torch-Pruning
Issue - How to get all groups of a network	Torch-Pruning
Issue - 论文中图3(b)的剪枝方案是怎么推出的 (in Chinese)	Torch-Pruning
Model Compression - the Pruning Approaches	Lecture Notes (Princeton)
CVPR 2023 | DepGraph：任意架构的结构化剪枝，CNN、Transformer、GNN等都适用！(In Chinese)	CVer
[CVPR 2023	DepGraph 通用结构化剪枝 (In Chinese)](https://zhuanlan.zhihu.com/p/619146631?)