Skip to content
/ DiffMM Public

[ACM MM'2024]"DiffMM: Multi-Modal Diffusion Model for Recommendation"

Notifications You must be signed in to change notification settings

HKUDS/DiffMM

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DiffMM: Multi-Modal Diffusion Model for Recommendation

This is the PyTorch implementation for DiffMM proposed in the paper DiffMM: Multi-Modal Diffusion Model for Recommendation, which is accepted by ACM MM 2024 Oral.

In this paper, we propose DiffMM, a new multi-modal recommendation model that enriches the probabilistic diffusion paradigm by incorporating modality awareness. Our approach utilizes a multi-modal graph diffusion model to reconstruct a comprehensive user-item graph, while harnessing the advantages of a cross-modal data augmen- tation module that provides valuable self-supervision signals. To assess the effectiveness of DiffMM, we conducted extensive experi- ments, comparing it to several competitive baselines. The results unequivocally establish the superiority of our approach in terms of recommendation performance, firmly establishing its efficacy.

πŸ“ Environment

We develop our codes in the following environment:

  • python==3.9.13
  • numpy==1.23.1
  • torch==1.11.0
  • scipy==1.9.1

🎯 Experimental Results

Performance comparison of baselines on different datasets in terms of Recall@20, NDCG@20 and Precision@20:

πŸš€ How to run the codes

The command lines to train DiffKG on the three datasets are as below. The un-specified hyperparameters in the commands are set as default.

! If you want to run the codes on baby or sports dataset, please firstly follow the instrcution in ./Datasets

  • TikTok
python Main.py --data tiktok --reg 1e-4 --ssl_reg 1e-2 --epoch 50 --trans 1 --e_loss 0.1 --cl_method 1
  • Baby
python Main.py --data baby --reg 1e-5 --ssl_reg 1e-1 --keepRate 1 --e_loss 0.01
  • Sports
python Main.py --data sports --reg 1e-6 --ssl_reg 1e-2 --temp 0.1 --ris_lambda 0.1 --e_loss 0.5 --keepRate 1 --trans 1

πŸ‘‰ Code Structure

.
β”œβ”€β”€ README.md
β”œβ”€β”€ Main.py
β”œβ”€β”€ Model.py
β”œβ”€β”€ Params.py
β”œβ”€β”€ DataHandler.py
β”œβ”€β”€ Utils
β”‚   β”œβ”€β”€ TimeLogger.py
β”‚   └── Utils.py
β”œβ”€β”€ figures
β”‚   β”œβ”€β”€ model.png
β”‚   β”œβ”€β”€ dataset.png
β”‚   └── performance.png
└── Datasets
    β”œβ”€β”€ tiktok
    β”‚   β”œβ”€β”€ trnMat.pkl
    β”‚   β”œβ”€β”€ tstMat.pkl
    β”‚   β”œβ”€β”€ valMat.pkl
    β”‚   β”œβ”€β”€ audio_feat.npy
    β”‚   β”œβ”€β”€ image_feat.npy
    β”‚   └── text_feat.npy
    β”œβ”€β”€ baby
    β”‚   β”œβ”€β”€ trnMat.pkl
    β”‚   β”œβ”€β”€ tstMat.pkl
    β”‚   β”œβ”€β”€ valMat.pkl
    β”‚   β”œβ”€β”€ text_feat.npy
    β”‚   └── image_feat.npy.zip
    └── README.md

πŸ“š Datasets

🌟 Citation

If you find this work helpful to your research, please kindly consider citing our paper.

@article{jiang2024diffmm,
  title={DiffMM: Multi-Modal Diffusion Model for Recommendation},
  author={Jiang, Yangqin and Xia, Lianghao and Wei, Wei and Luo, Da and Lin, Kangyi and Huang, Chao},
  journal={arXiv preprint arXiv:2406.11781},
  year={2024}
}