This repo hosts the offical implementation for the paper:
Two-Branch Pure Transformer for Hyperspectral Image Classification, IEEE Geoscience and Remote Sensing Letters (GRSL), Xin He, Yushi Chen, Qingyun Li.
The full text is also availiable at ResearchGate.
The implementation of Swin network is based on mmdetection.
Owing to its capability of building long-range dependencies and global context connections, Transformer has been used for hyperspectral image (HSI) classification. However, most of the existing Transformer-based HSI spatial–spectral classification methods consist of a convolutional neural network (CNN) and Transformer, which are used to extract the local and global information, respectively. In this study, to fully explore the potential of Transformer, a pure Transformer is investigated for HSI classification. First, a spatial Transformer (Spa-TR) is designed for HSI spatial classification, which learns the spatial features locally and globally by adopting the window partition and shifted window schemes. Especially, the self-attention computations are limited within the local windows and cross-windows. Second, to fully use the abundant spectral information in HSIs, a two-branch pure Transformer (i.e., Spa-Spe-TR) is proposed, which includes a spectral Transformer (Spe-TR) and a Spa-TR. The spectral sequence features learned by Spe-TR and the spatial features generated by Spa-TR are effectively fused with a branch fusion strategy, which explicitly and automatically measures the importance between the joint spatial–spectral features and improves the discriminability of the joint features. Experimental results on the two widely used HSI datasets (i.e., Pavia and Indian Pines) demonstrate the efficacy of the proposed methods in comparison with other state-of-the-art approaches.
- Windows, Ubuntu (as long as OpenMMLab can be installed)
- torch, torchvision (such as torch==1.10.0+cu111 torchvision==1.11.0+cu111)
- mmcv 1.x, mmdetection 2.x
- timm
- scikit-learn (sklearn)
conda create -n hsi_cls python=3.8 # take python 3.8 as example
conda activate hsi_cls
pip install torch==1.10.0+cu111 torchvision==1.11.0+cu111 -f https://download.pytorch.org/whl/cu111/torch_stable.html
pip install -U openmim
mim install mmcv-full mmdet
# or visit https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md/#Installation
pip install -r requirements.txt@ARTICLE{pure_Tr_hsi2022,
author={He, Xin and Chen, Yushi and Li, Qingyun},
journal={IEEE Geoscience and Remote Sensing Letters},
title={Two-Branch Pure Transformer for Hyperspectral Image Classification},
year={2022},
volume={19},
number={},
pages={1-5},
doi={10.1109/LGRS.2022.3217775}}
If you have any questions, please contact me via issue or email.