[Feature] add HSI-Drive dataset

[jonGuti13]

Motivation

The motivation is to add a hyperspectral dataset HSI Drive 2.0 to the dataset registry which would be, as far as I know, the first hyperspectral database of mmsegmentation. This database has been presented in HSI-Drive v2.0: More Data for New Challenges in Scene Understanding for Autonomous Driving and the initival v1 was presented in HSI-Drive: A Dataset for the Research of Hyperspectral Image Processing Applied to Autonomous Driving Systems

Modification

I have created/modified the following aspects:

• READMEs: README.md and README_zh-CN.md (sorry if translation is not accurate).
• Example project: projects/hsidrive20_dataset has been created and filled for users to know how to work with this database.
• Documentation: docs/en/user_guides/2_dataset_prepare.md and docs/zh_cn/user_guides/2_dataset_prepare.md (sorry if translation is not accurate) have been updated for users to know how to download and configure the dataset.
• Database related files: mmseg/datasets/__init__.py, mmseg/datasets/hsi_drive.py and configs/_base_/datasets/hsi_drive.py where the dataset is described and also prepared for training/validation/test.
• Transforms related files: mmsegmentation/mmseg/datasets/transforms/loading.py to include support for loading images from .npy files such as the hyperspectral images of this dataset.
• Training config with well-known neural network: configs/unet/unet-s5-d16_fcn_4xb4-160k_hsidrive-192x384.py for people to train a standard neural network with this dataset.
• Tests: added necessary files under tests/data/pseudo_hsidrive20_dataset.

Important: I have also modified .pre-commit-config.yaml to ignore HSI error in codespell.

BC-breaking (Optional)

No.

Use cases (Optional)

A train example has been added under projects/hsidrive20_dataset and documentation has been updated as it is explained in Modification section.

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMDet3D.
4. The documentation has been modified accordingly, like docstring or example tutorials.

Regarding 1. I don't know how to solve this problem. Could you help me, please? This causes 2 checks not to be successful.

[CLAassistant]

All committers have signed the CLA.

[jonGuti13]

I don't know why but checks fail as a consequence of error Exited with code exit status 137 which occurs when a process is terminated because it's using too much memory.

In a previous attempt, failure ocurred in maximum_version_cpu while in this attempt the error has ocurred in minimum_version_cpu, that is, previously.

Any kind of help is highly appreciated.

[YanivZimmer]

Hi, I would be very happy if this PR could be reviewed and approved as I'm waiting for it aswell :)

[YanivZimmer]

Any updates on this one?

[jonGuti13]

Hi @YanivZimmer !

This PR is blocked by open-mmlab/mmcv#2950 (which is also mine). I have just checked that repository and confirmed that the last approved pull request is from Aug 14, 2023 (mine is from Oct 5, 2023) so newer pull requests have not been approved yet and have a "review required (by reviewers with write access)" tag attached.

I have no further information about it.

[xiexinch]

Hi @jonGuti13,
Sorry for the late reply! I think we can implement a new Loading method, then we don't need to wait for the mmcv PR open-mmlab/mmcv#2950 .
Some examples: https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/datasets/transforms/loading.py

[jonGuti13]

Hi @xiexinch!

I will have a look at https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/datasets/transforms/loading.py and see if any of those methods fit the one I need.

When finishing that do I have to change the content of this PR or do I have to create a new one?

Thanks.

[xiexinch]

Hi @xiexinch!

I will have a look at https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/datasets/transforms/loading.py and see if any of those methods fit the one I need.

When finishing that do I have to change the content of this PR or do I have to create a new one?

Thanks.

I think directly updating the content of this pr would be fine, thanks a lot!

[jonGuti13]

I think directly updating the content of this pr would be fine, thanks a lot!

Following @xiexinch's advice, I have included the class LoadImageFromNpyFile in https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/datasets/transforms/loading.py and so the pull request is ready to be reviewed.