Comparing HFTL and OSM Networks in the context of Cross-Domain Plant Identification

This repository contains the implementation method of our Herbarium-Field Triplet Loss Network (HFTL Network) and One-streamed Mixed Network (OSM Network) in the context of Cross-Domain Plant Identification. Our results show that the HFTL Network can generalize rare species as equally as species with many training data better than the OSM Network (conventional CNNs).

Figure A and B below show the Top-5 predictions of a plant sample with its predicted scores and activation maps from the HFTL and OSM Networks respectively. More samples of comparison can be found here.

HFTL Network	OSM Network

Research article

A machine learning approach for cross-domain plant identification using herbarium specimens
https://doi.org/10.1007/s00521-022-07951-6

Requirements

• Tensorflow 1.12
• TensorFlow-Slim library
• imagenet pretrained models (inception v4 and inception resnet v2)

Data

• PlantCLEF 2021
• PlantCLEF 2017

Scripts

Training scripts

• HFTL Network
  • run_HFTL_network.py
  • network_module_HFTL.py
• OSM Network
  • run_OSM_network.py
  • network_module_OSM.py

Validation scripts

• HFTL Network

  1. get_herbarium_dictionary_HFTL.py
  2. validate_HFTL_network.py
  3. get_mrr_score.py

• OSM Network

  1. get_herbarium_dictionary_OSM.py
  2. validate_OSM_network.py
  3. get_mrr_score.py

Visualizing activation map scripts

• HFTL Network
  • visualise_prediction_HFTL-IR.py
• OSM Network
  • visualise_prediction_OSM-IR.py

Lists

Herbarium Network

• clef2021_herbarium_combined_train_306005.txt
• clef2021_herbarium_combined_test_15221.txt

Field Network (2017)

• clef2017_EOL_Web_train_server.txt
• clef2017_EOL_Web_test_server.txt

Field Network (2021)

• clef2021_field_combined_train.txt
• clef2021_field_combined_test.txt

HFTL Network

• clef2021_shared_class_herbarium_train.txt
• clef2021_shared_class_herbarium_test.txt
• clef2021_shared_class_field_train.txt
• clef2021_shared_class_field_test.txt

OSM Network

• clef2021_mixed_class_train.txt
• clef2021_mixed_class_test.txt

Herbarium Dictionary

• herbarium_dictionary.txt
• herbarium_dictionary_with_field_bias.txt

Test Sets

• test_set_1_seen.txt (with field training images)
• test_set_2_unseen.txt (without field training images)
  (note: the image files from Test Set 2 are sourced from Google Image queries)

Checkpoints / Trained models

• HFTL Network
  • HFTL inception resnet v2 model
  • HFTL inception v4 model
• OSM Network
  • OSM inception resnet v2 model
  • OSM inception v4 model