The reproduced results are relatively poor

[liuxingyu123]

Hello, the pretrained model I use for vit is vit_ large_ patch16_ 384, but the results of the first stage of training are ODS: 0.74116, OIS: 0.761012, AP: 0.736421. In the paper, ODS: 0.817, OIS: 0.835, AP: 0.867, the parameter of code has not changed. What is the problem？

[liuxingyu123]

Is it a problem of environment configuration? My environment is python3.8，pytorch1.7.0 cuda11.0 mmcv1.2.3.

[MengyangPu]

Python: 3.7.7 [GCC 7.3.0]
CUDA available: True
GPU: Tesla V100-PCIE-32GB
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.6.0
PyTorch compiling details: PyTorch built with:

GCC 7.3
C++ Version: 201402
Intel(R) Math Kernel Library Version 2020.0.1 Product Build 20200208 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v1.5.0 (Git Hash e2ac1fac44c5078ca927cb9b90e1b3066a0b2ed0)
OpenMP 201511 (a.k.a. OpenMP 4.5)
NNPACK is enabled
CPU capability usage: AVX2
CUDA Runtime 10.1
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
CuDNN 7.6.3
Magma 2.5.2
TorchVision: 0.7.0
OpenCV: 4.5.1
MMCV: 1.2.4
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 10.1
MMSegmentation: 0.6.0+

[MengyangPu]

We fixed and updated the codes, please redownload and run, thanks~

[yun-liu]

Have anyone reproduced the results? Could you share how to do it?

[hhqweasd]

I reproduced the first stage of training with vit large_ patch16_ 384_ backbone, the ODS and OIS are 0.813 and 0.830 respectively on the tesing set of BSDS at iter_10k, which are similar to the reported results: ODS: 0.817, OIS: 0.835

---pip envs---
addict 2.4.0
appdirs 1.4.4
certifi 2022.12.7
cityscapesScripts 2.2.1
clip 1.0
coloredlogs 15.0.1
contourpy 1.0.7
cycler 0.11.0
fonttools 4.39.3
ftfy 6.1.1
future 0.18.3
h5py 3.8.0
humanfriendly 10.0
importlib-resources 5.12.0
kiwisolver 1.4.4
matplotlib 3.7.1
mmcv-full 1.2.2
mmsegmentation 0.6.0 /root/EDTER-main
numpy 1.24.2
opencv-python 4.7.0.72
packaging 23.0
Pillow 9.5.0
pip 23.0.1
pyparsing 3.0.9
pyquaternion 0.9.9
python-dateutil 2.8.2
PyYAML 6.0
regex 2023.3.23
scipy 1.10.1
setuptools 65.6.3
six 1.16.0
torch 1.6.0+cu101
torchvision 0.7.0+cu101
tqdm 4.65.0
typing 3.7.4.3
wcwidth 0.2.6
wheel 0.38.4
yapf 0.32.0
zipp 3.15.0

---conda envs---
_libgcc_mutex 0.1 main https://mirrors.aliyun.com/anaconda/pkgs/main
_openmp_mutex 5.1 1_gnu https://mirrors.aliyun.com/anaconda/pkgs/main
addict 2.4.0 pypi_0 pypi
appdirs 1.4.4 pypi_0 pypi
ca-certificates 2023.01.10 h06a4308_0 https://mirrors.aliyun.com/anaconda/pkgs/main
certifi 2022.12.7 py38h06a4308_0 https://mirrors.aliyun.com/anaconda/pkgs/main
cityscapesscripts 2.2.1 pypi_0 pypi
clip 1.0 pypi_0 pypi
coloredlogs 15.0.1 pypi_0 pypi
contourpy 1.0.7 pypi_0 pypi
cycler 0.11.0 pypi_0 pypi
fonttools 4.39.3 pypi_0 pypi
ftfy 6.1.1 pypi_0 pypi
future 0.18.3 pypi_0 pypi
h5py 3.8.0 pypi_0 pypi
humanfriendly 10.0 pypi_0 pypi
importlib-resources 5.12.0 pypi_0 pypi
kiwisolver 1.4.4 pypi_0 pypi
ld_impl_linux-64 2.38 h1181459_1 https://mirrors.aliyun.com/anaconda/pkgs/main
libffi 3.4.2 h6a678d5_6 https://mirrors.aliyun.com/anaconda/pkgs/main
libgcc-ng 11.2.0 h1234567_1 https://mirrors.aliyun.com/anaconda/pkgs/main
libgomp 11.2.0 h1234567_1 https://mirrors.aliyun.com/anaconda/pkgs/main
libstdcxx-ng 11.2.0 h1234567_1 https://mirrors.aliyun.com/anaconda/pkgs/main
matplotlib 3.7.1 pypi_0 pypi
mmcv-full 1.2.2 pypi_0 pypi
mmsegmentation 0.6.0 dev_0
ncurses 6.4 h6a678d5_0 https://mirrors.aliyun.com/anaconda/pkgs/main
numpy 1.24.2 pypi_0 pypi
opencv-python 4.7.0.72 pypi_0 pypi
openssl 1.1.1t h7f8727e_0 https://mirrors.aliyun.com/anaconda/pkgs/main
packaging 23.0 pypi_0 pypi
pillow 9.5.0 pypi_0 pypi
pip 23.0.1 py38h06a4308_0 https://mirrors.aliyun.com/anaconda/pkgs/main
pyparsing 3.0.9 pypi_0 pypi
pyquaternion 0.9.9 pypi_0 pypi
python 3.8.16 h7a1cb2a_3 https://mirrors.aliyun.com/anaconda/pkgs/main
python-dateutil 2.8.2 pypi_0 pypi
pyyaml 6.0 pypi_0 pypi
readline 8.2 h5eee18b_0 https://mirrors.aliyun.com/anaconda/pkgs/main
regex 2023.3.23 pypi_0 pypi
scipy 1.10.1 pypi_0 pypi
setuptools 65.6.3 py38h06a4308_0 https://mirrors.aliyun.com/anaconda/pkgs/main
six 1.16.0 pypi_0 pypi
sqlite 3.41.1 h5eee18b_0 https://mirrors.aliyun.com/anaconda/pkgs/main
tk 8.6.12 h1ccaba5_0 https://mirrors.aliyun.com/anaconda/pkgs/main
torch 1.6.0+cu101 pypi_0 pypi
torchvision 0.7.0+cu101 pypi_0 pypi
tqdm 4.65.0 pypi_0 pypi
typing 3.7.4.3 pypi_0 pypi
wcwidth 0.2.6 pypi_0 pypi
wheel 0.38.4 py38h06a4308_0 https://mirrors.aliyun.com/anaconda/pkgs/main
xz 5.2.10 h5eee18b_1 https://mirrors.aliyun.com/anaconda/pkgs/main
yapf 0.32.0 pypi_0 pypi
zipp 3.15.0 pypi_0 pypi
zlib 1.2.13 h5eee18b_0 https://mirrors.aliyun.com/anaconda/pkgs/main

---gpustat---
[0] Tesla V100-PCIE-32GB | 70'C, 94 % | 25398 / 32768 MB |
[1] Tesla V100-PCIE-32GB | 68'C, 88 % | 25398 / 32768 MB |
Note that all the GPUs must be on the same machine, not distributed.

---samples_per_gpu---
samples_per_gpu=4

---training command---
./tools/dist_train.sh configs/bsds/EDTER_BIMLA_320x320_80k_bsds_bs_8.py 2

Feel free to contact me if you need more information. @liuxingyu123 @yun-liu

---update---
Note that all the GPUs must be on the same machine, not distributed.
iter_20k: ODS=0.816 OIS=0.832 AP=0.865.

[MengyangPu]

We update the reproduced results in https://github.com/MengyangPu/EDTER#4-the-original-results-vs-the-reproduced-results and provide all results.