How to use "Testing Models"

[charlescho64]

Hello,
It is fantastic, so cool.
Following to your guide, I try to test pre-trained model but I am not sure my result is right or not.

Step1. which path do these arguments(--exp, --reload) have? "Mine" command line is right, or not? if excite with mine, I don't get any in "out" directory. If you are possible, I hope you show example.
(Guide) python Scripts/prediction/generate_annotation.py --exp <path_to_corresponding_experiment> --reload <path_to_checkpoint> --output_dir <path_to_store_predictions>
(Mine) (env) charles@charles-ubuntu:~/Documents/polyrnnpp/polyrnn-pp-pytorch/code$ python Scripts/prediction/generate_annotation.py --exp Experiments/rl.json --reload ../models/rl_epoch1_step3000.pth --output_dir out/

Step2. which path do these arguments(--pred,--output) have? If you are possible, I hope you show example.
(Guide) python Scripts/get_scores.py --pred <path_to_preds> --output <path_to_file_to_save_results>
(Mine) (env) charles@charles-ubuntu:~/Documents/polyrnnpp/polyrnn-pp-pytorch/code$ python Scripts/get_scores.py --pred out/ --output out/

[charlescho64]

Hello, Help me..

About Step1, I try like below..
.....................
(env) charles@charles-ubuntu:~/Documents/polyrnnpp/polyrnn-pp-pytorch/code$ python Scripts/prediction/generate_annotation.py --exp Experiments/ggnn.json --reload ../models/ggnn_epoch5_step14000.pth --output_dir out/
Path out/ exists. Continue? [y/n]y
Building dataloaders
Dataset Options: {u'img_side': 224, u'random_context': [0.15, 0.15], u'data_dir': u'/ais/gobi5/davidj/cityscapes_project/polyrnn/myWiki/datasets/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': False, u'min_poly_len': 3, u'batch_size': 16, u'flip': False, u'random_start': False, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71}
Dropped 0 multi-component instances
Read 0 instances in val split
Building polyrnnpp with opts:
{u'grad_clip': 40, u'ggnn_n_steps': 5, u'val_freq': 500, u'use_bn_lstm': True, u'dataset': {u'train_val': {u'img_side': 224, u'random_context': [0.15, 0.15], u'data_dir': u'/ais/gobi5/davidj/cityscapes_project/polyrnn/myWiki/datasets/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': False, u'min_poly_len': 3, u'batch_size': 16, u'flip': False, u'random_start': False, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71}, u'train': {u'img_side': 224, u'random_context': [0.1, 0.2], u'data_dir': u'/ais/gobi5/davidj/cityscapes_project/polyrnn/myWiki/datasets/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': True, u'min_poly_len': 3, u'batch_size': 16, u'flip': True, u'random_start': True, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71}}, u'train_encoder': False, u'return_attention': False, u'max_epochs': 200, u'weight_decay': 1e-05, u'max_poly_len': 71, u'temperature': 0.0, u'ggnn_grid_size': 112, u'lr': 5e-05, u'exp_dir': u'/ais/gobi6/amlan/polygon/polyrnnpp_pytorch/ggnn_bs16_5_correction_t03_fine_tune_resnet_lr_5e-5_lr_decay_3', u'ggnn_state_dim': 1024, u'use_separate_encoder': False, u'fp_weight': 200, u'use_correction': True, u'use_ggnn': True, u'use_evaluator': True, u'correction_threshold': 5, u'print_freq': 20, u'xe_initializer': u'/ais/gobi6/amlan/polygon/polyrnnpp_pytorch/evaluator_t0.3_fix_training/checkpoints/epoch19_step56000.pth', u'lr_decay': 3, u'dt_threshold': 2, u'mode': 'test', u'train_attention': False, u'ggnn_output_dim': 15}
Building encoder
Building first vertex network
Building convlstm
Building Evaluator
Building GGNN
Building GGNN Feature Encoder
Reloading full model from: ../models/ggnn_epoch5_step14000.pth
Starting testing
0it [00:00, ?it/s]
.....................

I don't find any warning and error, but there is not file in output_dir.
what is problem?

[amlankar]

The error is here:
Read 0 instances in val split

It didn't find any data from the val split. This is because you haven't updated the data_dir in the options, and it's set to the default data_dir': "u'/ais/gobi5/davidj/cityscapes_project/polyrnn/myWiki/datasets/cityscapes_final_v5'"

[veeramallirajesh]

Hello @amlankar I have a similar issue in testing the model. I have changed the default 'data_dir' to my corresponding local path and get the following error:

python Scripts/prediction/generate_annotation.py --exp Experiments/ggnn.json --reload ../models/ggnn_epoch5_step14000.pth --output_dir ../outputs/
Device being used is: cuda
Path ../outputs/ exists. Continue? [y/n]y
Building dataloaders
Dataset Options: {u'img_side': 224, u'random_context': [0.15, 0.15], u'data_dir': u'/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/data/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': False, u'min_poly_len': 3, u'batch_size': 16, u'flip': False, u'random_start': False, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71}
Dropped 0 multi-component instances
Read 9784 instances in val split
Building polyrnnpp with opts:
{u'grad_clip': 40, u'ggnn_n_steps': 5, u'val_freq': 500, u'use_bn_lstm': True, u'dataset': {u'train_val': {u'img_side': 224, u'random_context': [0.15, 0.15], u'data_dir': u'/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/data/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': False, u'min_poly_len': 3, u'batch_size': 16, u'flip': False, u'random_start': False, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71}, u'train': {u'img_side': 224, u'random_context': [0.1, 0.2], u'data_dir': u'/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/data/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': True, u'min_poly_len': 3, u'batch_size': 16, u'flip': True, u'random_start': True, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71}}, u'train_encoder': False, u'return_attention': False, u'max_epochs': 200, u'weight_decay': 1e-05, u'max_poly_len': 71, u'temperature': 0.0, u'ggnn_grid_size': 112, u'lr': 5e-05, u'exp_dir': u'/ais/gobi6/amlan/polygon/polyrnnpp_pytorch/ggnn_bs16_5_correction_t03_fine_tune_resnet_lr_5e-5_lr_decay_3', u'ggnn_state_dim': 1024, u'use_separate_encoder': False, u'fp_weight': 200, u'use_correction': True, u'use_ggnn': True, u'use_evaluator': True, u'correction_threshold': 5, u'print_freq': 20, u'xe_initializer': u'/ais/gobi6/amlan/polygon/polyrnnpp_pytorch/evaluator_t0.3_fix_training/checkpoints/epoch19_step56000.pth', u'lr_decay': 3, u'dt_threshold': 2, u'mode': 'test', u'train_attention': False, u'ggnn_output_dim': 15}
Building encoder
Building first vertex network
Building convlstm
Building Evaluator
Building GGNN
Building GGNN Feature Encoder
^FReloading full model from: ../models/ggnn_epoch5_step14000.pth
Starting testing
0%| | 0/612 [00:00<?, ?it/s]
Traceback (most recent call last):
File "Scripts/prediction/generate_annotation.py", line 159, in
tester.test()
File "Scripts/prediction/generate_annotation.py", line 143, in test
for step, data in enumerate(tqdm(self.val_loader)):
File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/tqdm/std.py", line 1167, in iter
for obj in iterable:
File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 286, in next
return self._process_next_batch(batch)
File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 307, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
IOError: Traceback (most recent call last):
File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/code/DataProvider/cityscapes.py", line 113, in getitem
return self.prepare_instance(idx)
File "/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/code/DataProvider/cityscapes.py", line 134, in prepare_instance
results = self.prepare_component(instance, component)
File "/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/code/DataProvider/cityscapes.py", line 166, in prepare_component
crop_info = self.extract_crop(component, instance, context_expansion)
File "/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/code/DataProvider/cityscapes.py", line 245, in extract_crop
img = utils.rgb_img_read(instance['img_path'])
File "/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/code/Utils/utils.py", line 55, in rgb_img_read
img = imread(img_path)
File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/skimage/io/_io.py", line 62, in imread
img = call_plugin('imread', fname, plugin=plugin, **plugin_args)
File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/skimage/io/manage_plugins.py", line 214, in call_plugin
return func(*args, **kwargs)
File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/skimage/io/_plugins/pil_plugin.py", line 35, in imread
with open(fname, 'rb') as f:
IOError: [Errno 2] No such file or directory: u'/ais/gobi4/wiki/datasets/cityscapes/leftImg8bit/val/munster/munster_000138_000019_leftImg8bit.png'

Looks like in the image json file the image paths are hardcoded to the default path and at that path no image is found. So, do we have to change the image_path in each json file in ../cityscapes_final_v5/val directory? Could you please give me clear explanation on how to run inference on the own dataset with bounding boxes as well? Thank you.