Wrong Answer index is -1. Error at evaluation

[ayushgithub]

In the preprocessing script, if the answer is not in top 1000 it is removed from training example but is kept in test set and is encoded as -1. When we evaluate this throws assertion error in the criterion as it need target from 1 to max number. Is this handled in the code, cause I was not able to find out where.

[mrfarazi]

I am having the same problem. I did change the wrong answer to '1001' from '-1' by updating the prepo_vqa.py script following one earlier suggestions. Still haveing the error t >= 0 && t < n_classes failed.
THCudaCheck FAIL file=/tmp/luarocks_cutorch-scm-1-8061/cutorch/lib/THC/generic/THCStorage.c line=32 error=59 : device-side assert triggered

If anyone has a solution please do share.

[panfengli]

Since when I changed the label for error as 0, the error still existed. So I finally set the --num_ans in prepro_vqa.py as 999 and make a little change as:

# get top answers
top_ans = get_top_answers(imgs_train, params)
atoi = {w:i+1 for i,w in enumerate(top_ans)}
atoi['error label'] = 1000 # ClassNLLCriterion.cu: Assertion `t >= 0 && t < n_classes`
itoa = {i+1:w for i,w in enumerate(top_ans)}
itoa[1000] = 'error label' # ClassNLLCriterion.cu: Assertion `t >= 0 && t < n_classes`

def encode_answer(imgs, atoi):
    N = len(imgs)
    ans_arrays = np.zeros(N, dtype='uint32')

    for i, img in enumerate(imgs):
        #ans_arrays[i] = atoi.get(img['ans'], -1) # -1 means wrong answer.
        ans_arrays[i] = atoi.get(img['ans'], 1000) # 1000 means wrong answer.

    return ans_arrays

[panfengli]

I think the reason why it still shows error when change the label for error to 1001 or 0 is that after adding another class 0 or 1001, the total class will increase to 1001, which will influence a lot of other settings (or not satisfy t<n_classes)

[mrfarazi]

I tried with 1001 but total class 1000 it is hard coded sometimes, so changing it to a different value was creating some problems.

Thanks for your solution. It's working now. Cheers!

[Vapourain3]

@panfengli It can work when I change -1 to 1001. But the accuracy on validation set is about 50%. There is a big gap compare to the result in the paper. Have you met a similar situation？

[mrfarazi]

@panfengli Hello, For some strange reason train.lua is not running after 1 iteration. Its just generating the json file and comes to a standstill. Any ideas?

[shenzhi666]

@Vapourain3 now I have the same proble,can you tell me how to solve it?