Question about the search region update strategy in DaSiamRPN

[xxAna]

Thank you for your sharing. And I wonder if there is the search region update strategy in this project. I try to change the instance size and output size to enlarge the search, while it seems that the result will become even worser. It seems that a large search region imply the target becoming smaller? What is the corret implement of the strategy? Please correct me if I am wrong.Waiting for your reply.

[ZhiyuanChen]

It seems that a large search region imply the target becoming smaller?

Yes

What is the corret implement of the strategy?

It depends on your task. If the object moves really fast, then it would be a great idea to increase the seach region.

[xxAna]

Thank you for your reply. And I find that if I increase the search region, the anchor numbers will also increase, I try 255,300,400 and 500 as the search region size, the performance will drop accordinglly. I drawed several best bboxes and also superimpose the hanning window for better view.

[1]search region size: 255

[2]search region size: 300

[3]search region size: 400

[4]search region size: 510

It do confuses me. As we know, if the target move on a same horizon,or more commonly, the In-Plane movement, it may run fast while the size doesn't change a lot. What can I do in such cases? I am a new beginner in tracking. Please correct me if I am wrong. Waiting for your reply. Thank you again for your warm heart. @zyc-ai

[ZhiyuanChen]

And I find that if I increase the search region, the anchor numbers will also increase.

This is expected, I highly recommend you check out how the anchors are generated

it may run fast while the size doesn't change a lot. What can I do in such cases?

If it was me, I would increase the size and penalty while decrease the movement pentaly.

And in this particular case of basketball, I would also change the search region to a rectangle, so that it won't include those audience.

No worries, glad to be helpful

[xxAna]

Thank you for your reply!
I check the code and reconfirm that to enlarge the search region, I change the cfg.TRACK.INSTANCE_SIZE and recalculate the cfg.TRAIN.OUTPUT_SIZE by using the formula cfg.TRACK.INSTANCE_SIZE/cfg.TRACK.BASE_SIZE-7 as the template only use the 7*7 center feature map.

So the anchors will increases as the output size is enlarged with an enlarged instance size. Am I right? Please correct if I am wrong. And I wonder weather the change in anchor number during inference is also a factor that harm the performance？(So the instance size that closer to that in training, the anchor number won't change too much and the proposal quality will higher.)

If it was me, I would increase the size and penalty while decrease the movement pentaly.
And in this perticualr case of basketball, I would also change the search region to a rectangle, so that it won't include those audience.

Thank you again for your advices on these cases. I still have a question that for an unseen video, how can we detect such cases and implement corresponding measures? I try to retrain the SiamRPN++ and sorry to find that the confidence scores are not always related to its performance(ie,the iou between the predicted bbox and the gt bbox). Eventhough I already set the cfg.DATASET.NEG = 0.2.
It will drop drastic when track a background that looks totally different from the target,(eg, the target was suddendly occluded). While if the target move smoothly out of view, the confidence score will still be high. Does it a common limitation in the siamese tracker? What do you think about it?

Waiting for your reply. Thank you again for your warm heart! @zyc-ai

[ZhiyuanChen]

So the anchors will increases as the output size is enlarged with an enlarged instance size. Am I right? Please correct if I am wrong.

Anchors are generated on the feature map. Hence, no matter what you do, it is fixed (that's why it is called anchor). I strongly encourage you to check out how anchors are generated.

And I wonder weather the change in anchor number during inference is also a factor that harm the performance？(So the instance size that closer to that in training, the anchor number won't change too much and the proposal quality will higher.)

I highly doubt so, since there would be a NMS step which shall elimate all redunant anchors, and if I recall right, only top 300~ anchors will be considered as proposal during testing.

Thank you again for your advices on these cases. I still have a question that for an unseen video, how can we detect such cases and implement corresponding measures? I try to retrain the SiamRPN++ and sorry to find that the confidence scores are not always related to its performance(ie,the iou between the predicted bbox and the gt bbox).

If you have absolutely no idea what scenario you are facing, I'd recommend you do nothing. But of course, if you have time, you may try adding momentum or/and other tricks, but setting a hard constraint is usually not ideal for unseen cases.

Eventhough I already set the cfg.DATASET.NEG = 0.2.

This is one thing you shouldn't do, the weakness of SiamRPN is the ability to distinguish between background and foreground. More negative samples are generally helpful.

It will drop drastic when track a background that looks totally different from the target,(eg, the target was suddendly occluded). While if the target move smoothly out of view, the confidence score will still be high. Does it a common limitation in the siamese tracker? What do you think about it?

I'm a bit confused here, by track a background that looks totally different from the target, did you mean track a target that looks totally different from the background? I also couldn't see any relevance between track a background that looks totally different from the target and the target was suddendly occluded. And, by the target move smoothly out of view, did you mean the target is out of view and tracker tracked something on the background while with high confidence?

[xxAna]

Thank you so much for your patience! After your reply I think twice and so sorry for my late reply. ^-^

So the anchors will increases as the output size is enlarged with an enlarged instance size.

Here, the "anchors" I refer to the total anchor numbers in the responds map. As the search region enlarge, the output responds map will enlarge given a fixed template size, and as the anchor number in each location of the responds map are fixed to 5, therefore the total number of the anchor will increase 5*(new responds size-origin responds size). Please correct me if I am still missing something.

and if I recall right, only top 300~ anchors will be considered as proposal during testing.

emmm,I double check the code and couldn't find the nms operation in the lastest code. I guess maybe some improvements haven't released？If so, looking forward for your updates. ^^

And sorry for my confusing descriptions in the following,

And I wonder weather the change in anchor number during inference is also a factor that harm the performance.

Here, the anchor numbers I still refer to the total number in a responds map, that is 5*(reponds map size) in this code. As the SiamRPN++ mitigate the effects of paddings by adding random shift, it actually isn't a fully convolution network, I wonder weather the enlarge of search region would bring some side effects.

you may try adding momentum or/and other tricks,

As for the "momentum" here, do you refer to the momentum that in lr during inference? If it is, emmm, however, we can't get the gt information during inference and can't get some feedbacks. I search some data and still didn't find a profound interpretation for this trick during inference. Or maybe you refer to some online updating tricks?

More negative samples are generally helpful.

I set cfg.DATASET.NEG = 0.2 according to the config.yaml files in SiamRPN++. And I compare several config.yaml and find their cfg.DATASET.NEG are all 0.2. I try to use 0.3 in a same experiments and seems only trival improvements. Would you please share your latest configuration on it?

target is out of view and tracker tracked something on the background while with high confidence?

Yes. I met some cases of this kind. Could you please provide have any advices? Emmm, maybe improve the discriminative ability of SiamRPN++ is also a solution?

[1]New discovery 1
I find a siamrpnlt_tracker.py in the latest code. Does it include the search region uppdate strategy in DaSiamRPN? It seems I finally find the right file aha? I notice that

Long term lost search size:
cfg.TRACK.LOST_INSTANCE_SIZE = 831
the size is fixed and mach larger that 255 in normal short term tracking.

[2]New discovery 2
I try to add more traning datasets such as lasot and got10k to train SiamRPN++, while it seems can't improvement its performance anymore. Do you think that the original four datasets are already saturation to train the model? Do you have any other tricks exccept param searching and warming up? ^^

I am a new beginner in tracking. Please correct me if I am wrong. Waiting for your reply. Thank you again for your warm heart. @zyc-ai

[SteveTsg]

emmm,I double check the code and couldn't find the nms operation in the lastest code. I guess maybe some improvements haven't released？If so, looking forward for your updates. ^^
Hi, I cloud not find the nms neithe. And I tried increase the Search region, it is better when I track a car. Previously, the bbox will get smaller and smaller.

target is out of view and tracker tracked something on the background while with high confidence?
I changed the coefficient of pos and neg. It perform better, but can not get my expected unfortunately
If you have new idea to fix those problem , could you share please