HopSkipJumpAttack constraint is set to "linf" but is equal "l2"

[uriyapes]

Hi,

I test HopSkipJumpAttack with the following function:

def eval(model, attack="fgsm"):
    fmodel = foolbox.models.PyTorchModel( model, bounds =  get_dataset_min_max_val ("mnist_adversarial"), num_classes=10)
    images, labels = foolbox.utils.samples(dataset='mnist', batchsize=1, data_format='channels_first',
                                           bounds=(0, 1))
    
    attack = foolbox.attacks.HopSkipJumpAttack(fmodel, distance=foolbox.distances.Linf)

    adversarials = attack(images, labels, iterations=2)
    print(np.mean(fmodel.forward(adversarials).argmax(axis=-1) == labels))

I see that although I configured distance=foolbox.distances.Linf, the self.constraint attribute in HopSkipJumpAttack.attack method always equal "l2".

I suspect that the problem occurs because in batching.py line 243:

   attacks = [
        create_attack_fn().as_generator(adv, **kwargs)
        for adv, kwargs in zip(advs, individual_kwargs)
    ]

create_attack_fn which equals HopSkipJumpAttack class don't get any parameters telling it to use distance=Linf.

[jonasrauber]

Thanks for reporting this. The code in batching.py is correct, the distance is correctly passed as part of the Adversarial object. The bug is in the actual attack: https://github.com/bethgelab/foolbox/blob/v2/foolbox/attacks/hop_skip_jump_attack.py#L117
It should read a.distance, not self._default_distance.

[jonasrauber]

Could you try it out locally and open a PR (towards the v2 branch)?

[spencerwooo]

It should read a.distance, not self._default_distance.

I think it should be a._distance. a.distance gives me a TypeError: Comparisons are only possible between the same distance types.