araujoalexandre / lipschitz-sll-networks Goto Github PK

Dear Alexandre,

I am trying to train the SLL models on the cifar100 dataset with the option --last_layer 'lln'.

The learned model, which achieves similar performance in the paper, is not 1-Lipschitz. By searching the empirical lower Lipschitz bound

L=norm(model(x+dx)-model(x))/(norm(dx)+1e-6)

with PGD-like algorithm, I obtain L=1.8. I guess the possible reason might be that although Q in the output layer has unit row vectors, its singular value could be still larger than 1.

self.Q = F.normalize(self.weight, p=2, dim=1)

Best regards
Ray

--
Ruigang Wang
[email protected]

Postdoc, Australian Centre for Robotics (ACFR),
University of Sydney, Australia.