marcellodebernardi / loss-landscapes Goto Github PK

Stargazers

Forkers

develooper1994 alpha358 lilujunai wxjiao logancyang qiming-zou keitamasuda xiaoyuwang2821 szalata whw199833 arjunroyihrpa hongxin001 zhanzheng8585 lgoetz cherise215 udemirezen yulonghui zhichaozhu hongqwang xiaolin0520 mountains-high franciscoliu jiaqing-asu a-r-j namkyunghun jiapengwu himidev m1pro chenbin99 sinjini77 irish-kw tobiaslv ghweber sibnick jithamanyu001 sunwoong-yang adebayobraimah balditommaso fwao yifanycc beeeam riverback ashishjv1 shdchen andssy whuhxb

    def filter_normalize_(self, ref_point: 'ModelParameters', order=2):
        """
        In-place filter-wise normalization of the tensor.
        :param ref_point: use this model's filter norms, if given
        :param order: norm order, e.g. 2 for L2 norm
        :return: none
        """
        for l in range(len(self.parameters)):
            # normalize one-dimensional bias vectors
            if len(self.parameters[l].size()) == 1:
                self.parameters[l] *= (ref_point.parameters[l].norm(order) / self.parameters[l].norm(order))
            else:
                # normalize two-dimensional weight vectors
                for f in range(len(self.parameters[l])):
                    self.parameters[l][f] *= ref_point.filter_norm((l, f), order) / (self.filter_norm((l, f), order))