Does torch.nn.Linear check the weights shape before assignment? about pytorch

[ayush0x00]

Hey @lesjie-wen , I am not able to understand your query. Can you provide code snippets where are you getting errors? The snippet you have provided is absolutely fine and works as expected. Please elaborate when l set the output shape to [100, 70] which is the original shape l want this statement.

[lesjie-wen]

Sorry, there is a mistake, what l want to say is as follows, when l initialized input with the shape of [16,70], error comes:

import torch

linear1 = torch.nn.Linear(in_features=70, out_features=100, bias=False)
custom_weights = torch.ones((56, 90))
linear1.weight = torch.nn.Parameter(custom_weights)
print(linear1.weight.shape) 
# torch.Size([56, 90])

input = torch.ones((16, 70))
output = linear1(input)
Traceback (most recent call last):
  File "/Users/lesjie/PycharmProjects/test/test.py", line 10, in <module>
    output = linear1(input)
  File "/Users/lesjie/anaconda3/envs/vima/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/lesjie/anaconda3/envs/vima/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (16x70 and 90x56)

The original shape of input tensor is [N, 70] due to the in_features=70

[ayush0x00]

inear1.weight = torch.nn.Parameter(custom_weights) You are changing the weight matrix of the linear layer over here so the weight of linear1 is no longer [N,70]. It's [56,90] that won't allow multiplication by [16,70].