I’m trying to understand what values are being multiplied when I have higher dimensional tensors:

inp = torch.rand(1,2,3) # B,C,W
linear = nn.Linear(3,4)
out = linear(inp)
print(out.shape)
>>> torch.Size([1, 2, 4])

inp = torch.rand(1,2,3,4) # B,C,W,H
linear = nn.Linear(4,5)
out = linear(inp)
print(out.shape)
>>> torch.Size([1, 2, 3, 5])

It seems like only the last dimension is being changed, but when I try to manually multiply the linear weights (linear.weight.data) with each inp‘s last dimension, I can’t get to the correct answer (seems like all the values are changing and only the last dimension’s size is being modify somehow).