RuntimeError: could not create a primitive descriptor for a matmul primitive about pytorch

[marktansoprano]

I have the same problem on aarch64, when doing basic inference on this model https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0

site-packages/torch/nn/modules/linear.py", line 116, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: could not create a primitive descriptor for a matmul primitive

regardless of torch version, 2.0.1, 2.2.2, 2.3.0, all the same.

The above happens when using bfloat16

pipe = pipeline("text-generation", model="TinyLlama/TinyLlama-1.1B-Chat-v1.0", torch_dtype=torch.bfloat16, device_map="auto")

when change to use float16, no error & works. But takes up enormous memory & inference takes forever

pipe = pipeline("text-generation", model="TinyLlama/TinyLlama-1.1B-Chat-v1.0", torch_dtype=torch.float16, device_map="auto")

[WilliamTambellini]

likely an exception coming from intel oneDNN:
https://oneapi-src.github.io/oneDNN/v1/dev_guide_matmul.html