Converting a numpy array of size larger than 32,768 to a tensor causes a segmentation fault about pytorch

[qqaatw]

Trying to take a look but I was not able to reproduce it with torch nightly on my mac M1 max.

[ZailiWang]

Not reproduced with 0704 nightly on Intel Xeon.

[chadeos]

@qqaatw were you using an osx-arm64 Python install? I ask because I tried again with a Python 3.11.8 osx-arm64 virtual env and got the same result, but then a Python 3.12 built for the old architecture ran without issue. Below is the environment info for each.

Here's the Python 3.11.8 for Apple Silicon:

Collecting environment information...
PyTorch version: 2.3.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 14.5 (arm64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.3.9.4)
CMake version: Could not collect
Libc version: N/A

Python version: 3.11.8 | packaged by conda-forge | (main, Feb 16 2024, 20:49:36) [Clang 16.0.6 ] (64-bit runtime)
Python platform: macOS-14.5-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.3.0
[conda] numpy                     1.26.4          py311h7125741_0    conda-forge
[conda] pytorch                   2.3.0           cpu_py311h29cab49_0

Here's the Intel build:

Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Collecting environment information...
PyTorch version: 2.2.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 14.5 (x86_64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.3.9.4)
CMake version: Could not collect
Libc version: N/A

Python version: 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 10:14:12) [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.2.0
[conda] Could not collect

[qqaatw]

@chadeos Yes, I actually work on main.

PyTorch version: 2.5.0a0+git246732e
Is debug build: True
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.7.4 (arm64)
GCC version: Could not collect
Clang version: 18.1.5
CMake version: version 3.23.2
Libc version: N/A

Python version: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:00:33)  [Clang 13.0.1 ] (64-bit runtime)
Python platform: macOS-12.7.4-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: False

CPU:
Apple M1 Max

Versions of relevant libraries:
[pip3] clip-anytorch==2.5.2
[pip3] flake8==6.1.0
[pip3] flake8-bugbear==23.3.23
[pip3] flake8-comprehensions==3.12.0
[pip3] flake8-executable==2.1.3
[pip3] flake8-logging-format==0.9.0
[pip3] flake8-pyi==23.3.1
[pip3] flake8-simplify==0.19.3
[pip3] mypy==1.10.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.0
[pip3] onnx==1.13.1
[pip3] onnxruntime==1.14.1
[pip3] onnxruntime-tools==1.7.0
[pip3] optree==0.11.0
[pip3] torch==2.5.0a0+git3f43817
[pip3] torchaudio==2.0.1
[pip3] torchdata==0.6.0
[pip3] torchdiffeq==0.2.3
[pip3] torchsde==0.2.5
[pip3] torchtext==0.15.1
[pip3] torchvision==0.19.0a0+f1bcbd3
[conda] clip-anytorch             2.5.2                    pypi_0    pypi
[conda] numpy                     1.26.0                   pypi_0    pypi
[conda] optree                    0.11.0                   pypi_0    pypi
[conda] torch                     2.5.0a0+git3f43817           dev_0    <develop>
[conda] torchaudio                2.0.1                    pypi_0    pypi
[conda] torchdata                 0.6.0                    pypi_0    pypi
[conda] torchdiffeq               0.2.3                    pypi_0    pypi
[conda] torchfix                  0.4.0                    pypi_0    pypi
[conda] torchsde                  0.2.5                    pypi_0    pypi
[conda] torchtext                 0.15.1                   pypi_0    pypi
[conda] torchvision               0.19.0a0+f1bcbd3           dev_0    <develop>

[chadeos]

OK. Checking out the exact version of PyTorch you're using is beyond me, but I did verify I get the same issue with Python 3.9.

[malfet]

@chadeos Can you try torch-2.4 release candidate available at https://download.pytorch.org/whl/test/cpu ?

And can you please run this script with lldb attached and share the backtrace? (I suspect it's OpenMP conflict again)

[chadeos]

Problem is solved with torch-2.4.0-cp312-none-macosx_11_0_arm64.whl.