Fails to format multiple files on windows about ufmt

[zsol]

Hah, turns out the Windows debug binary overflows the stack when parsing relatively complex files (like the generated code in libcst itself). Release build works fine (which is why you couldn't repro 😄 ). There's no feedback from the OS/Python when this happens:

(libcst) PS C:\Users\zsolz\projects\libcst\libcst\testing> python
Python 3.9.5 (tags/v3.9.5:0a7dcbd, May  3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import libcst as cst
>>> f = open("utils.py").read()
>>> cst.parse_module(f)
(libcst) PS C:\Users\zsolz\projects\libcst\libcst\testing>

(I had to run the rust test binary -in debug mode- directly to observe the stack overflow).

Thanks again, this is not an issue in ufmt/usort.

[amyreese]

I can't replicate this error on either Python 3.8.6 or 3.9.9. What version of Windows are you testing on? Do you still see this error if you upgrade ufmt to 1.3.1post1? Can you provide a paste of pip list?

[zsol]

This is Windows 10, Python 3.9.5, and here's the output of pip list: https://gist.github.com/zsol/4d1ade439bb96693da50c3be1e205cdc

It happens with ufmt 1.3.1post1 too. I can try different python versions a bit later. Just plain black works fine

[amyreese]

Hmm, I still can't replicate this, even with 3.9.5 and installing libcst from the same rev listed in your gist. I even tried installing rustc, rebuilding libcst, and setting LIBCST_PARSER_TYPE=native, and was still able to ufmt --debug format libcst/ without an error. I am running this via git-bash, but I really hope that's not a determining factor in this. 😅

[amyreese]

My only thought is that something is causing the rust parser in libcst to segfault in a child process (or otherwise abort without raising an exception). If there's a specific file that triggers this behavior, you could try something like this to run usort in the parent process and hopefully get some better segfault/debugging out of it:

from usort import usort_file
from pathlib import Path
result = usort_file(Path("libcst/__init__.py"))
print(result.error)

[zsol]

My only thought is that something is causing the rust parser in libcst to segfault in a child process (or otherwise abort without raising an exception).

Yeah this seems plausible, because I can no longer reproduce the issue if I disable the rust parser. I'll poke around, thanks for the tips!