[0.18.0] Running black causes core dump: src/unix/core.c:633: uv__close: Assertion `fd > 2' failed. about uvloop

[khneal]

Does this help?

STEP 16/16: RUN venv/bin/pystack core --native --locals core.*
Using executable found in the core file: venv/bin/python

Core file information:
state: R zombie: True niceness: 0
pid: 1436 ppid: 1435 sid: 7
uid: 0 gid: 0 pgrp: 7
executable: python arguments: venv/bin/python -m black --verbose --check --diff . 

The process died due receiving signal SIGABRT
Traceback for thread 1436 [Has the GIL] (most recent call last):
    (C) File "???", line 0, in _start (python)
    (C) File "???", line 0, in __libc_start_main (libc.so.6)
    (C) File "???", line 0, in Py_BytesMain (libpython3.8.so.1.0)
    (C) File "???", line 0, in Py_RunMain (libpython3.8.so.1.0)
    (Python) File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
        return _run_code(code, main_globals, None,
      Arguments:
        alter_argv: 1
        mod_name: "black.__main__"
      Locals:
        main_globals: {"__name__": "__main__", "__doc__": None, "__package__": "black", ...}
        mod_spec: <ModuleSpec at 0x7f996f33a610>
        code: <code at 0x7f996f678be0>
    (Python) File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      Arguments:
        script_name: None
        pkg_name: "black"
        run_globals: {"__name__": "__main__", "__doc__": None, "__package__": "black", ...}
        code: <code at 0x7f996f678be0>
        init_globals: None
        mod_name: "__main__"
        mod_spec: <ModuleSpec at 0x7f996f33a610>
      Locals:
        cached: "/approot/venv/lib/python3.8/site-packages/black/__pycache__/__main__.cpython...
        loader: <SourceFileLoader at 0x7f99726ae730>
        fname: "/approot/venv/lib/python3.8/site-packages/black/__main__.py"
    (Python) File "/approot/venv/lib/python3.8/site-packages/black/__main__.py", line 3, in <module>
        patched_main()
    (C) File "???", line 0, in CPyPy_black___patched_main (2ec0e72aa72355e6eccf__mypyc.cpython-38-x86_64-linux-gnu.so)
    (C) File "???", line 0, in CPyDef_black___patched_main (2ec0e72aa72355e6eccf__mypyc.cpython-38-x86_64-linux-gnu.so)
    (Python) File "/approot/venv/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
        return self.main(*args, **kwargs)
      Arguments:
        self: <Command at 0x7f996f33a6d0>
      Locals:
        kwargs: {}
        args: ()
    (Python) File "/approot/venv/lib/python3.8/site-packages/click/core.py", line 1078, in main
        rv = self.invoke(ctx)
      Arguments:
        complete_var: None
        windows_expand_args: True
        standalone_mode: True
        prog_name: "python -m black"
        self: <Command at 0x7f996f33a6d0>
        args: []
      Locals:
        ctx: <Context at 0x7f99725f8310>
        extra: {}
    (Python) File "/approot/venv/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      Arguments:
        ctx: <Context at 0x7f99725f8310>
        self: <Command at 0x7f996f33a6d0>
    (Python) File "/approot/venv/lib/python3.8/site-packages/click/core.py", line 783, in invoke
        return __callback(*args, **kwargs)
      Arguments:
        _Context__callback: <function at 0x7f99725a9430>
        _Context__self: <Context at 0x7f99725f8310>
      Locals:
        ctx: <Context at 0x7f99725f8310>
        args: ()
        kwargs: {"src": ("."), "stdin_filename": None, "config": None, "verbose": True, ...}
    (Python) File "/approot/venv/lib/python3.8/site-packages/click/decorators.py", line 33, in new_func
        return f(get_current_context(), *args, **kwargs)
      Locals:
        kwargs: {"src": ("."), "stdin_filename": None, "config": None, "verbose": True, ...}
        args: ()
    (C) File "???", line 0, in CPyPy_black___main (2ec0e72aa72355e6eccf__mypyc.cpython-38-x86_64-linux-gnu.so)
    (C) File "???", line 0, in CPyDef_black___main (2ec0e72aa72355e6eccf__mypyc.cpython-38-x86_64-linux-gnu.so)
    (Python) File "/approot/venv/lib/python3.8/site-packages/black/concurrency.py", line 97, in reformat_many
        loop = asyncio.new_event_loop()
      Arguments:
        workers: 32
        report: <Report at 0x7f99726ae280>
        sources: <set at 0x7f996f591ac0>
        mode: < at 0x7f996f338ef0>
        write_back: <WriteBack at 0x7f996f6dd8b0>
        fast: False
      Locals:
        executor: <ProcessPoolExecutor at 0x7f996e4c3370>
    (Python) File "/usr/local/lib/python3.8/asyncio/events.py", line 758, in new_event_loop
        return get_event_loop_policy().new_event_loop()
    (Python) File "/usr/local/lib/python3.8/asyncio/events.py", line 656, in new_event_loop
        return self._loop_factory()
      Arguments:
        self: <EventLoopPolicy at 0x7f996f33a970>
    (Python) File "/approot/venv/lib/python3.8/site-packages/uvloop/__init__.py", line 137, in _loop_factory
        return new_event_loop()
      Arguments:
        self: <EventLoopPolicy at 0x7f996f33a970>
    (Python) File "/approot/venv/lib/python3.8/site-packages/uvloop/__init__.py", line 22, in new_event_loop
        return Loop()
    (C) File "uvloop/loop.c", line 152942, in __pyx_tp_new_6uvloop_4loop_Loop (loop.cpython-38-x86_64-linux-gnu.so)
    (C) File "uvloop/loop.c", line 10385, in __pyx_pw_6uvloop_4loop_4Loop_1__cinit__ (inlined) (loop.cpython-38-x86_64-linux-gnu.so)
    (C) File "uvloop/loop.c", line 10642, in __pyx_pf_6uvloop_4loop_4Loop___cinit__ (inlined) (loop.cpython-38-x86_64-linux-gnu.so)
    (C) File "src/unix/loop.c", line 80, in uv_loop_init (loop.cpython-38-x86_64-linux-gnu.so)
    (C) File "src/unix/linux.c", line 590, in uv__platform_loop_init (loop.cpython-38-x86_64-linux-gnu.so)
    (C) File "src/unix/linux.c", line 562, in uv__iou_init (loop.cpython-38-x86_64-linux-gnu.so)
    (C) File "???", line 0, in abort (libc.so.6)
    (C) File "???", line 0, in raise (libc.so.6)

[fantix]

This is probably happening in a situation not reflected in the provided Dockerfile because I cannot reproduce this with either Podman, Docker or a local run. The link to the libuv code (thanks!) is also a very generic fd close call used in many places; thus, I'm unable to locate the issue atm.

It looks like black is using asyncio (uvloop if present) for its concurrency, so it would be great if you could try debugging the crash (a call stack would be nice!) or even better - find a reproducible Python script.

[khneal]

Producing a stacktrace from the core dump is on my to-do list... thank you for your response, and for your patience with me following up.

[nayanmanojgupta]

Any update on this?

[khneal]

Closing... I'm now unable to reproduce the original error.