Concerning exception messages in pytest on Python 3.8 about bionic

[jqmp]

I also get a disturbing error message on shutdown when running a flow in parallel on the command line on Python 3.8.

To reproduce:

1. Use Python 3.8.3.
2. Check out this branch, which adds some parallelism to the ML workflow example.
3. Run the flow with python -m example.ml_workflow --parallel. (You can optionally add -C 1e4,1e2,1,1e-2,1e-4 to add more parallelism.)

The flow completes successfully, but I get one of these two error messages afterwards (it's not consistent which one):

Exception in thread Thread-1:
Traceback (most recent call last):
Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stderr>'> at interpreter shutdown, possibly due to daemon threads
Python runtime state: finalizing (tstate=0x7fd7a54082b0)

Thread 0x000070000b66e000 (most recent call first):
  File "/Users/janek/.pyenv/versions/3.8.3/lib/python3.8/threading.py", line 1202 in invoke_excepthook
  File "/Users/janek/.pyenv/versions/3.8.3/lib/python3.8/threading.py", line 934 in _bootstrap_inner
  File "/Users/janek/.pyenv/versions/3.8.3/lib/python3.8/threading.py", line 890 in _bootstrap

Current thread 0x0000000118dec5c0 (most recent call first):
<no Python frame>
Abort trap: 6

or

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/Users/janek/.pyenv/versions/3.8.3/lib/python3.8/threading.py", line 932, in _bootstrap_inner

[namanjain]

I don't understand what changed in python 3.8 for this to happen but these exceptions are coming from logging receiver thread at shutdown.

For the shorter exception, I see a similar issue previously reported for python 2.7 but that was supposed to be fixed in 3.2. Maybe this is a regression in 3.8. Either way, the exception seems to be incomplete and it's hard to debug.

For the longer exception, it looks like stderr buffer lock was held to be closed during cleanup and this got stuck. I see an issue where a user was using stdin and had the same exception but our case is different since it's happening in the default except hook (default since we didn't change the except hook for the daemon thread).

I'll keep digging to see if there is an easy to fix to these problems. In the meantime, I'll change the thread name to log receiver thread so it's a little more obvious in the future.

[namanjain]

I finally got the entire exception once.

  File "/Users/naman/.pyenv/versions/3.8.3/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/Users/naman/.pyenv/versions/3.8.3/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/naman/Development/bionic/bionic/executor.py", line 170, in _receive
    self.queue.empty()
  File "<string>", line 2, in empty
  File "/Users/naman/.pyenv/versions/3.8.3/lib/python3.8/multiprocessing/managers.py", line 850, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/naman/.pyenv/versions/3.8.3/lib/python3.8/multiprocessing/managers.py", line 250, in serve_client
    self.id_to_local_proxy_obj[ident]
KeyError: '138d2a4c0'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/naman/.pyenv/versions/3.8.3/lib/python3.8/multiprocessing/managers.py", line 252, in serve_client
    raise ke
  File "/Users/naman/.pyenv/versions/3.8.3/lib/python3.8/multiprocessing/managers.py", line 246, in serve_client
    obj, exposed, gettypeid = id_to_obj[ident]
KeyError: '138d2a4c0'
---------------------------------------------------------------------------```

[namanjain]

Fixed in #171