Support Python 3.8 onwards about amptk

[nextgenusfs]

Multiprocessing appears to have changed in py3.8 -- I didn't have time to diagnose/fix the problem so pinned to <3.8. I will try to find time to fix, but essentially how I'm using multiprocessing fails in v3.8:

def runMultiProgress(function, inputList, cpus, args=False):
    #setup pool
    p = multiprocessing.Pool(cpus)
    #setup results and split over cpus
    tasks = len(inputList)
    results = []
    for i in inputList:
        results.append(p.apply_async(function, args=([i]), kwds={'args':args}))
    #refresh pbar every 5 seconds
    while True:
        incomplete_count = sum(1 for x in results if not x.ready())
        if incomplete_count == 0:
            break
        sys.stdout.write("     Progress: %.2f%% \r" % (float(tasks - incomplete_count) / tasks * 100))
        sys.stdout.flush()
        time.sleep(1)
    p.close()
    p.join()

[nextgenusfs]

Okay a little digging suggests that in py3.8 the default method changed for multiprocessing https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods.

So it seems to keep this working I just needed to explicitly define the start method, ie

def runMultiProgress(function, inputList, cpus, args=False):
    #setup pool
    ctx = multiprocessing.get_context('fork')
    p = ctx.Pool(cpus)
    #setup results and split over cpus
    tasks = len(inputList)
    results = []
    for i in inputList:
        results.append(p.apply_async(function, args=([i]), kwds={'args':args}))
    #refresh pbar every 5 seconds
    while True:
        incomplete_count = sum(1 for x in results if not x.ready())
        if incomplete_count == 0:
            break
        sys.stdout.write("     Progress: %.2f%% \r" % (float(tasks - incomplete_count) / tasks * 100))
        sys.stdout.flush()
        time.sleep(1)
    p.close()
    p.join()

[peterjc]

So should be fixed by 285c3b4 and the release of v1.5.3 👍