Listen to multiple requests when running locally about runpod-python

[justinmerrell]

Is this the functionality you are looking for? Worker | Local API Server Introduced with runpod-python 0.10.0

[holwech]

Yes, this is what I was looking for. I did give it a try but got an error message that made me assume that this required a test file. If that's not the case, I'm not sure exactly why this is happening. When I run python your_handler.py --test_input '{"input": { ... some input }} it works fine, but with the --rp_serve_api, I get the following error outputted in a infinite loop:

python runpod_handler.py --rp_serve_api

...
ERROR  | Error while getting job: 'NoneType' object has no attribute 'get'
WARN   | RUNPOD_WEBHOOK_GET_JOB not set, switching to get_local
WARN   | test_input.json not found, skipping local testing
ERROR  | Error while getting job: 'NoneType' object has no attribute 'get'
WARN   | RUNPOD_WEBHOOK_GET_JOB not set, switching to get_local
WARN   | test_input.json not found, skipping local testing
ERROR  | Error while getting job: 'NoneType' object has no attribute 'get'
...

Stack trace:

Traceback (most recent call last):
  File "/home/user/runpod_handler.py", line 13, in <module>
    runpod.serverless.start({"handler": handler})
  File "/home/user/mambaforge/envs/tensorml/lib/python3.9/site-packages/runpod/serverless/__init__.py", line 75, in start
    asyncio.run(work_loop.start_worker(config))
  File "/home/user/mambaforge/envs/tensorml/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/user/mambaforge/envs/tensorml/lib/python3.9/asyncio/base_events.py", line 634, in run_until_complete
    self.run_forever()
  File "/home/user/mambaforge/envs/tensorml/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
    self._run_once()
  File "/home/user/mambaforge/envs/tensorml/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
    handle._run()
  File "/home/user/mambaforge/envs/tensorml/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/user/mambaforge/envs/tensorml/lib/python3.9/site-packages/runpod/serverless/work_loop.py", line 50, in start_worker
    job: Optional[Dict[str, Any]] = await get_job(session, config)
  File "/home/user/mambaforge/envs/tensorml/lib/python3.9/site-packages/runpod/serverless/modules/job.py", line 48, in get_job
    next_job = _get_local()
  File "/home/user/mambaforge/envs/tensorml/lib/python3.9/site-packages/runpod/serverless/modules/job.py", line 22, in _get_local
    log.warn("test_input.json not found, skipping local testing")
  File "/home/user/mambaforge/envs/tensorml/lib/python3.9/site-packages/runpod/serverless/modules/logging.py", line 59, in warn
    log(message, 'WARN')
  File "/home/user/mambaforge/envs/tensorml/lib/python3.9/site-packages/runpod/serverless/modules/logging.py", line 29, in log
    print(f'{level}| {message}', flush=True)
KeyboardInterrupt

[justinmerrell]

What version of the SDK are you using?

[holwech]

I'm currently on 0.9.12, which is what pip installed by default. Not sure if that's something on my side or not, as I see there are newer versions out. Does it install the newest version for you?

[justinmerrell]

For the new versions, you will need python 3.10+. We are working on expanding backward compatibility.

[holwech]

Thanks, that was it. Definitely a little pain to upgrade the default python version on VMs, so supporting older versions would be useful!