Comments (5)
Yup, I got it. The short answer is that we are trying to cancel coroutines
, which are unscheduled tasks, when only Futures can be cancelled. The quick fix is to call task = ensure_future(task)
to ensure the coroutines are actually scheduled tasks which can be cancelled (this would be a no-op if they were already Futures).
The bigger question is whether we should be cancelling them in the first place. It might actually make more sense to wait for them all to complete before raising. But that could end up delaying reporting of an error.
from kubespawner.
If these are futures, won't we then also see additional "CancellationErrors" raised from them when doing .cancel() on them?
You don't generally see CancelledError messages unless you await a future that has been cancelled (usually a bug, since once you've cancelled it, you shouldn't be awaiting it). If you're inside a task being cancelled, there will be no traceback.
For example:
import asyncio
async def printer(name):
for i in range(10):
await asyncio.sleep(1)
print(name)
async def main():
tasks = [
asyncio.ensure_future(
printer(name))
for name in ("1", "2", "3")
]
# create single task wrapping subtasks
gather_task = asyncio.gather(*tasks)
print("waiting")
try:
await asyncio.wait_for(gather_task, timeout=1.5)
except asyncio.TimeoutError:
print("timeout waiting (expected)")
# ensure all the sub-tasks are cancelled
# when wait_for hits timeout, it cancels gather_task
# when gather_task is cancelled, it cancels all of its sub-tasks
assert gather_task.done()
for task in tasks:
assert task.done()
assert task.cancelled()
# any of `await task` here would raise CancelledError,
# but no other
print("all done and cancelled")
if __name__ == "__main__":
asyncio.run(main())
will produce the output:
waiting
1
2
3
timeout waiting (expected)
all done and cancelled
No tracebacks or anything. If any code were to await task
for one of the cancelled tasks after it was cancelled, it would raise CancelledError
from kubespawner.
@minrk I think you have the asyncio experience to more clearly understand what is going on and how to handle it properly, can you help?
I figure there is a difference between a future and a coroutine, and we are using future logic on coroutines here. Do we need the coroutines to be made into futures? Do we need to do .cancel()
if one future of them has errored?
from kubespawner.
Based on https://superfastpython.com/asyncio-gather/#Example_of_Canceling_All_Tasks_in_gather, is the action to first stash the future returned by asyncio.gather in a variable, and then await it, allowing us to do .cancel()
on the future variable?
Hmm... I can't think clearly about this. What is the key motivation for doing the .cancel()
logic in the first place before re-raising? If these are futures, won't we then also see additional "CancellationErrors" raised from them when doing .cancel()
on them?
from kubespawner.
Ah, thank you again @minrk - now I feel quite confident about the cancellation errors!!
from kubespawner.
Related Issues (20)
- Servers being reported as down after hub restart that are not HOT 1
- KubeSpawner `latest` docs is missing some traitlets help HOT 4
- Doc blocks contain dead links to kubernetes API v1.20 HOT 3
- Kill k8s pods earlier when specific errors appear HOT 4
- certificate verify failed: self signed certificate HOT 4
- Options independent of any profile HOT 4
- Kubespawner fails to create service when internal_ssl is activated because of the metadata HOT 1
- KubeSpawner does not respect uid for mounts HOT 1
- Support for callables in env no longer works HOT 6
- Test failures in main branch HOT 1
- Broken Links to Kubernetes-docs in main README HOT 1
- c.KubeSpawner.environment not interpreting callables HOT 2
- JupyterHub Singleuser Port Routing Issue After Restart HOT 3
- Standalone Pods found when Hub pod restart HOT 3
- Cannot build wheel for package due to folder name being different to package name HOT 3
- Are we testing against JupyterHub 1.3?
- Stop hardcoding reflectors to monitor `component: singleuser-server` label
- Create new labels that we can switch over to later HOT 8
- Bring into line with the Kubernetes operator design pattern HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kubespawner.