asyncio `RuntimeError` about ruia

[abmyii]

I find the speed of your merging amazing! Thanks again!

[howie6879]

can you post some code about this issue？ abmyii <[email protected]>于2020年1月16日 周四上午7:10写道：

…

ERROR asyncio Exception in callback BaseSelectorEventLoop._sock_write_done(150)(<Future finished result=None>) handle: <Handle BaseSelectorEventLoop._sock_write_done(150)(<Future finished result=None>)> Traceback (most recent call last): File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run self._context.run(self._callback, *self._args) File "/usr/lib/python3.8/asyncio/selector_events.py", line 516, in _sock_write_done self.remove_writer(fd) File "/usr/lib/python3.8/asyncio/selector_events.py", line 346, in remove_writer self._ensure_fd_no_transport(fd) File "/usr/lib/python3.8/asyncio/selector_events.py", line 251, in _ensure_fd_no_transport raise RuntimeError( RuntimeError: File descriptor 150 is used by transport <_SelectorSocketTransport fd=150 read=idle write=<polling, bufsize=0>> Getting this quite a bit still. I don't think it's ruia directly, but aiohttp. Any ideas? One thing that may be causing it is that in clean functions I call other functions synchronously, i.e.: async def clean_<...>(self, value): return <function>(value) Could that be causing it? I tried doing return await ... but the error still persisted. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#91?email_source=notifications&email_token=AECB6XBWHOZ5WKJW7KWHIJ3Q56JX7A5CNFSM4KHLJAAKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IGPVI2Q>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AECB6XARNDQ4XCTXROXV463Q56JX7ANCNFSM4KHLJAAA> .

[abmyii]

Found what causes the problem. If you use pass both html and url it triggers it, but randomly. This makes me think that there are times when a routine (vague term usage as I am not familiar with the definitions) gets ahead of another somehow or they are out of "alignment". Any ideas?

value = await Field.get_item(html=response.html, url=response.url)

[abmyii]

I think this only happens when there are many concurrent actions (I've set mine to 150).

[howie6879]

Will you avoid this problem by setting the concurrency value lower?

[abmyii]

Will you avoid this problem by setting the concurrency value lower?

No, I fixed it by not having both html=response.html and url=response.url passed to get_item (I thought passing URL was necessary for debugging output to have URL in it, but I've found the new method now that I mentioned in the closed debugging issue).

I'm not sure if this issue is relevant any more, but perhaps there should be an error when both are passed to prevent this from happening any other time?

[howie6879]

I'm not sure if this issue is relevant any more, but perhaps there should be an error when both are passed to prevent this from happening any other time?

How about just use html if someone uses Item.get_item by having both html and url

[howie6879]

By the way, Ruia's test case seems to be wrong

[abmyii]

I'm not sure if this issue is relevant any more, but perhaps there should be an error when both are passed to prevent this from happening any other time?

How about just use html if someone uses Item.get_item by having both html and url

Yes, that would be good. Shall I implement it?

By the way, Ruia's test case seems to be wrong

Which?

[howie6879]

Yes, that would be good. Shall I implement it?

just do it.

Which?

Test cases always fail

[abmyii]

Test cases always fail

Fixed - a simple error (response.url instead of request.url):
4ed265c

just do it.

Done.