[Feature]: Configurable `networkidle` timeout (other than hard-coded 500ms) about playwright

[pavelfeldman]

We strongly discourage the use of networkidle. It is inherently flaky, buy design, so your scripts are guaranteed to flake or break. It might be not obvious, but you can only use networkidle if you control the page and should never use it on a page from the wild. When you control the page you can make assumptions regarding its network organization, while on the real world pages you can't rely on anything. Every other real world page will have a long poll for ads, so networkidle will never occur. Or will have load / web vitals optimizations that would trigger networkidle before the page is loaded, etc etc.

There is no universal way to detect when the page is loaded. Search engines handle it with some success, exploring SEO will give some clue on how. Web vitals will give some interesting signals that can also be used. Virtual clock and other techniques are also worth exploring.

[ccll]

@pavelfeldman Thanks very much for your explanation and guidance. I will research the topics you mentioned.

[pavelfeldman]

Closing as per above, please feel free to open a new issue if this does not cover your use case.