Performance issues about scraper

[teymour-aldridge]

I don't think so – unless this is something that scraper is doing wrong (rather than html5ever) – if you try that example with html5ever directly (scraper is just a set of convenience wrappers over https://github.com/servo/html5ever) do you find that there is an increase in speed?

Apart from this you might find that using a multiprocessing library such as rayon could speed up your program drastically.

[let4be]

Time spent per page is completely within normal bounds I'd expect,

I'm currently working on a toy project - implementing broad web crawler in rust, here is one curious stat(ran on i9 10900k, each CPU will have this a bit different)

What you need to do is use multiple threads either via rayon or directly with std::thread

[nathaniel-daniel]

If you only care about parsing and if the data you extract is simple enough, you might have better luck with lol-html.

[teymour-aldridge]

What @nathaniel-daniel said, and also it's not unexpected that in that program that parsing would take 97% of the running time of your program, given that your program is primarily parsing (with a little io). As for the 9ms figure, that seems fine – how big are your documents?