Scraper stops after ~3-5 minutes about google-maps-scraper

[TEST11R]

bro in windows it doesnt even start, can u help ?

[gosom]

@riku8405
What do you mean it stops after 3-5 minutes ?

This could be normal behaviour .

Please provide more information

[riku8405]

@riku8405 What do you mean it stops after 3-5 minutes ?

This could be normal behaviour .

Please provide more information

Theres no mroe completed jobs it gets stuck

[riku8405]

I tried using IP rotation with 2 machines, so rotating through 2 IPs and this still happens, I dont know if its an issue with the scraper or if I need more please let me know

[gosom]

Do you get any results?

Do you believe that google blocks the requests ?

[riku8405]

I get results, but they stop after about 3-5 minutes, probably but not sure.

[gosom]

can you please provide the input ? In particular I am interested in how many queries your input file contains.

[Neurovert]

@gosom I think it's about the fact that when using your app, "You've reached the end of the list" is shown in the places pane in Google maps - usually it shows no more than just 20 places.
I've tried it many times, always same result - but only with query 'accomodation, Czech Republic'. so maybe I have to query each village separately?
When I'm browsing maps with the same query on my normal browser, with Google account logged in, it loads the places endlessly when scrolling.

EDIT:
I tried going more granular with the query accomodation <city/town/village>, Czech Republic. Of couse now I find much more, but still usually only upto 20 hits.

I'll retry with a logged in Google account.

[darealdemayo]

When i run it with -c 8 or anything higher, it will produce the same error after about 20 minutes for me.
when running with -c 1 it can run for hours without a problem
-c 20 will cause timeouts after less than one minute

I also noticed that when I start the scraper, it takes less than one second per job and it slowly increases to almost 6 seconds per job. the speed at which it increases seems to depend on the concurrency settings.
and all that while top shows ~50-80% cpu and less than 50% ram usage. speedtest shows ping <20ms to google and 100+mbit up/down.

thus, I would guess that what we are seeing here is some sort of rate limitation.

how can we make sure that the browsers run by the scraper are logged into a google account?