Retrying same page over and over in some cases about triage-party

[mik-laj]

I have a similar but I don't know if the same problem when trying to fetch data for apache/airflow repository.

I get the impression that the program goes into an infinite loop and can never get all the data.

[wreiske]

👀 I like breaking things. :)

[geekgonecrazy]

Seems to also get stuck downloading the same list of PR's over and over again:

I0916 21:27:59.099728       1 pull_requests.go:79] Downloading open pull requests for rocketchat/rocket.chat (page 2)...
I0916 21:27:59.099814       1 similar.go:81] Updating similarity table from PR cache "rocketchat-rocket.chat-open-prs" (30 items)
I0916 21:27:59.941729       1 pull_requests.go:79] Downloading open pull requests for rocketchat/rocket.chat (page 2)...
I0916 21:27:59.942050       1 similar.go:81] Updating similarity table from PR cache "rocketchat-rocket.chat-open-prs" (30 items)
I0916 21:28:00.845414       1 pull_requests.go:79] Downloading open pull requests for rocketchat/rocket.chat (page 2)...
I0916 21:28:00.845505       1 similar.go:81] Updating similarity table from PR cache "rocketchat-rocket.chat-open-prs" (30 items)
I0916 21:28:02.205749       1 pull_requests.go:79] Downloading open pull requests for rocketchat/rocket.chat (page 2)...
I0916 21:28:02.205990       1 similar.go:81] Updating similarity table from PR cache "rocketchat-rocket.chat-open-prs" (30 items)
I0916 21:28:03.211754       1 pull_requests.go:79] Downloading open pull requests for rocketchat/rocket.chat (page 2)...
I0916 21:28:03.211843       1 similar.go:81] Updating similarity table from PR cache "rocketchat-rocket.chat-open-prs" (30 items)
I0916 21:28:04.190577       1 pull_requests.go:79] Downloading open pull requests for rocketchat/rocket.chat (page 2)...
I0916 21:28:04.190703       1 similar.go:81] Updating similarity table from PR cache "rocketchat-rocket.chat-open-prs" (30 items)
I0916 21:28:05.277635       1 pull_requests.go:79] Downloading open pull requests for rocketchat/rocket.chat (page 2)...
I0916 21:28:05.277678       1 similar.go:81] Updating similarity table from PR cache "rocketchat-rocket.chat-open-prs" (30 items)

Adding the same sort of logic to the pull_requests.go seemed to do the trick also. I don't know what kind of info might be missed because of this. Just treating the symptom basically 🙈

[geekgonecrazy]

Yup seems to try getting the same page over and over blowing through the api limits repeatedly. It wasn’t until I added the check both places that it started working.

I’ll open a PR with the changes I made