Screenshots upload (deliver) failed with App Store's Rate limit exceeded error about fastlane

[andrewdieken]

I'm facing a similar issue with uploading screenshots but with the fastlane deliver --overwrite_screenshots command

output:

[17:11:33]: Number of screenshots not deleted: 0
[17:11:33]: Successfully deleted all screenshots
[17:11:35]: Uploaded '/Users/user/screenshots/en-US/3apple_screenshot_55.jpeg'... (1.142534 secs)
[17:11:35]: Uploaded '/Users/user/screenshots/en-US/1apple_screenshot_55.jpeg'... (1.184768 secs)
[17:11:35]: Uploaded '/Users/user/screenshots/en-US/1apple_screenshot_65.jpeg'... (1.186299 secs)
[17:11:35]: Uploaded '/Users/user/screenshots/en-US/3apple_screenshot_65.jpeg'... (1.204265 secs)
[17:11:35]: Uploaded '/Users/user/screenshots/en-US/4apple_screenshot_65.jpeg'... (1.234147 secs)
[17:11:35]: Uploaded '/Users/user/screenshots/en-US/5apple_screenshot_55.jpeg'... (1.255372 secs)
[17:11:36]: Uploaded '/Users/user/screenshots/en-US/4apple_screenshot_55.jpeg'... (1.292782 secs)
[17:11:36]: Uploaded '/Users/user/screenshots/en-US/5apple_screenshot_65.jpeg'... (1.294069 secs)
[17:11:36]: Uploaded '/Users/user/screenshots/en-US/2apple_screenshot_65.jpeg'... (1.341499 secs)
[17:11:36]: Uploaded '/Users/user/screenshots/en-US/2apple_screenshot_55.jpeg'... (1.350164 secs)
[⠙] Waiting for all the screenshots to finish being processed...

There was a similar issue reported this past March, #20028, but I have yet to resolve my issue. @alexmarktl could you explain how you were able to remove all languages?

[andrewdieken]

Appears to be a similar if not the same issue described in this issue #20619

[TalRandi1]

@andrewdieken is right.
I got the same issue now.
It has started since I upgraded Fastlane to 2.209.1

[jakub-przy]

Unfortunately my team and I are facing a similar issue:

Uploading complete!
Uploading complete!
Uploading complete!
Uploading complete!
Uploading complete!
Uploading complete!
INFO [2022-09-07 12:23:00.25]: Uploaded 'appstoreconfig/screenshots/en-US/3_iphone6Plus.jpg'... (1.114392 secs)
INFO [2022-09-07 12:23:00.27]: Uploaded 'appstoreconfig/screenshots/en-US/4_iphone6Plus.jpg'... (1.123811 secs)
INFO [2022-09-07 12:23:00.30]: Uploaded 'appstoreconfig/screenshots/en-US/1_iphoneXSmax.jpg'... (1.138808 secs)
INFO [2022-09-07 12:23:00.34]: Uploaded 'appstoreconfig/screenshots/en-US/2_iphoneXSmax.jpg'... (1.160791 secs)
INFO [2022-09-07 12:23:00.38]: Uploaded 'appstoreconfig/screenshots/en-US/3_iphoneXSmax.jpg'... (1.128168 secs)
INFO [2022-09-07 12:23:00.51]: Uploaded 'appstoreconfig/screenshots/en-US/4_iphoneXSmax.jpg'... (1.250281 secs)
DEBUG [2022-09-07 12:23:00.51]: Uploading jobs are completed
INFO [2022-09-07 12:23:00.51]: Waiting for all the screenshots to finish being processed...
DEBUG [2022-09-07 12:23:00.74]: There are still incomplete screenshots - {"UPLOAD_COMPLETE"=>16}
DEBUG [2022-09-07 12:23:06.13]: There are still incomplete screenshots - {"COMPLETE"=>12, "UPLOAD_COMPLETE"=>4}
DEBUG [2022-09-07 12:23:11.47]: There are still incomplete screenshots - {"COMPLETE"=>12, "UPLOAD_COMPLETE"=>4}
DEBUG [2022-09-07 12:23:16.66]: There are still incomplete screenshots - {"COMPLETE"=>12, "UPLOAD_COMPLETE"=>4}
DEBUG [2022-09-07 12:23:21.81]: There are still incomplete screenshots - {"COMPLETE"=>12, "UPLOAD_COMPLETE"=>4}

and then the: "There are still incomplete screenshots[...]" issue keeps hanging for hours until the job times out and fails.

This happens regardless of the region and language for the screenshots etc. Seems to be happening for the better part of last 48hours 😢

[fastlane-bot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest fastlane version and check if that solves the issue. Let us know if that works for you by adding a comment 👍

Friendly reminder: contributions are always welcome! Check out CONTRIBUTING.md for more information on how to help with fastlane and feel free to tackle this issue yourself 💪

This issue will be auto-closed if there is no reply within 1 month.

[nguillot]

The issue still valid, please do not close it.

[drcreazy]

It seems that the The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate.
and the Waiting for all the screenshots to finish being processed... are separate issues.

I've checked if this rate limits are described somewhere and so far found only that:

https://developer.apple.com/forums/thread/110457

I didn't find any fastlane options allowing to slowdown the requests during the execution of fastlane deliver run

The problem I've faced with:

...
[09:36:55]: Uploaded './screenshots/ro/iphoneX_Romanian_09.png'... (3.433104764 secs)

Looking for related GitHub issues on fastlane/fastlane...

Found no similar issues. To create a new issue, please visit:
https://github.com/fastlane/fastlane/issues/new
Run `fastlane env` to append the fastlane environment to your issue
#<Thread:0x000055c0dd098b68 /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:24 run> terminated with exception (report_on_exception is true):
/var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:204:in `handle_response': The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate. (Spaceship::UnexpectedResponse)
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:113:in `get'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/tunes/tunes.rb:493:in `get_app_screenshot_sets'
	from /usr/lib/ruby/2.7.0/forwardable.rb:235:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_screenshot_set.rb:119:in `all'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_store_version_localization.rb:88:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:25:in `block (3 levels) in each_app_screenshot_set'
#<Thread:0x000055c0dd077120 /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:24 run> terminated with exception (report_on_exception is true):
/var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:204:in `handle_response': The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate. (Spaceship::UnexpectedResponse)
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:113:in `get'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/tunes/tunes.rb:493:in `get_app_screenshot_sets'
	from /usr/lib/ruby/2.7.0/forwardable.rb:235:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_screenshot_set.rb:119:in `all'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_store_version_localization.rb:88:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:25:in `block (3 levels) in each_app_screenshot_set'
#<Thread:0x000055c0dd077490 /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:24 run> terminated with exception (report_on_exception is true):
/var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:204:in `handle_response': The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate. (Spaceship::UnexpectedResponse)
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:113:in `get'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/tunes/tunes.rb:493:in `get_app_screenshot_sets'
	from /usr/lib/ruby/2.7.0/forwardable.rb:235:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_screenshot_set.rb:119:in `all'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_store_version_localization.rb:88:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:25:in `block (3 levels) in each_app_screenshot_set'
#<Thread:0x000055c0dd098f00 /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:24 run> terminated with exception (report_on_exception is true):
/var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:204:in `handle_response': The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate. (Spaceship::UnexpectedResponse)
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:113:in `get'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/tunes/tunes.rb:493:in `get_app_screenshot_sets'
	from /usr/lib/ruby/2.7.0/forwardable.rb:235:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_screenshot_set.rb:119:in `all'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_store_version_localization.rb:88:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:25:in `block (3 levels) in each_app_screenshot_set'
#<Thread:0x000055c0dd077ad0 /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:24 run> terminated with exception (report_on_exception is true):
/var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:204:in `handle_response': The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate. (Spaceship::UnexpectedResponse)
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:113:in `get'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/tunes/tunes.rb:493:in `get_app_screenshot_sets'
	from /usr/lib/ruby/2.7.0/forwardable.rb:235:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_screenshot_set.rb:119:in `all'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_store_version_localization.rb:88:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:25:in `block (3 levels) in each_app_screenshot_set'
#<Thread:0x000055c0dd076e00 /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:24 run> terminated with exception (report_on_exception is true):
/var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:204:in `handle_response': The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate. (Spaceship::UnexpectedResponse)
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:113:in `get'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/tunes/tunes.rb:493:in `get_app_screenshot_sets'
	from /usr/lib/ruby/2.7.0/forwardable.rb:235:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_screenshot_set.rb:119:in `all'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_store_version_localization.rb:88:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:25:in `block (3 levels) in each_app_screenshot_set'
#<Thread:0x000055c0dd099158 /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:24 run> terminated with exception (report_on_exception is true):
/var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:204:in `handle_response': The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate. (Spaceship::UnexpectedResponse)
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:113:in `get'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/tunes/tunes.rb:493:in `get_app_screenshot_sets'
	from /usr/lib/ruby/2.7.0/forwardable.rb:235:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_screenshot_set.rb:119:in `all'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_store_version_localization.rb:88:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:25:in `block (3 levels) in each_app_screenshot_set'
#<Thread:0x000055c0dd0985a0 /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:24 run> terminated with exception (report_on_exception is true):
/var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:204:in `handle_response': The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate. (Spaceship::UnexpectedResponse)
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:113:in `get'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/tunes/tunes.rb:493:in `get_app_screenshot_sets'
	from /usr/lib/ruby/2.7.0/forwardable.rb:235:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_screenshot_set.rb:119:in `all'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_store_version_localization.rb:88:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:25:in `block (3 levels) in each_app_screenshot_set'
#<Thread:0x000055c0dd098d20 /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:24 run> terminated with exception (report_on_exception is true):
/var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:204:in `handle_response': The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate. (Spaceship::UnexpectedResponse)
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:113:in `get'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/tunes/tunes.rb:493:in `get_app_screenshot_sets'
	from /usr/lib/ruby/2.7.0/forwardable.rb:235:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_screenshot_set.rb:119:in `all'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_store_version_localization.rb:88:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:25:in `block (3 levels) in each_app_screenshot_set'
/var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:204:in `handle_response': \e[31m[!] The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate.\e[0m (Spaceship::UnexpectedResponse)
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:113:in `get'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/tunes/tunes.rb:493:in `get_app_screenshot_sets'
	from /usr/lib/ruby/2.7.0/forwardable.rb:235:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_screenshot_set.rb:119:in `all'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_store_version_localization.rb:88:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:25:in `block (3 levels) in each_app_screenshot_set'
/var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:204:in `handle_response': The request rate limit has been reached. - We've received too many requests for this API. Please wait and try again or slow down your request rate. (Spaceship::UnexpectedResponse)
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/api_client.rb:113:in `get'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/tunes/tunes.rb:493:in `get_app_screenshot_sets'
	from /usr/lib/ruby/2.7.0/forwardable.rb:235:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_screenshot_set.rb:119:in `all'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/spaceship/lib/spaceship/connect_api/models/app_store_version_localization.rb:88:in `get_app_screenshot_sets'
	from /var/lib/gems/2.7.0/gems/fastlane-2.210.1/deliver/lib/deliver/app_screenshot_iterator.rb:25:in `block (3 levels) in each_app_screenshot_set'

[Updated]

I guess we can try to tune number of threads to slowdown uploading a bit:

https://github.com/fastlane/fastlane/blob/master/deliver/lib/deliver/upload_screenshots.rb#L116
https://github.com/fastlane/fastlane/blob/master/fastlane_core/lib/fastlane_core/queue_worker.rb#L9

[atrebbi]

I'm facing the same issue ; the problem is that the screenshots never end the process on App Store, I have checked after several days and still find screenshots in processing ; and, because of this, I can't submit the app for review

[anh1979]

We are facing the same problem from time to time -> our Jenkins builds get stuck on this until we manually delete the screenshots in question (it is showing grey and with a cloud symbol).
When you delete it, the process wakes up and tries to upload the file again. Most of the times this helps it.
Still annoying.

[maugusto-cit]

Getting the same issue here.
The first attempt delivering dozens of screenshots reached the API rate limit after 1 hour waiting for the images to be processed.
Same with the second attempt.
For the third attempt, now reducing to 1 locale, I got the error right away. I believe I must have reached a hour/day limit.

I also could not find a way to configure Fastlane request rate, as suggested in the error message.