Client.head_object raises ClientError instead of NoSuchKey about botocore

[tim-finnigan]

Thanks for reaching out - this has come up a few times before, I'll share the recent comment that I made here on a related issue to provide more context:

...Related issues have come up here a few times before, for example boto/boto3#2499. Both the HeadObject and HeadBucket APIs return 404 errors if the object or bucket does not exist. And as noted in the documentation:

A HEAD request has the same options as a GET operation on an object. The response is identical to the GET response except that there is no response body. Because of this, if the HEAD request generates an error, it returns a generic code, such as 400 Bad Request, 403 Forbidden, 404 Not Found, 405 Method Not Allowed, 412 Precondition Failed, or 304 Not Modified. It's not possible to retrieve the exact exception of these error codes.

So this is the behavior of the underlying S3 API and therefore not something that will be addressed in Boto3. As mentioned in previous issues it would be a breaking change to update the model. But please use the Provide feedback at the bottom of API documentation pages to send any additional feedback directly to the S3 team.

[vlaci]

I've been digging a bit further and finally figured out why botocore is unable to handle these cases for HEAD request.

I was initially under the impression that based on the current schema, it is possible for botocore to "recover" the error code from the HTTP status code given that all HTTP status codes can be mapped unambiguously to error codes (per endpoint). Unfortunately, this is not the case. Although there are only 3(!) HEAD requests in the whole API and 2 of them are HeadObject and HeadBucket which has a single error code specified, but Media Store's DescribeObject has 2 error cases¹ which result in HTTP 404 status codes (in addition to a bunch of common 400 errors), so my assumption is proven wrong.

Footnotes

1. https://docs.aws.amazon.com/mediastore/latest/apireference/API_objstore_DescribeObject.html#API_objstore_DescribeObject_Errors ↩