S3 upload_file() and upload_fileobj() do not return the object's ETag.

[toojays]

Describe the bug

When uploading an object to S3, the S3 server response includes the object's entity tag (ETag). Boto3's put_object() call returns this ETag. But the upload_file() and upload_fileobj() calls do not.

Steps to reproduce

import boto3
import io

session = boto3.Session(aws_access_key_id=creds["AccessKeyId"], aws_secret_access_key=creds["SecretAccessKey"], aws_session_token=creds["SessionToken"])
s3 = session.resource("s3")
bucket = s3.Bucket("redacted")
fp = io.BytesIO("test".encode("utf-8"))
resp = bucket.upload_fileobj(fp, "test")
print(resp)

The output of this is just None, because upload_fileobj() returns nothing.

Expected behavior

Ideally the above code sequence would output some object containing the ETag.
For example, compare with put_object():

resp = bucket.put_object(Key="test", Body=b"abc")
print(f"ETag from put_object: {resp.e_tag}.")

This outputs:

ETag from put_object: "900150983cd24fb0d6963f7d28e17f72".

Workaround
Monkeypatching a couple of classes in s3transfer as described at boto/s3transfer#82 (comment) gets the response passed back from S3, and I can pull the ETag out of that.

[stobrien89]

Hi @toojays,

Thanks for resurfacing this! Marking as a feature request for now.

[adamsc64]

Hi @toojays and @stobrien89.

I have submitted what I believe is a complete implementation for this. The solution is across two separate pull requests:

• Returned ETag for uploaded entities s3transfer#244
• Returned future result in S3Transfer's download and upload #3316

From what I can tell, due to the nature of the changes in both cases, I believe that the pull requests are not logically coupled and can be merged or released in any order, or independently. In other words, one does not require the other, but when both are released, you should get the desired behavior, i.e. a return value that contains the ETag for the uploaded entity when using upload_file() and upload_fileobj() on a bucket object.

I've never contributed to boto3 or s3transfer in the past. What's the best place for me to find out how to get the review process going for both of these pull requests?

[matteofigus]

@adamsc64 would you be able to return the VersionId too with your Pull Request?

[adamsc64]

@matteofigus Possibly. But I want to keep this PR as simple as possible to get it through review. So unless it's necessary and related to the change, I probably would recommend keeping it as is for now. This doesn't mean there can't be a second PR with the additional information. Do people have opinions?

[adamsc64]

Also, I'm getting complete silence on the two pull requests. Does anyone know how to officially request a pull request review from the boto team?

[skidson]

So a fix has been ready for 2 years but nobody has reviewed or merged it? Is Boto3 actually maintained? This functionality is quite important for common S3 usecases such as passing an uploaded object reference to a consumer in a way that ensures the content hasn't changed since that upload.