Skip to content

Commit

Permalink
fix: usr: Bug fix for access denied error while downloading file. Pul…
Browse files Browse the repository at this point in the history
…l Request #190.

SDK-191: Aws throws Access denied errors in situations where the versioning on
s3 files is enabled and an attempt to download the file is done by specifying the
version(default beahvior by aws boto2). This is seen in boto2 and the workaround is to
retry downloading again without specifying the version of the file.
  • Loading branch information
shahharsh87 committed May 19, 2017
1 parent 6f60dbb commit b4bc114
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion qds_sdk/commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -1342,7 +1342,18 @@ def _is_complete_data_available(bucket_paths, num_result_dir):
raise Exception("Results file not available on s3 yet. This can be because of s3 eventual consistency issues.")
log.info("Downloading file from %s" % s3_path)
if delim is None:
key_instance.get_contents_to_file(fp) # cb=_callback
try:
key_instance.get_contents_to_file(fp) # cb=_callback
except boto.exception.S3ResponseError as e:
if (e.status == 403):
# SDK-191, boto gives an error while fetching the objects using versions which happens by default
# in the get_contents_to_file() api. So attempt one without specifying version.
log.warn("Access denied while fetching the s3 object. Retrying without specifying the version....")
key_instance.open()
fp.write(key_instance.read())
key_instance.close()
else:
raise
else:
# Get contents as string. Replace parameters and write to file.
_read_iteratively(key_instance, fp, delim=delim)
Expand Down

0 comments on commit b4bc114

Please sign in to comment.