Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Decrypting non-encrypted content with AmazonS3EncryptionClientV2 #71

Open
ashishdhingra opened this issue Nov 11, 2024 Discussed in #63 · 3 comments
Open

Decrypting non-encrypted content with AmazonS3EncryptionClientV2 #71

ashishdhingra opened this issue Nov 11, 2024 Discussed in #63 · 3 comments
Labels
bug Something isn't working p2 This is a standard priority issue queued s Effort estimation: small

Comments

@ashishdhingra
Copy link
Contributor

Discussed in #63

Originally posted by simenstensas September 30, 2024
Hi! I've upgraded to the latest preview of AmazonS3EncryptionClientV2 and so far it works out of the box with encrypted content.

However it fails when getting non-encrypted content with message "Amazon.Runtime.AmazonServiceException: Unable to decrypt data for object [object] in bucket [bucket]". Since the metadata contains information about whether the content is encrypted or not, could one just not decrypt it if no headers are found?

If this is not possible, what could be a solution for me?
@ashishdhingra ashishdhingra added needs-review bug Something isn't working queued p2 This is a standard priority issue s Effort estimation: small and removed needs-review labels Nov 11, 2024
@ashishdhingra
Copy link
Contributor Author

ashishdhingra commented Nov 11, 2024 •
edited
Loading

We would need to check:

  • Behavior on how it worked with previous obsolete version of encryption client, and also with Java encryption client.
  • Need to figure out if the behavior was implemented as a result of security practice.
  • If changing behavior would be a breaking change.

@ashishdhingra
Copy link
Contributor Author

ashishdhingra commented Nov 12, 2024 •
edited
Loading

Findings when using Amazon.Extensions.S3.Encryption version 2.2.0
Reproducible using code below:

using Amazon.Extensions.S3.Encryption;
using Amazon.Extensions.S3.Encryption.Primitives;
using Amazon.S3;
using Amazon.S3.Model;

string bucketName = "<<bucket-name>>";
Amazon.AWSConfigs.LoggingConfig.LogResponses = Amazon.ResponseLoggingOption.Always;
Amazon.AWSConfigs.LoggingConfig.LogTo = Amazon.LoggingOptions.Console;
Amazon.AWSConfigs.LoggingConfig.LogMetrics = true;
Amazon.AWSConfigs.AddTraceListener("Amazon", new System.Diagnostics.ConsoleTraceListener());

S3EncryptionTest(bucketName, "<<kms-key-guid>>");

static void S3EncryptionTest(string bucketName, string kmsKeyId)
{
    var encryptionContext = new Dictionary<string, string>();
    var encryptionMaterial =
        new EncryptionMaterialsV2(kmsKeyId, KmsType.KmsContext, encryptionContext);
    var configuration = new AmazonS3CryptoConfigurationV2(SecurityProfile.V2);

    using (AmazonS3Client amazonS3Client = new AmazonS3Client())
    {
        var response = amazonS3Client.PutObjectAsync(new PutObjectRequest()
        {
            BucketName = bucketName,
            Key = "test-notencrypted-file.txt",
            ContentBody = "Testing S3 Encryption"
        }).Result;
    }
    
    using (var encryptionClient = new AmazonS3EncryptionClientV2(configuration, encryptionMaterial))
    {
        EncryptObject(encryptionClient, bucketName, "test-encrypted-file.txt", "Testing S3 Encryption");
        DecryptObject(encryptionClient, bucketName, "test-encrypted-file.txt");
        DecryptObject(encryptionClient, bucketName, "test-notencrypted-file.txt");
    }
}

static void EncryptObject(AmazonS3EncryptionClientV2 encryptionClient, string bucketName, string key, string contentBody)
{
    var putRequest = new PutObjectRequest
    {
        BucketName = bucketName,
        Key = key,
        ContentBody = contentBody
    };

    var response = encryptionClient.PutObjectAsync(putRequest).Result;
}

static void DecryptObject(AmazonS3EncryptionClientV2 encryptionClient, string bucketName, string key)
{
    var getRequest = new GetObjectRequest
    {
        BucketName = bucketName,
        Key = key
    };

    using (var response = encryptionClient.GetObjectAsync(getRequest).Result)
    {
        using (StreamReader reader = new StreamReader(response.ResponseStream))
        {
            string contents = reader.ReadToEnd();
            Console.WriteLine("Contents: " + contents);
        }
    }
}

Throws the exception System.AggregateException: 'System.AggregateException: 'One or more errors occurred. (Unable to decrypt data for object test-notencrypted-file.txt in bucket <<bucket-name>>)''.

Below is the stack trace:

AmazonS3EncryptionClientV2 59|2024-11-11T14:25:30.400Z|ERROR|An exception of type AmazonServiceException was handled in ErrorHandler. --> Amazon.Runtime.AmazonServiceException: Unable to decrypt data for object test-notencrypted-file.txt in bucket <<bucket-name>>
 ---> Amazon.S3.AmazonS3Exception: The specified key does not exist.
 ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.HttpWebRequestMessage.ProcessHttpResponseMessage(HttpResponseMessage responseMessage)
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception, Stream responseStream)
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext executionContext, HttpErrorResponseException exception)
   at Amazon.Runtime.Internal.ExceptionHandler`1.HandleAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.ProcessExceptionAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.S3Express.S3ExpressPreSigner.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Extensions.S3.Encryption.Internal.SetupDecryptionHandler.GetInstructionFileAsync(GetObjectRequest instructionFileRequest) in D:\source\GitHub\amazon-s3-encryption-client-dotnet\src\Internal\SetupDecryptionHandler.cs:line 220
   at Amazon.Extensions.S3.Encryption.Internal.SetupDecryptionHandler.DecryptObjectAsync(Byte[] decryptedEnvelopeKeyKMS, GetObjectResponse getObjectResponse) in D:\source\GitHub\amazon-s3-encryption-client-dotnet\src\Internal\SetupDecryptionHandler.cs:line 202
   --- End of inner exception stack trace ---
   at Amazon.Extensions.S3.Encryption.Internal.SetupDecryptionHandler.DecryptObjectAsync(Byte[] decryptedEnvelopeKeyKMS, GetObjectResponse getObjectResponse) in D:\source\GitHub\amazon-s3-encryption-client-dotnet\src\Internal\SetupDecryptionHandler.cs:line 206
   at Amazon.Extensions.S3.Encryption.Internal.SetupDecryptionHandler.PostInvokeAsync(IExecutionContext executionContext) in D:\source\GitHub\amazon-s3-encryption-client-dotnet\src\Internal\SetupDecryptionHandler.cs:line 151
   at Amazon.Extensions.S3.Encryption.Internal.SetupDecryptionHandler.InvokeAsync[T](IExecutionContext executionContext) in D:\source\GitHub\amazon-s3-encryption-client-dotnet\src\Internal\SetupDecryptionHandler.cs:line 127
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
AmazonS3EncryptionClientV2 60|2024-11-11T14:25:30.432Z|ERROR|AmazonServiceException making request GetObjectRequest to https://testbucket-issue1880.s3.us-east-2.amazonaws.com/. Attempt 1. --> Amazon.Runtime.AmazonServiceException: Unable to decrypt data for object test-notencrypted-file.txt in bucket testbucket-issue1880
 ---> Amazon.S3.AmazonS3Exception: The specified key does not exist.
 ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.HttpWebRequestMessage.ProcessHttpResponseMessage(HttpResponseMessage responseMessage)
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
...

Raw HTTPS response:

HTTP/1.1 404 Not Found
x-amz-request-id: 0FC12N5M2S52JH98
x-amz-id-2: gDohIlO6nQoEL316QyTIZnKXv1Yl95v45etM8Eph3O4WEGCm9Q0gNpjWIah2KggESrVURHFmpS4SCRJFdjp9L2gkCPux3iaNA86SwUcABz4=
Content-Type: application/xml
Date: Tue, 12 Nov 2024 18:05:52 GMT
Server: AmazonS3
Content-Length: 347

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>test-notencrypted-file.txtINSTRUCTION_SUFFIX</Key><RequestId>0FC12N5M2S52JH98</RequestId><HostId>gDohIlO6nQoEL316QyTIZnKXv1Yl95v45etM8Eph3O4WEGCm9Q0gNpjWIah2KggESrVURHFmpS4SCRJFdjp9L2gkCPux3iaNA86SwUcABz4=</HostId></Error>

Testing using Java amazon-s3-encryption-client-java version 3.1.2:

Make sure to add following dependencies in pom.xml (refer Amazon S3 Encryption Client for Java):

<dependency>
  <groupId>software.amazon.awssdk</groupId>
  <artifactId>kms</artifactId>
</dependency>

<dependency>
  <groupId>software.amazon.encryption.s3</groupId>
  <artifactId>amazon-s3-encryption-client-java</artifactId>
  <version>3.1.2</version>
</dependency>

Used below code:

package org.example;

import software.amazon.awssdk.core.ResponseBytes;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectResponse;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.encryption.s3.S3EncryptionClient;

public class Handler {
    public void sendRequest() {
        String bucketName = "<<bucket-name>>";

        String encryptedObjectKey = "test-s3-encryption-key.txt";
        String nonencryptedObjectKey = "test-notencrypted-file.txt";
        String kmsKeyId = "<<kms-key-guid>>";
        S3Client v3Client = S3EncryptionClient.builder()
                .kmsKeyId(kmsKeyId)
                .build();

        v3Client.putObject(PutObjectRequest.builder()
                .bucket(bucketName)
                .key(encryptedObjectKey)
                .build(), RequestBody.fromString("Testing S3 Encryption"));

        ResponseBytes<GetObjectResponse> objectResponse = v3Client.getObjectAsBytes(builder -> builder
                .bucket(bucketName)
                .key(encryptedObjectKey));
        String output = objectResponse.asUtf8String();
        System.out.println(output);

        ResponseBytes<GetObjectResponse> nonencryptedObjectResponse = v3Client.getObjectAsBytes(builder -> builder
                .bucket(bucketName)
                .key(nonencryptedObjectKey));
    }
}

It throws below exception:

Exception in thread "main" software.amazon.encryption.s3.S3EncryptionClientException: Instruction file not found! Please ensure the object you are attempting to decrypt has been encrypted using the S3 Encryption Client.
	at software.amazon.encryption.s3.S3EncryptionClient.getObject(S3EncryptionClient.java:255)
	at software.amazon.awssdk.services.s3.S3Client.getObjectAsBytes(S3Client.java:9542)
	at software.amazon.awssdk.services.s3.S3Client.getObjectAsBytes(S3Client.java:9784)
	at org.example.Handler.sendRequest(Handler.java:50)
	at org.example.App.main(App.java:12)
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: Instruction file not found! Please ensure the object you are attempting to decrypt has been encrypted using the S3 Encryption Client.
	at software.amazon.encryption.s3.internal.ContentMetadataStrategy$1.decodeMetadata(ContentMetadataStrategy.java:49)
	at software.amazon.encryption.s3.internal.ContentMetadataStrategy.decode(ContentMetadataStrategy.java:226)
	at software.amazon.encryption.s3.internal.GetEncryptedObjectPipeline$DecryptingResponseTransformer.onResponse(GetEncryptedObjectPipeline.java:120)
	at software.amazon.encryption.s3.internal.GetEncryptedObjectPipeline$DecryptingResponseTransformer.onResponse(GetEncryptedObjectPipeline.java:91)
	at software.amazon.awssdk.core.async.listener.AsyncResponseTransformerListener$NotifyingAsyncResponseTransformer.onResponse(AsyncResponseTransformerListener.java:87)
	at software.amazon.awssdk.core.internal.http.async.AsyncStreamingResponseHandler.onHeaders(AsyncStreamingResponseHandler.java:55)
	at software.amazon.awssdk.core.internal.http.IdempotentAsyncResponseHandler.onHeaders(IdempotentAsyncResponseHandler.java:103)
	at software.amazon.awssdk.core.internal.http.async.CombinedResponseAsyncHttpResponseHandler.onHeaders(CombinedResponseAsyncHttpResponseHandler.java:58)
	at software.amazon.awssdk.core.internal.http.async.AsyncAfterTransmissionInterceptorCallingResponseHandler.onHeaders(AsyncAfterTransmissionInterceptorCallingResponseHandler.java:67)
	at software.amazon.awssdk.core.internal.http.async.FilterTransformingAsyncHttpResponseHandler.onHeaders(FilterTransformingAsyncHttpResponseHandler.java:44)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage$ReadMetricsTrackingResponseHandler.onHeaders(MakeAsyncHttpRequestStage.java:305)
	at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler.channelRead0(ResponseHandler.java:102)
	at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler.channelRead0(ResponseHandler.java:75)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher.channelRead(HandlerPublisher.java:396)
...
Caused by: software.amazon.awssdk.services.s3.model.NoSuchKeyException: The specified key does not exist. (Service: S3, Status Code: 404, Request ID: 4YCCW3PNJTWFES45, Extended Request ID: u4ixnW71kxhLzVcvzf0pArGvFodvHlcdc3SpAjuIu1CdK1SThw6/HMLd1vXkQ+2xLOMnuY9+jkxSIzJ4wTUy4g==)
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:125)
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:82)
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:60)
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:41)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
	at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$0(BaseSyncClientHandler.java:66)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:60)
	at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:52)
	at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:60)
	at software.amazon.awssdk.services.s3.DefaultS3Client.getObject(DefaultS3Client.java:5203)
	at software.amazon.awssdk.services.s3.S3Client.getObject(S3Client.java:9063)
	at software.amazon.encryption.s3.internal.ContentMetadataStrategy$1.decodeMetadata(ContentMetadataStrategy.java:45)
	... 65 more

Looks like both Java and .NET clients throw the same exception.
For .NET, S3 service is returning <Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>test-notencrypted-file.txtINSTRUCTION_SUFFIX</Key>. Unsure why key is test-notencrypted-file.txtINSTRUCTION_SUFFIX.

Dive Deep:

  • INSTRUCTION_SUFFIX in .NET is declared here.

  • While getting S3 object, INSTRUCTION_SUFFIX in .NET is used here.

  • The logic is able to get object from S3 at SetupDecryptionHandler.InvokeAsync(). The return type is GetObjectResponse.

    • For encrypted object, the raw response contains encrypted payload: 
      HTTP/1.1 200 OK
x-amz-id-2: mYKRjrrSPTUOZvwPZNbRfJNH+3wCD3lPs5Mz707UJyGuHCobbe4eHoWcv96piqVNKQZ51EqpfI0=
x-amz-request-id: 3XGDND5DBQFGB32Y
Date: Tue, 12 Nov 2024 18:51:10 GMT
Last-Modified: Tue, 12 Nov 2024 18:50:58 GMT
ETag: "c7326cf3620ddcc74c303f64f3dbfd42"
x-amz-server-side-encryption: AES256
x-amz-meta-x-amz-tag-len: 128
x-amz-meta-x-amz-unencrypted-content-length: 21
x-amz-meta-x-amz-wrap-alg: kms+context
x-amz-meta-x-amz-matdesc: {"aws:x-amz-cek-alg":"AES/GCM/NoPadding"}
x-amz-meta-x-amz-key-v2: AQIDAHi8yqejJR+QrQe/PgC2s6LGCKcm/5WNl0gJCQcyEZvhKAGP/C6PRG6lEGKMqkFnlSr2AAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQM2PMDbhWgPH5x3+/5AgEQgDtfYbEHpy3Zk1MoIQzZS9Wm7SNCu3C3B/UuMshxmJHT1XcgscZmojwQps5hUkC5Fv7pvHcRm/YfuSLBiQ==
x-amz-meta-x-amz-cek-alg: AES/GCM/NoPadding
x-amz-meta-x-amz-iv: RM4mkwiOPuVAGCds
Accept-Ranges: bytes
Content-Type: text/plain
Content-Length: 37
Server: AmazonS3

 w/	 6 \  %h C  ()-  ` [   v  +  6x
    • For not encrypted object, the raw response contains actual payload: 
      HTTP/1.1 200 OK
x-amz-id-2: 9qVnIBMN3ibXzvc5O9bIXP6WD3HFkWB7gF3ymjN72gk2LU3I0q3OViN9JB5wua6SDJM8K/F1ISLFVCc1yWd62w==
x-amz-request-id: 8NG56RTQZ9PKB9WQ
Date: Tue, 12 Nov 2024 18:52:34 GMT
Last-Modified: Tue, 12 Nov 2024 18:50:56 GMT
ETag: "b8952c8b639021bb47c1fee1d69cde25"
x-amz-server-side-encryption: AES256
Accept-Ranges: bytes
Content-Type: text/plain
Content-Length: 21
Server: AmazonS3

Testing S3 Encryption

  • Thereafter, here it tries to call SetupDecryptionHandler.PostInvokeAsync(), which tries to decrypt response using SetupDecryptionHandler.DecryptObjectAsync().

  • In SetupDecryptionHandler.DecryptObjectAsync(), for not encrypted file, since encrypted metadata is absent:

    So, in network monitoring tool, we would see 2 failed requests with below responses:
    Response 1

    HTTP/1.1 404 Not Found
x-amz-request-id: CJE4JDEHQY2DXYVV
x-amz-id-2: 9Wb00m6Qdn36lezQUat54krE8PN5jhK730dqzvg+TdFNLNYdrSSS/OgfSvKMphPeICY+X8yxbpTfo6T3fSBb0fxNPjr29m5AalYzh1Q3LlA=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Tue, 12 Nov 2024 19:12:39 GMT
Server: AmazonS3

155
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>test-notencrypted-file.txt.instruction</Key><RequestId>CJE4JDEHQY2DXYVV</RequestId><HostId>9Wb00m6Qdn36lezQUat54krE8PN5jhK730dqzvg+TdFNLNYdrSSS/OgfSvKMphPeICY+X8yxbpTfo6T3fSBb0fxNPjr29m5AalYzh1Q3LlA=</HostId></Error>
0

    Response 2

    HTTP/1.1 404 Not Found
x-amz-request-id: CMWSPS4CZ66XXVF8
x-amz-id-2: ktAXd7/l7LlS67ancqdMUeQIuYOyerl4bq2MGwkf9lPM8NmV0qlERawK3foPSjWJBK38lGACXt6szBBICbjgYIoJnOV3Rdd6Jnjsx5spalE=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Tue, 12 Nov 2024 19:12:47 GMT
Server: AmazonS3

15b
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>test-notencrypted-file.txtINSTRUCTION_SUFFIX</Key><RequestId>CMWSPS4CZ66XXVF8</RequestId><HostId>ktAXd7/l7LlS67ancqdMUeQIuYOyerl4bq2MGwkf9lPM8NmV0qlERawK3foPSjWJBK38lGACXt6szBBICbjgYIoJnOV3Rdd6Jnjsx5spalE=</HostId></Error>
0

@ashishdhingra
Copy link
Contributor Author

We had contacted Crypto Tools team (which owns Java S3 Encryption client) and got the below response:

The behavior to return plaintext violates the security guarantees of the library. A threat actor with write access to S3 can replace an encrypted object with a plaintext object, and the GetObject operation succeeds. This violates the integrity guarantee, i.e. that the original plaintext has not replaced with a different plaintext. Therefore, plaintext objects must be handled outside of the security boundary of the S3EC.

The current behavior makes sense. We would work on improving the error messaging to return exception with message like Please ensure the object you are attempting to decrypt has been encrypted using the S3 Encryption Client., rather than Amazon.Runtime.AmazonServiceException: Unable to decrypt data for object [object] in bucket [bucket] when instruction file is not found.

@ashishdhingra ashishdhingra mentioned this issue Dec 18, 2024
Open
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working p2 This is a standard priority issue queued s Effort estimation: small
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ashishdhingra