Incorrect Base64 encoded x-amz-content-sha256 value

[danielcweeks]

Describe the bug

SigV4 Signing produces incorrect value for checksum header

This line in AbstractAws4Signer is producing base64 encoded values for the x-amz-content-sha256 header instead of the required "Lowercase base 16 encoding." value.

Expected Behavior

The x-amz-content-sha256 header value should be e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Current Behavior

The x-amz-content-sha256 header value currently produced is 47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU= and signature verification fails

Reproduction Steps

Construct signing params

      Aws4SignerParams params =
          Aws4SignerParams.builder()
              .signingName(signingName)
              .signingRegion(signingRegion)
              .awsCredentials(credentialsProvider.resolveCredentials())
              .checksumParams(
                  SignerChecksumParams.builder()
                      .algorithm(Algorithm.SHA256)
                      .isStreamingRequest(false)
                      .checksumHeaderName(SignerConstant.X_AMZ_CONTENT_SHA256)
                      .build())
              .build();

Create a signer

Aws4Signer signer = Aws4Signer.create();

Sign and inspect the header value

SdkHttpFullRequest sdkRequest = SdkHttpFullRequest.builder()....build();
SdkHttpFullRequest signedSdkRequest = signer.sign(sdkRequest, params);

signedSdkRequest.headers().get("x-amz-content-sha256")

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

master

JDK version used

Java 17

Operating System and version

OSC

[debora-ito]

@danielcweeks can you provide a self-contained code example I can use to reproduce the error?
Also provide the stacktrace with the error.

[danielcweeks]

Here's a complete example:

import java.net.URI;
import org.junit.Test;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.signer.Aws4Signer;
import software.amazon.awssdk.auth.signer.internal.SignerConstant;
import software.amazon.awssdk.auth.signer.params.Aws4SignerParams;
import software.amazon.awssdk.auth.signer.params.SignerChecksumParams;
import software.amazon.awssdk.core.checksums.Algorithm;
import software.amazon.awssdk.http.SdkHttpFullRequest;
import software.amazon.awssdk.http.SdkHttpMethod;
import software.amazon.awssdk.regions.Region;

public class TestSigV4Example {

  @Test
  public void testEmptyBodySignature() {
    Aws4SignerParams params =
        Aws4SignerParams.builder()
            .signingName("execute-api")
            .signingRegion(Region.US_WEST_2)
            .awsCredentials(AwsBasicCredentials.create("key", "secret"))
            .checksumParams(
                SignerChecksumParams.builder()
                    .algorithm(Algorithm.SHA256)
                    .isStreamingRequest(false)
                    .checksumHeaderName(SignerConstant.X_AMZ_CONTENT_SHA256)
                    .build())
            .build();

    SdkHttpFullRequest sdkRequestBuilder = SdkHttpFullRequest.builder()
        .method(SdkHttpMethod.GET)
        .protocol("https")
        .uri(URI.create("https://localhots:80/some/path"))
        .build();

    Aws4Signer signer = Aws4Signer.create();
    SdkHttpFullRequest signedSdkRequest = signer.sign(sdkRequestBuilder, params);

    String expected = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855";
    String actual = signedSdkRequest.firstMatchingHeader(SignerConstant.X_AMZ_CONTENT_SHA256).get();

    assertThat(actual).isEqualTo(expected);
  }
}

The result:

expected: "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
 but was: "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU="

[danielcweeks]

According to the SigV4 Documentation

The x-amz-content-sha256 header is required for all AWS Signature Version 4 requests. It provides a hash of the request payload. If there is no payload, you must provide the hash of an empty string.

And

HashedPayload is the hexadecimal value of the SHA256 hash of the request payload.

Hex()	Lowercase base 16 encoding.
SHA256Hash()	Secure Hash Algorithm (SHA) cryptographic hash function.

Hex(SHA256Hash()
If there is no payload in the request, you compute a hash of the empty string as follows:
Hex(SHA256Hash(""))
The hash returns the following value:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

As you can see, the resulting value is not base 16 encoded and is not equal to the required value.

[debora-ito]

@danielcweeks Thank you for the repro code.
When you say the signature verification fails, can you tell us where it fails, with which service? Can you give us more details about your use case?

Asking this because there are different uses for the X_AMZ_CONTENT_SHA256 header that are not documented in the SigV4 documentation page. I see you have a related PR made to Apache Iceberg, I just want to determine if a change is needed and if the change is in scope of the SDK.

[debora-ito]

Brought this up for discussion, the team also want to know more details about the use case.