上传文件到s3时,JMeter抛出错误“签名不匹配”

bjp0bcyl  于 2022-11-09  发布在  其他
关注(0)|答案(2)|浏览(424)

有人能帮我弄清楚到底发生了什么吗?
我正在尝试对一个用户可以上传和下载文件的应用程序进行负载测试。
旅程:
步骤-1:当用户从磁盘中选择一个文件(POST请求)时,它会创建一个fileID和带有uuid的路径。
响应如下所示:{"id":"FILE-VX-1234","path":"uuid/filename.jpg","uri":["s3://{location}/{uuid}/{filename}?endpoint=s3.dualstack.eu-west-1.amazonaws.com"],"state":"OPEN","size":-1,"timestamp":"2020-02-13T10:59:43.146+0000","refreshFlag":1,"storage":"STORAGEID","metadata":{}
步骤2:使用这些(POST请求),它使用带有assesskeyID、secretaccesskey和sessionToken的s3 uri进行响应。
响应如下所示:{"uri":["s3://{accesskeyID}:{secretaccesskey}@{storage location}/{uuid}/{filename}?endpoint=s3.dualstack.eu-west-1.amazonaws.com&sessionToken={security token}"]}
第3步:使用这些和添加的临时参数(日期),PUT请求上传s3桶中的文件。
标题类似于:

Accept: */*

Accept-Encoding: gzip, deflate, br

Accept-Language: en-GB,en-US;q=0.9,en;q=0.8

Authorization: AWS4-HMAC-SHA256 Credential=${accesskeyID}/${currentDate}/{region}/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token;x-amz-user-agent, Signature=${secretaccesskey}

Connection: keep-alive

Content-Length: 145541

Content-Type: image/jpeg

Host: <the host address>

Origin: https://{url}

Referer: https://{url}/upload/

Sec-Fetch-Mode: cors

Sec-Fetch-Site: cross-site

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36

X-Amz-Content-Sha256: UNSIGNED-PAYLOAD

X-Amz-Date:${currentDateInUTC}

x-amz-security-token: ${sessionToken}

X-Amz-User-Agent: aws-sdk-js/2.409.0 callback

错误:<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
我已经验证了在PUT请求中传递的签名(secretaccesskey)、accesskeyID和sessionToken是正确的。

  • 注意:附加参数日期和“{region}/s3/aws4_request,SignedHeaders=主机; x-AMZ-内容-SHA 256; x-amz-日期;x-amz-安全令牌;授权错误中的x-amz-user-agent”是硬编码的。*
628mspwn

628mspwn1#

如果不了解如何为请求生成签名(即Authorization头),就不可能提供任何帮助
根据Signing and Authenticating REST Requests文章
Amazon S3 REST API使用基于密钥HMAC的自定义HTTP方案(散列消息验证代码)。要验证请求,首先将请求的选定元素连接起来形成一个字符串。然后使用AWS秘密访问密钥计算该字符串的HMAC。我们将此过程非正式地称为“对请求进行签名”,并将HMAC算法的输出称为签名。因为它模拟了真实的签名的安全属性。最后,使用本节中描述的语法将此签名作为请求的参数添加。
下面是一个伪代码,演示了如何生成标头:

Authorization = "AWS" + " " + AWSAccessKeyId + ":" + Signature;

Signature = Base64( HMAC-SHA1( YourSecretAccessKey, UTF-8-Encoding-Of( StringToSign ) ) );

StringToSign = HTTP-Verb + "\n" +
    Content-MD5 + "\n" +
    Content-Type + "\n" +
    Date + "\n" +
    CanonicalizedAmzHeaders +
    CanonicalizedResource;

CanonicalizedResource = [ "/" + Bucket ] +
    <HTTP-Request-URI, from the protocol name up to the query string> +
    [ subresource, if present. For example "?acl", "?location", "?logging", or "?torrent"];

CanonicalizedAmzHeaders = <described below>

您可以查看How to Handle Dynamic AWS SigV4 in JMeter for API Testing文章中的示例实现。

vatpfxk5

vatpfxk52#

AWS现在支持签名版本4,该版本需要在将文件上传到S3存储桶时生成。
上载时可以使用两种类型凭据:

  • 服务凭据是以下各项的组合:accessKey、secretKey、服务名称和区域
  • 临时凭证,由以下各项组合而成:访问密钥、保密密钥、服务名称、会话令牌和区域

在这两种情况下,您都需要在HTTP请求采样器下添加JSR 223预处理器。
将以下代码添加到JSR 223预处理器中,并提供所需的详细信息:

import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
import java.security.InvalidKeyException
import java.security.MessageDigest
import groovy.json.JsonSlurper
import java.text.SimpleDateFormat

//Defined in User Defined Variables
def access_key = <access key>
def secret_key = <secret key>
def host = <host>
def service = "s3"
def region = "us-east-1"
def token = <session token>
def localfile = <absolute path for local file>
def fileData = readBinaryFile(localfile)

//Create SHA256 of file data
def contentSha256 = org.apache.commons.codec.digest.DigestUtils.sha256Hex(fileData)
vars.put("aws_sha256", contentSha256)
vars.put("aws_content_length", fileData.length + "")

log.info("access key: " + access_key)
log.info("secret_key: " + secret_key)
log.info("host: " + host)
log.info("service: " + service)
log.info("region: " + region)
log.info("file: " + props.get("file_1.jpg"))
log.info("file sha256: " + contentSha256)
log.info("token: " + token)
log.info("content-length: " + fileData.length)

//Obtain data form the Http Request Sampler
def method = sampler.getMethod()
def url = sampler.getUrl()
def req_path = url.getPath()
def req_query_string = orderQuery(url)
def request_parameters = '';

sampler.getArguments().each {arg ->
    request_parameters = arg.getStringValue().substring(1)
}

//Create the variable x-amz-date 
def now = new Date()
def amzFormat = new SimpleDateFormat( "yyyyMMdd'T'HHmmss'Z'" )
def stampFormat = new SimpleDateFormat( "yyyyMMdd" )
amzFormat.setTimeZone(TimeZone.getTimeZone("UTC"));  //server timezone
def amzDate = amzFormat.format(now)
def dateStamp = stampFormat.format(now)
vars.put("x_amz_date", amzDate)

//Create a Canonical Request
def canonical_uri = req_path
def canonical_querystring = req_query_string
def canonical_headers = "host:" + host + "\n" + "x-amz-date:" + amzDate + "\n"
canonical_headers = "host:" + host + "\n" + "x-amz-content-sha256:" + contentSha256 + "\n" + "x-amz-date:" + amzDate + "\n" + "x-amz-security-token:" + token + "\n"

//def signed_headers = "host;x-amz-date"
def signed_headers = "host;x-amz-content-sha256;x-amz-date;x-amz-security-token"
def payload_hash = getHexDigest(request_parameters)
def canonical_request = method + "\n" + canonical_uri + "\n" + canonical_querystring + "\n" + canonical_headers + "\n" + signed_headers + "\n" + contentSha256

log.info("---------canonical_request-------: " + canonical_request)

//Create the String to Sign
def algorithm = "AWS4-HMAC-SHA256"
def credential_scope = dateStamp + "/" + region + "/" + service + "/" + "aws4_request"
def hash_canonical_request = getHexDigest(canonical_request)
def string_to_sign = algorithm + "\n" +  amzDate + "\n" +  credential_scope + "\n" +  hash_canonical_request

log.info("------string to sign-----: " + string_to_sign)

//Calculate the String to Sign

log.info("------datestamp to sign----: "+ dateStamp)

def signing_key = getSignatureKey(secret_key, dateStamp, region, service)

log.info("------key to sign----: "+ org.apache.commons.codec.digest.DigestUtils.sha256Hex(signing_key))

def signature = hmac_sha256Hex(signing_key, string_to_sign)

//Add Signing information to Variable
def authorization_header = algorithm + " " + "Credential=" + access_key + "/" + credential_scope + ", " +  "SignedHeaders=" + signed_headers + ", " + "Signature=" + signature
vars.put("aws_authorization", authorization_header)

def hmac_sha256(secretKey, data) {
    Mac mac = Mac.getInstance("HmacSHA256")
    SecretKeySpec secretKeySpec = new SecretKeySpec(secretKey, "HmacSHA256")
    mac.init(secretKeySpec)
    byte[] digest = mac.doFinal(data.getBytes())
    return digest
}

def hmac_sha256Hex(secretKey, data) {
    def result = hmac_sha256(secretKey, data)
    return result.encodeHex()
}

def getSignatureKey(key, dateStamp, regionName, serviceName) {
    def kDate = hmac_sha256(("AWS4" + key).getBytes(), dateStamp)
    def kRegion = hmac_sha256(kDate, regionName)
    def kService = hmac_sha256(kRegion, serviceName)
    def kSigning = hmac_sha256(kService, "aws4_request")
    return kSigning
}

def getHexDigest(text) {
    def md = MessageDigest.getInstance("SHA-256")
    md.update(text.getBytes())
    return md.digest().encodeHex()
}

public static String orderQuery(URL url) throws UnsupportedEncodingException {

    def orderQueryString = "";
    Map<String, String> queryPairs = new LinkedHashMap<>();
    String queryParams = url.getQuery();

    if (queryParams != null) {
        String[] pairs = queryParams.split("&");

        for (String pair : pairs) {
            int idx = pair.indexOf("=");
            queryPairs.put(URLDecoder.decode(pair.substring(0, idx), "UTF-8"), URLDecoder.decode(pair.substring(idx + 1), "UTF-8"));
        }
        def orderQueryArray = new TreeMap<String, String>(queryPairs);
        orderQueryString  = urlEncodeUTF8(orderQueryArray)
    }
    return orderQueryString;
}

public static String urlEncodeUTF8(String s) {
    try {
        return URLEncoder.encode(s, "UTF-8");
    } catch (UnsupportedEncodingException e) {
        throw new UnsupportedOperationException(e);
    }
}

public static String urlEncodeUTF8(Map<?,?> map) {
    StringBuilder sb = new StringBuilder();
    for (Map.Entry<?,?> entry : map.entrySet()) {
        if (sb.length() > 0) {
            sb.append("&");
        }
        sb.append(String.format("%s=%s",
            urlEncodeUTF8(entry.getKey().toString()),
            urlEncodeUTF8(entry.getValue().toString())
        ));
    }
    return sb.toString();       
}

public static byte[] readBinaryFile(String filePath) {
    File file = new File(filePath)
    byte[] binaryContent = file.bytes
    return binaryContent
}

将以下参数提供给预处理器:

def access_key = <access key>
def secret_key = <secret key>
def host = <host>
def service = "s3" //is any specific name given for service else it will be s3 by default
def region = "us-east-1" //the required region
def token = <session token>
def localfile = <absolute path for local file>

然后,将HTTP标头管理器添加到HTTP请求采样器中,其中添加了以下标头:

Header Name:    Value
X-Amz-Date: ${x_amz_date}
X-Amz-Security-Token: ${sessionToken}
X-Amz-Content-Sha256: ${aws_sha256}
Authorization: ${aws_authorization}
Content-Length: ${aws_content_length}
Content-Type: <file content type>
Host: ${host}

这里,下面是文件上传时由预处理器动态计算的头值:

  • X-Amz-日期
  • X-Amz-内容-Sha 256
  • 授权书
  • 内容长度

现在,在HTTP请求采样器中设置请求方法(通常为PUT)、协议(例如https)、服务器(... s3.service.com)、端口(例如443)、路径(包括存储桶名称的文件路径)。
单击HTTP请求采样器中的"文件上载“选项卡,分析需要上载的本地文件的绝对路径,并提供mime类型。
注:不要提供参数名称(由于文件将以原始二进制格式传输,因此该名称将为空),并且由于文件将以原始格式传输,因此应取消选中“使用多部分/表单数据”复选框。
为了给予更多帮助,您可以使用JSR223 Sampler通过不同的文件上传/下载方法发出HTTP请求。例如,将动态计算的数据上传为文件。请参考链接:How to pass data from JSR223 sampler to Http Request sampler in JMeter
希望能对你有所帮助。

相关问题