S3用于枚举bucket内容的底层API ListObjectsV2最多返回1000个项目，并且要对结果进行分页，则需要以前调用的opaque值。这意味着从遗传上讲，不可能并行获取多个页面的结果。但是，当您将对象命名为可预测的模式时，如您所建议的，您可以并行请求多个部分。
我唯一的建议是将你请求的部分进一步分割，而不仅仅是第一个字符，以允许超过16个可能的worker。此外，检查预期字符之前和之后的字符，以验证没有预期字符之外的对象。
把它们放在一起看起来像这样：

import boto3
import ctypes
import multiprocessing

def validate_hex_digits(s3, bucket, prefix):
    # Make sure the first character under the prefix is some hex digit
    resp = s3.list_objects_v2(Bucket=bucket, Prefix=prefix, MaxKeys=1)
    assert('0' <= resp['Contents'][0]['Key'][len(prefix)] <= 'f')
    # Make sure there's nothing after 'f'
    resp = s3.list_objects_v2(Bucket=bucket, Prefix=prefix, MaxKeys=1, StartAfter=prefix+'g')
    assert('Contents' not in resp)

def worker(queue, queue_out, jobs):
    # A single worker, pull a job from 'queue', send results to 'queue_out'
    # Use jobs value to track the number of jobs in flight
    s3 = boto3.client('s3')
    while True:
        with jobs.get_lock():
            if jobs.value == 0:
                # Nothing left to do
                queue_out.put(None)
                break
            jobs.value -= 1
            bucket, prefix, token = queue.get()
        # Build up args for the call to list_objects
        args = {
            "Bucket": bucket,
            "Prefix": prefix,
        }
        if token is not None:
            args["ContinuationToken"] = token
        resp = s3.list_objects_v2(**args)
        if 'Contents' in resp:
            queue_out.put(resp['Contents'])
        if 'NextContinuationToken' in resp:
            # There's another page for this prefix, send it off
            # for the next available worker to pick up
            with jobs.get_lock():
                queue.put((bucket, prefix, resp['NextContinuationToken']))
                jobs.value += 1

def main():
    bucket = 'sorel-20m'
    prefix = '09-DEC-2020/binaries/'

    s3 = boto3.client('s3')

    # Verify all of the objects are at least two digit hex digits under the prefix
    validate_hex_digits(s3, bucket, prefix)
    for i in range(16):
        validate_hex_digits(s3, bucket, prefix + f"{i:x}")

    # If we get here, all the keys follow the pattern we expect for at 
    # least two digits.  Go ahead and use multi processing to pull down 
    # the list of objects as fast as possible

    # A queue to store work items
    queue = multiprocessing.Queue()
    # A queue to get pages of results
    queue_out = multiprocessing.Queue()
    # How many jobs are left to process?
    jobs = multiprocessing.Value(ctypes.c_int, 0)
    # Place some seeds in the queue for the first two hex characters
    for i in range(256):
        queue.put((bucket, prefix + f"{i:02x}", None))
        jobs.value += 1
    # Create and start some worker threads, two per process 
    # to allow for network wait time
    workers = multiprocessing.cpu_count() * 2
    procs = []
    for _ in range(workers):
        proc = multiprocessing.Process(target=worker, args=(queue, queue_out, jobs))
        proc.start()
        procs.append(proc)

    # While the workers are working, pull down pages and do something with them
    while workers > 0:
        result = queue_out.get()
        if result is None:
            # Signal that a worker finished
            workers -= 1
        else:
            for cur in result:
                # Just show the results like the AWS CLI does
                print(f"{cur['LastModified'].strftime('%Y-%m-%d %H:%M:%S')} {cur['Size']:10d} {cur['Key'][len(prefix):]}")

    # Clean up
    for proc in procs:
        proc.join()

if __name__ == "__main__":
    main()

在我的机器上，这需要大约5分钟来枚举这个存储桶中的对象，而AWS CLI做同样的事情需要将近一个小时。应该注意的是，结果将以任意顺序排列。如果这是一个问题，则需要进行一些枚举后的排序。除非存储桶所有者启用并发布S3 Inventory Report，你可能无法更快地获得对象列表。