Support S3 unsigned body #3323
Support S3 unsigned body #3323
Conversation
|
Detected 1 possible performance regressions:
|
| MINIMUM_CORE_VERSION = "3.239.1" | ||
| MINIMUM_CORE_VERSION = "3.240.0" | ||
|
|
||
| # Minimum `aws-sdk-core` version for new S3 gem builds | ||
| MINIMUM_CORE_VERSION_S3 = "3.234.0" | ||
| MINIMUM_CORE_VERSION_S3 = "3.240.0" |
There was a problem hiding this comment.
Self-reminder to update the minimum core version when this is ready for release.
gems/aws-sdk-core/lib/aws-sdk-core/plugins/checksum_algorithm.rb
Outdated
Show resolved
Hide resolved
richardwang1124
left a comment
richardwang1124
left a comment
There was a problem hiding this comment.
Nice! I think it looks good overall to me. Added a few comments.
gems/aws-sdk-core/lib/aws-sdk-core/plugins/checksum_algorithm.rb
Outdated
Show resolved
Hide resolved
gems/aws-sdk-core/lib/aws-sdk-core/plugins/checksum_algorithm.rb
Outdated
Show resolved
Hide resolved
| end | ||
|
|
||
| # Trailer implementation within JRUBY environment is facing some | ||
| # network issues that will need further investigation |
There was a problem hiding this comment.
maybe link to the possible open issue?
| @@ -208,6 +208,7 @@ def download_file(destination, bucket:, key:, **options) | |||
| # @option option [Integer] :http_chunk_size (16384) Size in bytes for each chunk when streaming request bodies | |||
| # over HTTP. Controls the buffer size used when sending data to S3. Larger values may improve throughput by | |||
| # reducing the number of network writes, but use more memory. Custom values must be at least 16KB. | |||
| # Only Ruby MRI is supported. | |||
There was a problem hiding this comment.
The value is still being set on the thread and the patch is being applied right? So the low level http chunk size would still be applied in both cruby and jruby?
| if @trailer_io | ||
| return @trailer_io.read(length, buf) | ||
| def read(length = nil, buf = nil) | ||
| return @io.read unless length |
There was a problem hiding this comment.
I'm a little confused about this - this means we won't use AWS chunked encoding when we're not given a length?
| end | ||
|
|
||
| def fill_chunk | ||
| chunk = @io.read(@base_chunk_size) |
There was a problem hiding this comment.
What happens here is fewer (but non-zero) bytes than the base_chunk_size are returned? (I can't 100% remember Ruby, but I think IO objects are allowed to return <= the requested length when the rest of the bytes aren't yet available)
3 participants
This PR intends to improve performance for S3's
PutObjectandUploadPart.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
To make sure we include your contribution in the release notes, please make sure to add description entry for your changes in the "unreleased changes" section of the
CHANGELOG.mdfile (at corresponding gem). For the description entry, please make sure it lives in one line and starts withFeatureorIssuein the correct format.For generated code changes, please checkout below instructions first:
https://github.com/aws/aws-sdk-ruby/blob/version-3/CONTRIBUTING.md
Thank you for your contribution!