MantaObjectOutputStreamIT tests fail when using AES128/CBC/PKCS5Padding #530
Description
When running the integration tests with MantaObjectOutputStreamIT they fail when executing using the encryption settings AES128/CBC/PKCS5Padding. See below:
nta.client.MantaObjectOutputStreamIT
canUploadMuchLargerFile
com.joyent.manta.exception.MantaIOException: An exception was thrown within the thread writing to the network socket
Exception Context:
[1:mantaSdkVersion=3.4.1-SNAPSHOT]
[2:path=/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-3c285f67-3635-4c19-bcaf-46d4beca52a9.txt]
---------------------------------
at com.joyent.manta.client.MantaObjectOutputStream.close(MantaObjectOutputStream.java:369)
at com.joyent.manta.client.MantaObjectOutputStreamIT.canUploadMuchLargerFile(MantaObjectOutputStreamIT.java:129)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.joyent.manta.exception.MantaIOException: Unable to update metadata with original plaintext content length
Exception Context:
[1:mantaSdkVersion=3.4.1-SNAPSHOT]
[2:requestId=7983de30-819b-11e9-b738-d313687ec84e]
[3:request=HttpPut[entity=com.joyent.manta.http.entity.NoContentEntity@619c3512,version=<null>,uri=https://us-east.manta.joyent.com/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-3c285f67-3635-4c19-bcaf-46d4beca52a9.txt?metadata=true,config=<null>,aborted=false,cancellableRef=com.joyent.manta.org.apache.http.impl.execchain.ConnectionHolder@117f4a2a,headergroup=[m-encrypt-key-id: integration-test-key, m-encrypt-plaintext-content-length: 6127332, m-encrypt-iv: vCfCY+Ns66GcBujaMYzyVA==, if-unmodified-since: Tue May 28 22:53:56 UTC 2019, m-encrypt-metadata: q3YnKU5RNh3IxrtAaLPwUTraU5L4N9eN9n1psJLjAmg=, m-encrypt-metadata-hmac: ln+y3sdzM7/ktKlG2xyqnA==, m-encrypt-type: client/1, m-encrypt-cipher: AES128/CBC/PKCS5Padding, m-encrypt-metadata-iv: FFo0GFhjz8qgtCDyyWP00A==, if-match: c9d98d38-fbe9-eeee-fa35-c4c2c48e3a77, m-encrypt-hmac-type: HmacMD5],params=[parameters={}]]]
[4:requestMethod=PUT]
[5:requestURL=https://us-east.manta.joyent.com/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-3c285f67-3635-4c19-bcaf-46d4beca52a9.txt?metadata=true]
[6:requestHeaders=m-encrypt-key-id: integration-test-key, m-encrypt-plaintext-content-length: 6127332, m-encrypt-iv: vCfCY+Ns66GcBujaMYzyVA==, if-unmodified-since: Tue May 28 22:53:56 UTC 2019, m-encrypt-metadata: q3YnKU5RNh3IxrtAaLPwUTraU5L4N9eN9n1psJLjAmg=, m-encrypt-metadata-hmac: ln+y3sdzM7/ktKlG2xyqnA==, m-encrypt-type: client/1, m-encrypt-cipher: AES128/CBC/PKCS5Padding, m-encrypt-metadata-iv: FFo0GFhjz8qgtCDyyWP00A==, if-match: c9d98d38-fbe9-eeee-fa35-c4c2c48e3a77, m-encrypt-hmac-type: HmacMD5]
[7:loadBalancerAddress=165.225.164.10]
[8:response=HttpResponseProxy[original=HTTP/1.1 403 Forbidden [Content-Type: application/json, Content-Length: 241, Content-MD5: +qyQGBMRliw/Ohl4ObkzJA==, Date: Tue, 28 May 2019 22:53:57 GMT, Server: Manta, x-request-id: 7983de30-819b-11e9-b738-d313687ec84e, x-response-time: 4, x-server-name: 562a0e29-5024-4482-8f6c-26e0d95c5a36, Connection: keep-alive, x-load-balancer: 165.225.164.10] ResponseEntityProxy{[Content-Type: application/json,Content-Length: 241,Chunked: false]},connHolder=com.joyent.manta.org.apache.http.impl.execchain.ConnectionHolder@117f4a2a]]
[9:responseHeaders=Content-Type: application/json, Content-Length: 241, Content-MD5: +qyQGBMRliw/Ohl4ObkzJA==, Date: Tue, 28 May 2019 22:53:57 GMT, Server: Manta, x-request-id: 7983de30-819b-11e9-b738-d313687ec84e, x-response-time: 4, x-server-name: 562a0e29-5024-4482-8f6c-26e0d95c5a36, Connection: keep-alive, x-load-balancer: 165.225.164.10]
[10:responseStatusCode=403]
[11:responseStatusReason=Forbidden]
---------------------------------
at com.joyent.manta.http.EncryptionHttpHelper.appendPlaintextContentLength(EncryptionHttpHelper.java:1021)
at com.joyent.manta.http.EncryptionHttpHelper.httpPut(EncryptionHttpHelper.java:280)
at com.joyent.manta.client.MantaObjectOutputStream.lambda$new$0(MantaObjectOutputStream.java:180)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
... Removed 10 stack frames
canUploadMuchLargerFileWithPeriodicWaits
com.joyent.manta.exception.MantaIOException: An exception was thrown within the thread writing to the network socket
Exception Context:
[1:mantaSdkVersion=3.4.1-SNAPSHOT]
[2:path=/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-f8e1bf2c-8594-4bd8-91a6-08c1b31dfdce.txt]
---------------------------------
at com.joyent.manta.client.MantaObjectOutputStream.close(MantaObjectOutputStream.java:369)
at com.joyent.manta.client.MantaObjectOutputStreamIT.canUploadMuchLargerFileWithPeriodicWaits(MantaObjectOutputStreamIT.java:177)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.joyent.manta.exception.MantaIOException: Unable to update metadata with original plaintext content length
Exception Context:
[1:mantaSdkVersion=3.4.1-SNAPSHOT]
[2:requestId=84cb22d0-819b-11e9-9b9e-838d2080107c]
[3:request=HttpPut[entity=com.joyent.manta.http.entity.NoContentEntity@619c3512,version=<null>,uri=https://us-east.manta.joyent.com/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-f8e1bf2c-8594-4bd8-91a6-08c1b31dfdce.txt?metadata=true,config=<null>,aborted=false,cancellableRef=com.joyent.manta.org.apache.http.impl.execchain.ConnectionHolder@4b60f974,headergroup=[m-encrypt-key-id: integration-test-key, m-encrypt-plaintext-content-length: 6231236, m-encrypt-iv: MC6kfQc/pAEczY1b4jSbjg==, if-unmodified-since: Tue May 28 22:54:15 UTC 2019, m-encrypt-metadata: g1Wt5+iuo65NlNYCDt2tbQFys9hH3XMdJVuh4oPAyt4=, m-encrypt-metadata-hmac: 2jyAayuxwD7PWheuTQrtZw==, m-encrypt-type: client/1, m-encrypt-cipher: AES128/CBC/PKCS5Padding, m-encrypt-metadata-iv: ShL0lGo9kzO/77Q2fhxpBQ==, if-match: 414186f1-ce39-c5c6-ad37-b59c23aa135c, m-encrypt-hmac-type: HmacMD5],params=[parameters={}]]]
[4:requestMethod=PUT]
[5:requestURL=https://us-east.manta.joyent.com/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-f8e1bf2c-8594-4bd8-91a6-08c1b31dfdce.txt?metadata=true]
[6:requestHeaders=m-encrypt-key-id: integration-test-key, m-encrypt-plaintext-content-length: 6231236, m-encrypt-iv: MC6kfQc/pAEczY1b4jSbjg==, if-unmodified-since: Tue May 28 22:54:15 UTC 2019, m-encrypt-metadata: g1Wt5+iuo65NlNYCDt2tbQFys9hH3XMdJVuh4oPAyt4=, m-encrypt-metadata-hmac: 2jyAayuxwD7PWheuTQrtZw==, m-encrypt-type: client/1, m-encrypt-cipher: AES128/CBC/PKCS5Padding, m-encrypt-metadata-iv: ShL0lGo9kzO/77Q2fhxpBQ==, if-match: 414186f1-ce39-c5c6-ad37-b59c23aa135c, m-encrypt-hmac-type: HmacMD5]
[7:loadBalancerAddress=72.2.112.194]
[8:response=HttpResponseProxy[original=HTTP/1.1 403 Forbidden [Content-Type: application/json, Content-Length: 241, Content-MD5: 4fcYgjJh+h/Qhzw31MqtWA==, Date: Tue, 28 May 2019 22:54:15 GMT, Server: Manta, x-request-id: 84cb22d0-819b-11e9-9b9e-838d2080107c, x-response-time: 3, x-server-name: 3d2b5d91-5cd9-4123-89a5-794f44eab9fd, Connection: keep-alive, x-load-balancer: 72.2.112.194] ResponseEntityProxy{[Content-Type: application/json,Content-Length: 241,Chunked: false]},connHolder=com.joyent.manta.org.apache.http.impl.execchain.ConnectionHolder@4b60f974]]
[9:responseHeaders=Content-Type: application/json, Content-Length: 241, Content-MD5: 4fcYgjJh+h/Qhzw31MqtWA==, Date: Tue, 28 May 2019 22:54:15 GMT, Server: Manta, x-request-id: 84cb22d0-819b-11e9-9b9e-838d2080107c, x-response-time: 3, x-server-name: 3d2b5d91-5cd9-4123-89a5-794f44eab9fd, Connection: keep-alive, x-load-balancer: 72.2.112.194]
[10:responseStatusCode=403]
[11:responseStatusReason=Forbidden]
---------------------------------
at com.joyent.manta.http.EncryptionHttpHelper.appendPlaintextContentLength(EncryptionHttpHelper.java:1021)
at com.joyent.manta.http.EncryptionHttpHelper.httpPut(EncryptionHttpHelper.java:280)
at com.joyent.manta.client.MantaObjectOutputStream.lambda$new$0(MantaObjectOutputStream.java:180)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
... Removed 10 stack frames
canUploadSmallString
com.joyent.manta.exception.MantaIOException: An exception was thrown within the thread writing to the network socket
Exception Context:
[1:mantaSdkVersion=3.4.1-SNAPSHOT]
[2:path=/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-83e31e99-cbf4-4917-8f3a-4f4163eedfcb.txt]
---------------------------------
at com.joyent.manta.client.MantaObjectOutputStream.close(MantaObjectOutputStream.java:369)
at com.joyent.manta.client.MantaObjectOutputStreamIT.canUploadSmallString(MantaObjectOutputStreamIT.java:71)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.joyent.manta.exception.MantaIOException: Unable to update metadata with original plaintext content length
Exception Context:
[1:mantaSdkVersion=3.4.1-SNAPSHOT]
[2:requestId=85370400-819b-11e9-802b-7f3f9cebfcc2]
[3:request=HttpPut[entity=com.joyent.manta.http.entity.NoContentEntity@619c3512,version=<null>,uri=https://us-east.manta.joyent.com/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-83e31e99-cbf4-4917-8f3a-4f4163eedfcb.txt?metadata=true,config=<null>,aborted=false,cancellableRef=com.joyent.manta.org.apache.http.impl.execchain.ConnectionHolder@5bd5f18e,headergroup=[m-encrypt-key-id: integration-test-key, m-encrypt-plaintext-content-length: 25, m-encrypt-iv: iNbIFCzBLY0j2Hidm+0M1Q==, if-unmodified-since: Tue May 28 22:54:16 UTC 2019, m-encrypt-metadata: nRZJ/GtgNwfPjk5y7x5dHHKF+Lp73eufQvcYaJpPM5U=, m-encrypt-metadata-hmac: OdG3NRmQNEdeYt9MDurrXA==, m-encrypt-type: client/1, m-encrypt-cipher: AES128/CBC/PKCS5Padding, m-encrypt-metadata-iv: BqFYM3Oco4A391Eg+ot1mg==, if-match: 816610dc-915f-4d36-b482-a9d2f6e2e92a, m-encrypt-hmac-type: HmacMD5],params=[parameters={}]]]
[4:requestMethod=PUT]
[5:requestURL=https://us-east.manta.joyent.com/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-83e31e99-cbf4-4917-8f3a-4f4163eedfcb.txt?metadata=true]
[6:requestHeaders=m-encrypt-key-id: integration-test-key, m-encrypt-plaintext-content-length: 25, m-encrypt-iv: iNbIFCzBLY0j2Hidm+0M1Q==, if-unmodified-since: Tue May 28 22:54:16 UTC 2019, m-encrypt-metadata: nRZJ/GtgNwfPjk5y7x5dHHKF+Lp73eufQvcYaJpPM5U=, m-encrypt-metadata-hmac: OdG3NRmQNEdeYt9MDurrXA==, m-encrypt-type: client/1, m-encrypt-cipher: AES128/CBC/PKCS5Padding, m-encrypt-metadata-iv: BqFYM3Oco4A391Eg+ot1mg==, if-match: 816610dc-915f-4d36-b482-a9d2f6e2e92a, m-encrypt-hmac-type: HmacMD5]
[7:loadBalancerAddress=165.225.172.11]
[8:response=HttpResponseProxy[original=HTTP/1.1 403 Forbidden [Content-Type: application/json, Content-Length: 241, Content-MD5: Of86XSW5J+3T75E220s2sg==, Date: Tue, 28 May 2019 22:54:16 GMT, Server: Manta, x-request-id: 85370400-819b-11e9-802b-7f3f9cebfcc2, x-response-time: 3, x-server-name: 07188c71-5cb5-497c-b5ed-3e82654aa9d0, Connection: keep-alive, x-load-balancer: 165.225.172.11] ResponseEntityProxy{[Content-Type: application/json,Content-Length: 241,Chunked: false]},connHolder=com.joyent.manta.org.apache.http.impl.execchain.ConnectionHolder@5bd5f18e]]
[9:responseHeaders=Content-Type: application/json, Content-Length: 241, Content-MD5: Of86XSW5J+3T75E220s2sg==, Date: Tue, 28 May 2019 22:54:16 GMT, Server: Manta, x-request-id: 85370400-819b-11e9-802b-7f3f9cebfcc2, x-response-time: 3, x-server-name: 07188c71-5cb5-497c-b5ed-3e82654aa9d0, Connection: keep-alive, x-load-balancer: 165.225.172.11]
[10:responseStatusCode=403]
[11:responseStatusReason=Forbidden]
---------------------------------
at com.joyent.manta.http.EncryptionHttpHelper.appendPlaintextContentLength(EncryptionHttpHelper.java:1021)
at com.joyent.manta.http.EncryptionHttpHelper.httpPut(EncryptionHttpHelper.java:280)
at com.joyent.manta.client.MantaObjectOutputStream.lambda$new$0(MantaObjectOutputStream.java:180)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
... Removed 10 stack frames
canUploadSmallStringWithErrorProneName
com.joyent.manta.exception.MantaIOException: An exception was thrown within the thread writing to the network socket
Exception Context:
[1:mantaSdkVersion=3.4.1-SNAPSHOT]
[2:path=/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-761e076c-2d48-4bea-b459-3c1a62262d90-%20-~!@%23$%25%5E&*().txt]
---------------------------------
at com.joyent.manta.client.MantaObjectOutputStream.close(MantaObjectOutputStream.java:369)
at com.joyent.manta.client.MantaObjectOutputStreamIT.canUploadSmallStringWithErrorProneName(MantaObjectOutputStreamIT.java:94)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.joyent.manta.exception.MantaIOException: Unable to update metadata with original plaintext content length
Exception Context:
[1:mantaSdkVersion=3.4.1-SNAPSHOT]
[2:requestId=85cfc190-819b-11e9-a989-c3864b8ed467]
[3:request=HttpPut[entity=com.joyent.manta.http.entity.NoContentEntity@619c3512,version=<null>,uri=https://us-east.manta.joyent.com/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-761e076c-2d48-4bea-b459-3c1a62262d90-%20-~!@%23$%25%5E&*().txt?metadata=true,config=<null>,aborted=false,cancellableRef=com.joyent.manta.org.apache.http.impl.execchain.ConnectionHolder@6d62c482,headergroup=[m-encrypt-key-id: integration-test-key, m-encrypt-plaintext-content-length: 25, m-encrypt-iv: TnQBlPCLxeZvjXnAWr4fwA==, if-unmodified-since: Tue May 28 22:54:17 UTC 2019, m-encrypt-metadata: Duw1Q2ZHUTvqXGjo52yDLUWfgV2ujYtwhVIwt4x2x7U=, m-encrypt-metadata-hmac: vxzP08PP5/PjYnTt21rMXg==, m-encrypt-type: client/1, m-encrypt-cipher: AES128/CBC/PKCS5Padding, m-encrypt-metadata-iv: Bqg/iOq367rhSyBK0JX82Q==, if-match: d6cfb91b-1099-e0c9-eb59-cd2f1fdc45ec, m-encrypt-hmac-type: HmacMD5],params=[parameters={}]]]
[4:requestMethod=PUT]
[5:requestURL=https://us-east.manta.joyent.com/elijah.zupancic/stor/java-manta-integration-tests/2a812a53-52b8-4141-afd1-20d0de4c3e58/MantaObjectOutputStreamIT/uploaded-761e076c-2d48-4bea-b459-3c1a62262d90-%20-~!@%23$%25%5E&*().txt?metadata=true]
[6:requestHeaders=m-encrypt-key-id: integration-test-key, m-encrypt-plaintext-content-length: 25, m-encrypt-iv: TnQBlPCLxeZvjXnAWr4fwA==, if-unmodified-since: Tue May 28 22:54:17 UTC 2019, m-encrypt-metadata: Duw1Q2ZHUTvqXGjo52yDLUWfgV2ujYtwhVIwt4x2x7U=, m-encrypt-metadata-hmac: vxzP08PP5/PjYnTt21rMXg==, m-encrypt-type: client/1, m-encrypt-cipher: AES128/CBC/PKCS5Padding, m-encrypt-metadata-iv: Bqg/iOq367rhSyBK0JX82Q==, if-match: d6cfb91b-1099-e0c9-eb59-cd2f1fdc45ec, m-encrypt-hmac-type: HmacMD5]
[7:loadBalancerAddress=165.225.164.26]
[8:response=HttpResponseProxy[original=HTTP/1.1 403 Forbidden [Content-Type: application/json, Content-Length: 263, Content-MD5: c3teJp/aAz6/XEvQQDBMXw==, Date: Tue, 28 May 2019 22:54:17 GMT, Server: Manta, x-request-id: 85cfc190-819b-11e9-a989-c3864b8ed467, x-response-time: 3, x-server-name: 02d02889-cd80-4ac1-bc0c-4775b86661e4, Connection: keep-alive, x-load-balancer: 165.225.164.26] ResponseEntityProxy{[Content-Type: application/json,Content-Length: 263,Chunked: false]},connHolder=com.joyent.manta.org.apache.http.impl.execchain.ConnectionHolder@6d62c482]]
[9:responseHeaders=Content-Type: application/json, Content-Length: 263, Content-MD5: c3teJp/aAz6/XEvQQDBMXw==, Date: Tue, 28 May 2019 22:54:17 GMT, Server: Manta, x-request-id: 85cfc190-819b-11e9-a989-c3864b8ed467, x-response-time: 3, x-server-name: 02d02889-cd80-4ac1-bc0c-4775b86661e4, Connection: keep-alive, x-load-balancer: 165.225.164.26]
[10:responseStatusCode=403]
[11:responseStatusReason=Forbidden]
---------------------------------
at com.joyent.manta.http.EncryptionHttpHelper.appendPlaintextContentLength(EncryptionHttpHelper.java:1021)
at com.joyent.manta.http.EncryptionHttpHelper.httpPut(EncryptionHttpHelper.java:280)