org.apache.hadoop.fs.s3.S3Exception

org.jets3t.service.S3ServiceException: Request Error: Failed to automatically set required header "x-amz-content-sha256" for request with entity org.jets3t.service.impl.rest.httpclient.RepeatableRequestEntity@51194de9

Samebug tips0

We couldn't find tips for this exception.

Don't give up yet. Paste your full stack trace to get a solution.

Solutions on the web63

  • via Unknown by 唐焱,
  • via Unknown by Sander van Dijk,
  • Stack trace

    • org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException: Request Error: Failed to automatically set required header "x-amz-content-sha256" for request with entity org.jets3t.service.impl.rest.httpclient.RepeatableRequestEntity@51194de9 at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleS3ServiceException(Jets3tNativeFileSystemStore.java:310)[hadoop-common-2.3.0.jar:?] at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.storeFile(Jets3tNativeFileSystemStore.java:89)[hadoop-common-2.3.0.jar:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[?:1.7.0_101] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)[?:1.7.0_101] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[?:1.7.0_101] at java.lang.reflect.Method.invoke(Method.java:606)[?:1.7.0_101] at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)[hadoop-common-2.3.0.jar:?] at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)[hadoop-common-2.3.0.jar:?] at org.apache.hadoop.fs.s3native.$Proxy191.storeFile(Unknown Source)[?:?] at org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsOutputStream.close(NativeS3FileSystem.java:221)[hadoop-common-2.3.0.jar:?] at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:70)[hadoop-common-2.3.0.jar:?] at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:103)[hadoop-common-2.3.0.jar:?] at java.util.zip.DeflaterOutputStream.close(DeflaterOutputStream.java:241)[?:1.7.0_101] at java.util.zip.ZipOutputStream.close(ZipOutputStream.java:360)[?:1.7.0_101] at io.druid.indexer.JobHelper.zipAndCopyDir(JobHelper.java:511)[druid-indexing-hadoop-0.9.1.1.jar:0.9.1.1] at io.druid.indexer.JobHelper$4.push(JobHelper.java:374)[druid-indexing-hadoop-0.9.1.1.jar:0.9.1.1] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[?:1.7.0_101] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)[?:1.7.0_101] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[?:1.7.0_101] at java.lang.reflect.Method.invoke(Method.java:606)[?:1.7.0_101] at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)[hadoop-common-2.3.0.jar:?] at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)[hadoop-common-2.3.0.jar:?] at com.sun.proxy.$Proxy192.push(Unknown Source)[?:?] at io.druid.indexer.JobHelper.serializeOutIndex(JobHelper.java:386)[druid-indexing-hadoop-0.9.1.1.jar:0.9.1.1] at io.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.reduce(IndexGeneratorJob.java:703)[druid-indexing-hadoop-0.9.1.1.jar:0.9.1.1] at io.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.reduce(IndexGeneratorJob.java:469)[druid-indexing-hadoop-0.9.1.1.jar:0.9.1.1] at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)[hadoop-mapreduce-client-core-2.3.0.jar:?] at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)[hadoop-mapreduce-client-core-2.3.0.jar:?] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)[hadoop-mapreduce-client-core-2.3.0.jar:?] at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)[hadoop-mapreduce-client-common-2.3.0.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[?:1.7.0_101] at java.util.concurrent.FutureTask.run(FutureTask.java:262)[?:1.7.0_101] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[?:1.7.0_101] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[?:1.7.0_101] at java.lang.Thread.run(Thread.java:745)[?:1.7.0_101] Caused by: org.jets3t.service.S3ServiceException: Request Error: Failed to automatically set required header "x-amz-content-sha256" for request with entity org.jets3t.service.impl.rest.httpclient.RepeatableRequestEntity@51194de9 at org.jets3t.service.S3Service.putObject(S3Service.java:2358)[jets3t-0.9.4.jar:0.9.4] at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.storeFile(Jets3tNativeFileSystemStore.java:87)[hadoop-common-2.3.0.jar:?] ... 33 more Caused by: java.lang.RuntimeException: Failed to automatically set required header "x-amz-content-sha256" for request with entity org.jets3t.service.impl.rest.httpclient.RepeatableRequestEntity@51194de9 at org.jets3t.service.utils.SignatureUtils.awsV4GetOrCalculatePayloadHash(SignatureUtils.java:259)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.authorizeHttpRequest(RestStorageService.java:778)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:326)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:279)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestPut(RestStorageService.java:1157)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.createObjectImpl(RestStorageService.java:1968)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.putObjectWithRequestEntityImpl(RestStorageService.java:1889)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.putObjectImpl(RestStorageService.java:1881)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.StorageService.putObject(StorageService.java:840)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.S3Service.putObject(S3Service.java:2212)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.S3Service.putObject(S3Service.java:2356)[jets3t-0.9.4.jar:0.9.4] ... 34 more Caused by: java.io.IOException: Resetting to invalid mark at java.io.BufferedInputStream.reset(BufferedInputStream.java:437)[?:1.7.0_101] at org.jets3t.service.utils.ServiceUtils.hash(ServiceUtils.java:238)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.utils.ServiceUtils.hashSHA256(ServiceUtils.java:267)[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.utils.SignatureUtils.awsV4GetOrCalculatePayloadHash(SignatureUtils.java:251)[jets3t-0.9.4.jar:0.9.4] ... 44 more

    Write tip

    You have a different solution? A short tip here would help you and many other users who saw this issue last week.

    Users with the same issue

    Unknown visitor
    Unknown visitorOnce,
    Unknown visitor
    Unknown visitorOnce,
    Unknown visitor
    Unknown visitorOnce,