org.apache.hadoop.fs.s3.S3Exception

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • Fail after lengthy upload: s3 plugin
    via GitHub by jgn
    ,
  • When trying to download an non-cached artefact from jcenter, Artifactory verifies first if it's in its binary storage and if not there, it downloads and stores in its binary storage. In case of a Google Cloud binary storage, trying to download an non-existing resource raises a RuntimeException but the calling method expects a BinaryNotFoundException, see org.artifactory.storage.binstore.service.BinaryStore#getBinary. That causes a plain failure. Steps to reproduce : log into https://gcartifactory-us.jfrog.info, try to download https://gcartifactory-us.jfrog.info/artifactory/jcenter/org/codehaus/groovy/groovy/2.4.6/groovy-2.4.6.pom If it's not already cached, you'll get this stacktrace in the logs : {code} 2016-03-08 11:15:22,077 [http-nio-8081-exec-4] [ERROR] (o.a.s.b.s.p.RetryBinaryProvider:100) - Failed to fetch blob '3bb6de29a42e4c98bff1a49c17ec47d89978f5a5' from next binary provider java.lang.RuntimeException: Failed to get Stream from s3 at org.artifactory.addon.filestore.type.jets3t.gcs.GoogleJets3tBinaryProvider.getStream(GoogleJets3tBinaryProvider.java:167) ~[artifactory-addon-filestore-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:89) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:98) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:98) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:98) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:98) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:98) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:39) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.addon.filestore.eventual.EventuallyPersistedBinaryProvider.forwardStreamInternal(EventuallyPersistedBinaryProvider.java:316) [artifactory-addon-filestore-4.5.2.jar:na] at org.artifactory.addon.filestore.eventual.EventuallyPersistedBinaryProvider.getStream(EventuallyPersistedBinaryProvider.java:170) [artifactory-addon-filestore-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.FileCacheBinaryProviderImpl.getStream(FileCacheBinaryProviderImpl.java:106) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.db.binstore.service.UsageTrackingBinaryProvider.getStream(UsageTrackingBinaryProvider.java:49) [artifactory-storage-db-4.5.2.jar:na] at org.artifactory.storage.db.binstore.service.BinaryStoreImpl.getBinary(BinaryStoreImpl.java:410) [artifactory-storage-db-4.5.2.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_71] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_71] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_71] at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_71] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at com.sun.proxy.$Proxy60.getBinary(Unknown Source) [na:na] at org.artifactory.repo.RemoteRepoBase.getExistingResourceByChecksum(RemoteRepoBase.java:795) [artifactory-core-4.5.2.jar:na] at org.artifactory.repo.RemoteRepoBase.doDownloadAndSave(RemoteRepoBase.java:690) [artifactory-core-4.5.2.jar:na] at org.artifactory.repo.RemoteRepoBase.downloadAndSave(RemoteRepoBase.java:585) [artifactory-core-4.5.2.jar:na] at org.artifactory.repo.service.RepositoryServiceImpl.downloadAndSave(RepositoryServiceImpl.java:1607) [artifactory-core-4.5.2.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_71] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_71] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_71] at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_71] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at com.sun.proxy.$Proxy46.downloadAndSave(Unknown Source) [na:na] at org.artifactory.repo.RemoteRepoBase.getResourceStreamHandle(RemoteRepoBase.java:496) [artifactory-core-4.5.2.jar:na] at org.artifactory.repo.service.RepositoryServiceImpl.getResourceStreamHandle(RepositoryServiceImpl.java:1651) [artifactory-core-4.5.2.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_71] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_71] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_71] at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_71] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at com.sun.proxy.$Proxy46.getResourceStreamHandle(Unknown Source) [na:na] at org.artifactory.engine.DownloadServiceImpl.respondFoundResource(DownloadServiceImpl.java:336) [artifactory-core-4.5.2.jar:na] at org.artifactory.engine.DownloadServiceImpl.respond(DownloadServiceImpl.java:282) [artifactory-core-4.5.2.jar:na] at org.artifactory.engine.DownloadServiceImpl.process(DownloadServiceImpl.java:215) [artifactory-core-4.5.2.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_71] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_71] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_71] at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_71] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.artifactory.request.aop.RequestAdvice.invoke(RequestAdvice.java:65) [artifactory-core-4.5.2.jar:na] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at com.sun.proxy.$Proxy93.process(Unknown Source) [na:na] at org.artifactory.webapp.servlet.RepoFilter.doDownload(RepoFilter.java:248) [artifactory-web-application-4.5.2.jar:na] at org.artifactory.webapp.servlet.RepoFilter.execute(RepoFilter.java:173) [artifactory-web-application-4.5.2.jar:na] at org.artifactory.webapp.servlet.RepoFilter.doFilter(RepoFilter.java:89) [artifactory-web-application-4.5.2.jar:na] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.22] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.22] at org.artifactory.webapp.servlet.AccessFilter.useAuthentication(AccessFilter.java:334) [artifactory-web-application-4.5.2.jar:na] at org.artifactory.webapp.servlet.AccessFilter.doFilterInternal(AccessFilter.java:196) [artifactory-web-application-4.5.2.jar:na] at org.artifactory.webapp.servlet.AccessFilter.doFilter(AccessFilter.java:156) [artifactory-web-application-4.5.2.jar:na] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.22] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.22] at org.artifactory.webapp.servlet.RequestFilter.doFilter(RequestFilter.java:65) [artifactory-web-application-4.5.2.jar:na] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.22] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.22] at org.artifactory.webapp.servlet.ArtifactoryFilter.doFilter(ArtifactoryFilter.java:109) [artifactory-web-application-4.5.2.jar:na] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.22] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.22] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) [catalina.jar:8.0.22] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) [catalina.jar:8.0.22] at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:673) [catalina.jar:8.0.22] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142) [catalina.jar:8.0.22] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [catalina.jar:8.0.22] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) [catalina.jar:8.0.22] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518) [catalina.jar:8.0.22] at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091) [tomcat-coyote.jar:8.0.22] at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668) [tomcat-coyote.jar:8.0.22] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1521) [tomcat-coyote.jar:8.0.22] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1478) [tomcat-coyote.jar:8.0.22] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_71] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:8.0.22] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_71] Caused by: org.jets3t.service.ServiceException: Service Error Message. at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:409) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:279) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestGet(RestStorageService.java:1104) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestGet(RestStorageService.java:1076) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.getObjectImpl(RestStorageService.java:2267) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.getObjectImpl(RestStorageService.java:2204) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.StorageService.getObject(StorageService.java:1167) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.GoogleStorageService.getObject(GoogleStorageService.java:425) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.GoogleStorageService.getObject(GoogleStorageService.java:62) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.StorageService.getObject(StorageService.java:552) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.GoogleStorageService.getObject(GoogleStorageService.java:410) ~[jets3t-0.9.4.jar:0.9.4] at org.artifactory.addon.filestore.type.jets3t.gcs.GoogleBaseService.get(GoogleBaseService.java:39) ~[artifactory-addon-filestore-4.5.2.jar:na] at org.artifactory.addon.filestore.type.jets3t.gcs.GoogleJets3tBinaryProvider.getStream(GoogleJets3tBinaryProvider.java:157) ~[artifactory-addon-filestore-4.5.2.jar:na] ... 84 common frames omitted {code}
    via by Alexis Tual,
  • When trying to download an non-cached artefact from jcenter, Artifactory verifies first if it's in its binary storage and if not there, it downloads and stores in its binary storage. In case of a Google Cloud binary storage, trying to download an non-existing resource raises a RuntimeException but the calling method expects a BinaryNotFoundException, see org.artifactory.storage.binstore.service.BinaryStore#getBinary. That causes a plain failure. Steps to reproduce : log into https://gcartifactory-us.jfrog.info, try to download https://gcartifactory-us.jfrog.info/artifactory/jcenter/org/codehaus/groovy/groovy/2.4.6/groovy-2.4.6.pom If it's not already cached, you'll get this stacktrace in the logs : {code} 2016-03-08 11:15:22,077 [http-nio-8081-exec-4] [ERROR] (o.a.s.b.s.p.RetryBinaryProvider:100) - Failed to fetch blob '3bb6de29a42e4c98bff1a49c17ec47d89978f5a5' from next binary provider java.lang.RuntimeException: Failed to get Stream from s3 at org.artifactory.addon.filestore.type.jets3t.gcs.GoogleJets3tBinaryProvider.getStream(GoogleJets3tBinaryProvider.java:167) ~[artifactory-addon-filestore-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:89) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:98) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:98) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:98) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:98) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:98) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.RetryBinaryProvider.getStream(RetryBinaryProvider.java:39) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.addon.filestore.eventual.EventuallyPersistedBinaryProvider.forwardStreamInternal(EventuallyPersistedBinaryProvider.java:316) [artifactory-addon-filestore-4.5.2.jar:na] at org.artifactory.addon.filestore.eventual.EventuallyPersistedBinaryProvider.getStream(EventuallyPersistedBinaryProvider.java:170) [artifactory-addon-filestore-4.5.2.jar:na] at org.artifactory.storage.binstore.service.providers.FileCacheBinaryProviderImpl.getStream(FileCacheBinaryProviderImpl.java:106) [artifactory-storage-common-4.5.2.jar:na] at org.artifactory.storage.db.binstore.service.UsageTrackingBinaryProvider.getStream(UsageTrackingBinaryProvider.java:49) [artifactory-storage-db-4.5.2.jar:na] at org.artifactory.storage.db.binstore.service.BinaryStoreImpl.getBinary(BinaryStoreImpl.java:410) [artifactory-storage-db-4.5.2.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_71] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_71] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_71] at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_71] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at com.sun.proxy.$Proxy60.getBinary(Unknown Source) [na:na] at org.artifactory.repo.RemoteRepoBase.getExistingResourceByChecksum(RemoteRepoBase.java:795) [artifactory-core-4.5.2.jar:na] at org.artifactory.repo.RemoteRepoBase.doDownloadAndSave(RemoteRepoBase.java:690) [artifactory-core-4.5.2.jar:na] at org.artifactory.repo.RemoteRepoBase.downloadAndSave(RemoteRepoBase.java:585) [artifactory-core-4.5.2.jar:na] at org.artifactory.repo.service.RepositoryServiceImpl.downloadAndSave(RepositoryServiceImpl.java:1607) [artifactory-core-4.5.2.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_71] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_71] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_71] at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_71] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at com.sun.proxy.$Proxy46.downloadAndSave(Unknown Source) [na:na] at org.artifactory.repo.RemoteRepoBase.getResourceStreamHandle(RemoteRepoBase.java:496) [artifactory-core-4.5.2.jar:na] at org.artifactory.repo.service.RepositoryServiceImpl.getResourceStreamHandle(RepositoryServiceImpl.java:1651) [artifactory-core-4.5.2.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_71] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_71] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_71] at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_71] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at com.sun.proxy.$Proxy46.getResourceStreamHandle(Unknown Source) [na:na] at org.artifactory.engine.DownloadServiceImpl.respondFoundResource(DownloadServiceImpl.java:336) [artifactory-core-4.5.2.jar:na] at org.artifactory.engine.DownloadServiceImpl.respond(DownloadServiceImpl.java:282) [artifactory-core-4.5.2.jar:na] at org.artifactory.engine.DownloadServiceImpl.process(DownloadServiceImpl.java:215) [artifactory-core-4.5.2.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_71] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_71] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_71] at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_71] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.artifactory.request.aop.RequestAdvice.invoke(RequestAdvice.java:65) [artifactory-core-4.5.2.jar:na] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) [spring-aop-4.1.5.RELEASE.jar:4.1.5.RELEASE] at com.sun.proxy.$Proxy93.process(Unknown Source) [na:na] at org.artifactory.webapp.servlet.RepoFilter.doDownload(RepoFilter.java:248) [artifactory-web-application-4.5.2.jar:na] at org.artifactory.webapp.servlet.RepoFilter.execute(RepoFilter.java:173) [artifactory-web-application-4.5.2.jar:na] at org.artifactory.webapp.servlet.RepoFilter.doFilter(RepoFilter.java:89) [artifactory-web-application-4.5.2.jar:na] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.22] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.22] at org.artifactory.webapp.servlet.AccessFilter.useAuthentication(AccessFilter.java:334) [artifactory-web-application-4.5.2.jar:na] at org.artifactory.webapp.servlet.AccessFilter.doFilterInternal(AccessFilter.java:196) [artifactory-web-application-4.5.2.jar:na] at org.artifactory.webapp.servlet.AccessFilter.doFilter(AccessFilter.java:156) [artifactory-web-application-4.5.2.jar:na] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.22] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.22] at org.artifactory.webapp.servlet.RequestFilter.doFilter(RequestFilter.java:65) [artifactory-web-application-4.5.2.jar:na] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.22] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.22] at org.artifactory.webapp.servlet.ArtifactoryFilter.doFilter(ArtifactoryFilter.java:109) [artifactory-web-application-4.5.2.jar:na] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.22] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.22] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) [catalina.jar:8.0.22] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) [catalina.jar:8.0.22] at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:673) [catalina.jar:8.0.22] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142) [catalina.jar:8.0.22] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [catalina.jar:8.0.22] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) [catalina.jar:8.0.22] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518) [catalina.jar:8.0.22] at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091) [tomcat-coyote.jar:8.0.22] at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668) [tomcat-coyote.jar:8.0.22] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1521) [tomcat-coyote.jar:8.0.22] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1478) [tomcat-coyote.jar:8.0.22] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_71] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:8.0.22] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_71] Caused by: org.jets3t.service.ServiceException: Service Error Message. at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:409) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:279) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestGet(RestStorageService.java:1104) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestGet(RestStorageService.java:1076) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.getObjectImpl(RestStorageService.java:2267) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.RestStorageService.getObjectImpl(RestStorageService.java:2204) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.StorageService.getObject(StorageService.java:1167) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.GoogleStorageService.getObject(GoogleStorageService.java:425) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.GoogleStorageService.getObject(GoogleStorageService.java:62) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.StorageService.getObject(StorageService.java:552) ~[jets3t-0.9.4.jar:0.9.4] at org.jets3t.service.impl.rest.httpclient.GoogleStorageService.getObject(GoogleStorageService.java:410) ~[jets3t-0.9.4.jar:0.9.4] at org.artifactory.addon.filestore.type.jets3t.gcs.GoogleBaseService.get(GoogleBaseService.java:39) ~[artifactory-addon-filestore-4.5.2.jar:na] at org.artifactory.addon.filestore.type.jets3t.gcs.GoogleJets3tBinaryProvider.getStream(GoogleJets3tBinaryProvider.java:157) ~[artifactory-addon-filestore-4.5.2.jar:na] ... 84 common frames omitted {code}
    via by Alexis Tual,
    • org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.ServiceException: Service Error Message. -- ResponseCode: 404, ResponseStatus: Not Found, XML Error Message: <Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>temp_spark_streaming_data/temp_events/49b2d38f-bf9b-4512-85b3-6f6c60629bfc/_temporary/0/_temporary/attempt_201609280849_0058_m_000000_0/part-r-00000-6eed0a45-8732-4524-a342-0cdca3ccb43b.avro</Key><RequestId>84F26D842580A8C9</RequestId><HostId>8UlMGpqB1svcQl4/QhKwnt+NsOWGOgKYJes/W8RffrM+H+tKPuHCuhCWnQ8nmzu8inRr6UWlIMU=</HostId></Error> at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.processException(Jets3tNativeFileSystemStore.java:470) at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleException(Jets3tNativeFileSystemStore.java:411) at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.copy(Jets3tNativeFileSystemStore.java:326) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at org.apache.hadoop.fs.s3native.$Proxy20.copy(Unknown Source) at org.apache.hadoop.fs.s3native.NativeS3FileSystem.rename(NativeS3FileSystem.java:713) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:435) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:415) at org.apache.spark.mapred.SparkHadoopMapRedUtil$.performCommit$1(SparkHadoopMapRedUtil.scala:98) at org.apache.spark.mapred.SparkHadoopMapRedUtil$.commitTask(SparkHadoopMapRedUtil.scala:124) at org.apache.spark.sql.execution.datasources.BaseWriterContainer.commitTask(WriterContainer.scala:219) at org.apache.spark.sql.execution.datasources.DefaultWriterContainer.org$apache$spark$sql$execution$datasources$DefaultWriterContainer$$commitTask$1(WriterContainer.scala:278) at org.apache.spark.sql.execution.datasources.DefaultWriterContainer$$anonfun$writeRows$1.apply$mcV$sp(WriterContainer.scala:265) at org.apache.spark.sql.execution.datasources.DefaultWriterContainer$$anonfun$writeRows$1.apply(WriterContainer.scala:260) at org.apache.spark.sql.execution.datasources.DefaultWriterContainer$$anonfun$writeRows$1.apply(WriterContainer.scala:260) at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1277) at org.apache.spark.sql.execution.datasources.DefaultWriterContainer.writeRows(WriterContainer.scala:266) at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelation$$anonfun$run$1$$anonfun$apply$mcV$sp$3.apply(InsertIntoHadoopFsRelation.scala:148) at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelation$$anonfun$run$1$$anonfun$apply$mcV$sp$3.apply(InsertIntoHadoopFsRelation.scala:148) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) at org.apache.spark.scheduler.Task.run(Task.scala:89) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.jets3t.service.ServiceException: Service Error Message. -- ResponseCode: 404, ResponseStatus: Not Found, XML Error Message: <Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>temp_spark_streaming_data/temp_events/49b2d38f-bf9b-4512-85b3-6f6c60629bfc/_temporary/0/_temporary/attempt_201609280849_0058_m_000000_0/part-r-00000-6eed0a45-8732-4524-a342-0cdca3ccb43b.avro</Key><RequestId>84F26D842580A8C9</RequestId><HostId>8UlMGpqB1svcQl4/QhKwnt+NsOWGOgKYJes/W8RffrM+H+tKPuHCuhCWnQ8nmzu8inRr6UWlIMU=</HostId></Error> at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:407) at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:277) at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestPut(RestStorageService.java:1143) at org.jets3t.service.impl.rest.httpclient.RestStorageService.copyObjectImpl(RestStorageService.java:2117) at org.jets3t.service.StorageService.copyObject(StorageService.java:898) at org.jets3t.service.StorageService.copyObject(StorageService.java:943) at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.copy(Jets3tNativeFileSystemStore.java:323) ... 26 more

    Users with the same issue

    Unknown visitor1 times, last one,