org.springframework.xd.dirt.zookeeper.ZooKeeperAccessException: KeeperErrorCode = NoNode for /xd/deployments/streams/testCreateStream_SrcHttp_SinkFile1413234903170/status

Spring JIRA | Venkatesh Sivasubramanian | 2 years ago
  1. 0

    Sorry to set it "Blocker"; but the problem makes SpringXD unusable. We are getting this weird, NoNode exception on the status ZNode. Example and Log given below. Once this happens, both streams and jobs cannot be deployed. For whatever reason the "status" znode goes missing. The only way for us to get the cluster back to working state, is to clear the zk znode /xd tree and restart spring-xd. At which point, we have to recreate all our streams and jobs back again.. /xd/deployments/streams/testCreateStream_SrcHttp_SinkFile1413234903170/status NoNode Exception: 13 Oct 2014 14:16:16,044 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/streams/testDestroyStream1413234903170, type: CHILD_REMOVED 13 Oct 2014 14:16:16,705 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/streams/testListStreams1413234903170, type: CHILD_ADDED 13 Oct 2014 14:16:22,818 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/streams/testListStreams1413234903170, type: CHILD_REMOVED 13 Oct 2014 14:16:23,585 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/streams/testStreamSrcHttpTimeseriesSink1413234903170, type: CHILD_ADDED 13 Oct 2014 14:16:37,694 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/filetsjob-newjob001, type: CHILD_ADDED 13 Oct 2014 14:16:49,950 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/streams/testStreamSrcHttpTimeseriesSink1413234903170, type: CHILD_REMOVED 13 Oct 2014 14:16:54,490 INFO Deployer server.StreamDeploymentListener - Deployment status for stream 'testCreateStream_SrcHttp_SinkFile1413234903170': DeploymentStatus{state=failed,error(s)=Deployment of module 'ModuleDeploymentKey{stream='testCreateStream_SrcHttp_SinkFile1413234903170', type=sink, label='file'}' to container 'd03bccd6-524b-4ff8-84d2-88f3f6daac42' timed out after 30000 ms; Deployment of module 'ModuleDeploymentKey{stream='testCreateStream_SrcHttp_SinkFile1413234903170', type=source, label='http'}' to container '52abf1c8-ba45-4994-8324-6079b03c670c' timed out after 30000 ms} 13 Oct 2014 14:16:54,493 ERROR Deployer server.InitialDeploymentListener - Exception caught while handling event org.springframework.xd.dirt.zookeeper.ZooKeeperAccessException: KeeperErrorCode = NoNode for /xd/deployments/streams/testCreateStream_SrcHttp_SinkFile1413234903170/status at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:111) at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:95) at org.springframework.xd.dirt.server.StreamDeploymentListener.deployStream(StreamDeploymentListener.java:185) at org.springframework.xd.dirt.server.StreamDeploymentListener.onChildAdded(StreamDeploymentListener.java:100) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:217) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:186) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /xd/deployments/streams/testCreateStream_SrcHttp_SinkFile1413234903170/status at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1270) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:274) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:270) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.SetDataBuilderImpl.pathInForeground(SetDataBuilderImpl.java:266) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:253) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:41) at org.springframework.xd.dirt.server.StreamDeploymentListener.deployStream(StreamDeploymentListener.java:179) ... 7 more 13 Oct 2014 14:16:56,251 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/filetsjob-newjob001, type: CHILD_REMOVED 13 Oct 2014 14:17:08,179 INFO Deployer server.JobDeploymentListener - Deployment status for job 'filetsjob-newjob001': DeploymentStatus{state=failed,error(s)=Deployment of module 'ModuleDeploymentKey{stream='filetsjob-newjob001', type=job, label='filepolltimeseries'}' to container '244d5076-f69d-42a4-8110-3b046cea2667' timed out after 30000 ms} 13 Oct 2014 14:17:08,181 ERROR Deployer server.InitialDeploymentListener - Exception caught while handling event org.springframework.xd.dirt.zookeeper.ZooKeeperAccessException: KeeperErrorCode = NoNode for /xd/deployments/jobs/filetsjob-newjob001/status at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:111) at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:95) at org.springframework.xd.dirt.server.JobDeploymentListener.deployJob(JobDeploymentListener.java:175) at org.springframework.xd.dirt.server.JobDeploymentListener.onChildAdded(JobDeploymentListener.java:99) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:217) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:186) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /xd/deployments/jobs/filetsjob-newjob001/status at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1270) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:274) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:270) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.SetDataBuilderImpl.pathInForeground(SetDataBuilderImpl.java:266) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:253) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:41) at org.springframework.xd.dirt.server.JobDeploymentListener.deployJob(JobDeploymentListener.java:165) ... 7 more 13 Oct 2014 14:17:10,553 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/filetsjob-newjob001, type: CHILD_ADDED

    Spring JIRA | 2 years ago | Venkatesh Sivasubramanian
    org.springframework.xd.dirt.zookeeper.ZooKeeperAccessException: KeeperErrorCode = NoNode for /xd/deployments/streams/testCreateStream_SrcHttp_SinkFile1413234903170/status
  2. 0

    Sorry to set it "Blocker"; but the problem makes SpringXD unusable. We are getting this weird, NoNode exception on the status ZNode. Example and Log given below. Once this happens, both streams and jobs cannot be deployed. For whatever reason the "status" znode goes missing. The only way for us to get the cluster back to working state, is to clear the zk znode /xd tree and restart spring-xd. At which point, we have to recreate all our streams and jobs back again.. /xd/deployments/streams/testCreateStream_SrcHttp_SinkFile1413234903170/status NoNode Exception: 13 Oct 2014 14:16:16,044 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/streams/testDestroyStream1413234903170, type: CHILD_REMOVED 13 Oct 2014 14:16:16,705 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/streams/testListStreams1413234903170, type: CHILD_ADDED 13 Oct 2014 14:16:22,818 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/streams/testListStreams1413234903170, type: CHILD_REMOVED 13 Oct 2014 14:16:23,585 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/streams/testStreamSrcHttpTimeseriesSink1413234903170, type: CHILD_ADDED 13 Oct 2014 14:16:37,694 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/filetsjob-newjob001, type: CHILD_ADDED 13 Oct 2014 14:16:49,950 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/streams/testStreamSrcHttpTimeseriesSink1413234903170, type: CHILD_REMOVED 13 Oct 2014 14:16:54,490 INFO Deployer server.StreamDeploymentListener - Deployment status for stream 'testCreateStream_SrcHttp_SinkFile1413234903170': DeploymentStatus{state=failed,error(s)=Deployment of module 'ModuleDeploymentKey{stream='testCreateStream_SrcHttp_SinkFile1413234903170', type=sink, label='file'}' to container 'd03bccd6-524b-4ff8-84d2-88f3f6daac42' timed out after 30000 ms; Deployment of module 'ModuleDeploymentKey{stream='testCreateStream_SrcHttp_SinkFile1413234903170', type=source, label='http'}' to container '52abf1c8-ba45-4994-8324-6079b03c670c' timed out after 30000 ms} 13 Oct 2014 14:16:54,493 ERROR Deployer server.InitialDeploymentListener - Exception caught while handling event org.springframework.xd.dirt.zookeeper.ZooKeeperAccessException: KeeperErrorCode = NoNode for /xd/deployments/streams/testCreateStream_SrcHttp_SinkFile1413234903170/status at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:111) at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:95) at org.springframework.xd.dirt.server.StreamDeploymentListener.deployStream(StreamDeploymentListener.java:185) at org.springframework.xd.dirt.server.StreamDeploymentListener.onChildAdded(StreamDeploymentListener.java:100) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:217) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:186) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /xd/deployments/streams/testCreateStream_SrcHttp_SinkFile1413234903170/status at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1270) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:274) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:270) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.SetDataBuilderImpl.pathInForeground(SetDataBuilderImpl.java:266) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:253) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:41) at org.springframework.xd.dirt.server.StreamDeploymentListener.deployStream(StreamDeploymentListener.java:179) ... 7 more 13 Oct 2014 14:16:56,251 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/filetsjob-newjob001, type: CHILD_REMOVED 13 Oct 2014 14:17:08,179 INFO Deployer server.JobDeploymentListener - Deployment status for job 'filetsjob-newjob001': DeploymentStatus{state=failed,error(s)=Deployment of module 'ModuleDeploymentKey{stream='filetsjob-newjob001', type=job, label='filepolltimeseries'}' to container '244d5076-f69d-42a4-8110-3b046cea2667' timed out after 30000 ms} 13 Oct 2014 14:17:08,181 ERROR Deployer server.InitialDeploymentListener - Exception caught while handling event org.springframework.xd.dirt.zookeeper.ZooKeeperAccessException: KeeperErrorCode = NoNode for /xd/deployments/jobs/filetsjob-newjob001/status at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:111) at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:95) at org.springframework.xd.dirt.server.JobDeploymentListener.deployJob(JobDeploymentListener.java:175) at org.springframework.xd.dirt.server.JobDeploymentListener.onChildAdded(JobDeploymentListener.java:99) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:217) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:186) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /xd/deployments/jobs/filetsjob-newjob001/status at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1270) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:274) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:270) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.SetDataBuilderImpl.pathInForeground(SetDataBuilderImpl.java:266) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:253) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:41) at org.springframework.xd.dirt.server.JobDeploymentListener.deployJob(JobDeploymentListener.java:165) ... 7 more 13 Oct 2014 14:17:10,553 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/filetsjob-newjob001, type: CHILD_ADDED

    Spring JIRA | 2 years ago | Venkatesh Sivasubramanian
    org.springframework.xd.dirt.zookeeper.ZooKeeperAccessException: KeeperErrorCode = NoNode for /xd/deployments/streams/testCreateStream_SrcHttp_SinkFile1413234903170/status
  3. 0

    SHA: 33de93797106c8dd413dfb08f2fdbbb4931b528c Deployment: 1 Admin, 1 Container DataStore: MySQL A SpringXDException is thrown when running the testJobDeployUndeployFlow test in Acceptance tests back to back. # Ran the test once. Success # Ran the test a second time. ## The following exception is thrown: SpringXDException: Batch Job with the name deployundeployjob already exists ## The XD_JOB_REGISTRY_STEP_NAMES & XD_JOB_REGISTRY still have job_name deployunderployjob still stored. ## The following Exception is seen on the admin server ### 20:45:59,214 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/deployundeployjob, type: CHILD_ADDED 20:46:04,741 INFO Deployer server.JobDeploymentListener - Deployment status for job 'deployundeployjob': DeploymentStatus{state=deployed} 20:46:05,436 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/deployundeployjob, type: CHILD_REMOVED 20:46:12,471 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/deployundeployjob, type: CHILD_ADDED 20:46:16,330 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/deployundeployjob, type: CHILD_REMOVED 20:46:17,819 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/deployundeployjob, type: CHILD_ADDED 20:46:17,832 INFO Deployer server.JobDeploymentListener - Deployment status for job 'deployundeployjob': DeploymentStatus{state=deployed} 20:46:17,834 ERROR Deployer server.InitialDeploymentListener - Exception caught while handling event org.springframework.xd.dirt.zookeeper.ZooKeeperAccessException: KeeperErrorCode = NoNode for /xd/deployments/jobs/deployundeployjob/status at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:111) at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:95) at org.springframework.xd.dirt.server.JobDeploymentListener.deployJob(JobDeploymentListener.java:175) at org.springframework.xd.dirt.server.JobDeploymentListener.onChildAdded(JobDeploymentListener.java:99) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:217) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:186) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /xd/deployments/jobs/deployundeployjob/status at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1270) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:274) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:270) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.SetDataBuilderImpl.pathInForeground(SetDataBuilderImpl.java:266) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:253) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:41) at org.springframework.xd.dirt.server.JobDeploymentListener.deployJob(JobDeploymentListener.java:165) ... 7 more # Clear XD_JOB_REGISTRY_STEP_NAMES & XD_JOB_REGISTRY tables and run the test a third time # Ran the test a third time. ## The following exception is thrown: SpringXDException: The job named 'deployundeployjob' is already deployed ## The following Exception was reported by the admin server. ### 21:18:29,355 WARN http-nio-9393-exec-4 zookeeper.ZooKeeperJobRepository - Exception while transitioning job 'deployundeployjob' state to undeploying org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /xd/deployments/jobs/deployundeployjob/status at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1270) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:274) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:270) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.SetDataBuilderImpl.pathInForeground(SetDataBuilderImpl.java:266) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:253) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:41) at org.springframework.xd.dirt.stream.zookeeper.ZooKeeperJobRepository.delete(ZooKeeperJobRepository.java:174) at org.springframework.xd.dirt.stream.zookeeper.ZooKeeperJobRepository.delete(ZooKeeperJobRepository.java:196) at org.springframework.xd.dirt.stream.zookeeper.ZooKeeperJobRepository.delete(ZooKeeperJobRepository.java:54) at org.springframework.xd.dirt.stream.AbstractInstancePersistingDeployer.undeploy(AbstractInstancePersistingDeployer.java:77) at org.springframework.xd.dirt.stream.AbstractInstancePersistingDeployer.undeployAll(AbstractInstancePersistingDeployer.java:103) at org.springframework.xd.dirt.stream.AbstractInstancePersistingDeployer.deleteAll(AbstractInstancePersistingDeployer.java:111) at org.springframework.xd.dirt.rest.XDController.deleteAll(XDController.java:114) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:749) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:689) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:938) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:870) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961) at org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:885) at javax.servlet.http.HttpServlet.service(HttpServlet.java:652) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:110) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextFilterConfiguration$1.doFilterInternal(EndpointWebMvcAutoConfiguration.java:257) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:88) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.boot.actuate.autoconfigure.MetricFilterAutoConfiguration$MetricsFilter.doFilterInternal(MetricFilterAutoConfiguration.java:85) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:683) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1720) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1679) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) # Ran the test a fourth time. Success

    Spring JIRA | 2 years ago | Glenn renfro
    org.springframework.xd.dirt.zookeeper.ZooKeeperAccessException: KeeperErrorCode = NoNode for /xd/deployments/jobs/deployundeployjob/status
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    SHA: 33de93797106c8dd413dfb08f2fdbbb4931b528c Deployment: 1 Admin, 1 Container DataStore: MySQL A SpringXDException is thrown when running the testJobDeployUndeployFlow test in Acceptance tests back to back. # Ran the test once. Success # Ran the test a second time. ## The following exception is thrown: SpringXDException: Batch Job with the name deployundeployjob already exists ## The XD_JOB_REGISTRY_STEP_NAMES & XD_JOB_REGISTRY still have job_name deployunderployjob still stored. ## The following Exception is seen on the admin server ### 20:45:59,214 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/deployundeployjob, type: CHILD_ADDED 20:46:04,741 INFO Deployer server.JobDeploymentListener - Deployment status for job 'deployundeployjob': DeploymentStatus{state=deployed} 20:46:05,436 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/deployundeployjob, type: CHILD_REMOVED 20:46:12,471 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/deployundeployjob, type: CHILD_ADDED 20:46:16,330 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/deployundeployjob, type: CHILD_REMOVED 20:46:17,819 INFO DeploymentSupervisorCacheListener-0 server.InitialDeploymentListener - Path cache event: /deployments/jobs/deployundeployjob, type: CHILD_ADDED 20:46:17,832 INFO Deployer server.JobDeploymentListener - Deployment status for job 'deployundeployjob': DeploymentStatus{state=deployed} 20:46:17,834 ERROR Deployer server.InitialDeploymentListener - Exception caught while handling event org.springframework.xd.dirt.zookeeper.ZooKeeperAccessException: KeeperErrorCode = NoNode for /xd/deployments/jobs/deployundeployjob/status at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:111) at org.springframework.xd.dirt.zookeeper.ZooKeeperUtils.wrapThrowable(ZooKeeperUtils.java:95) at org.springframework.xd.dirt.server.JobDeploymentListener.deployJob(JobDeploymentListener.java:175) at org.springframework.xd.dirt.server.JobDeploymentListener.onChildAdded(JobDeploymentListener.java:99) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:217) at org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:186) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /xd/deployments/jobs/deployundeployjob/status at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1270) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:274) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:270) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.SetDataBuilderImpl.pathInForeground(SetDataBuilderImpl.java:266) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:253) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:41) at org.springframework.xd.dirt.server.JobDeploymentListener.deployJob(JobDeploymentListener.java:165) ... 7 more # Clear XD_JOB_REGISTRY_STEP_NAMES & XD_JOB_REGISTRY tables and run the test a third time # Ran the test a third time. ## The following exception is thrown: SpringXDException: The job named 'deployundeployjob' is already deployed ## The following Exception was reported by the admin server. ### 21:18:29,355 WARN http-nio-9393-exec-4 zookeeper.ZooKeeperJobRepository - Exception while transitioning job 'deployundeployjob' state to undeploying org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /xd/deployments/jobs/deployundeployjob/status at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1270) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:274) at org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:270) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.SetDataBuilderImpl.pathInForeground(SetDataBuilderImpl.java:266) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:253) at org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:41) at org.springframework.xd.dirt.stream.zookeeper.ZooKeeperJobRepository.delete(ZooKeeperJobRepository.java:174) at org.springframework.xd.dirt.stream.zookeeper.ZooKeeperJobRepository.delete(ZooKeeperJobRepository.java:196) at org.springframework.xd.dirt.stream.zookeeper.ZooKeeperJobRepository.delete(ZooKeeperJobRepository.java:54) at org.springframework.xd.dirt.stream.AbstractInstancePersistingDeployer.undeploy(AbstractInstancePersistingDeployer.java:77) at org.springframework.xd.dirt.stream.AbstractInstancePersistingDeployer.undeployAll(AbstractInstancePersistingDeployer.java:103) at org.springframework.xd.dirt.stream.AbstractInstancePersistingDeployer.deleteAll(AbstractInstancePersistingDeployer.java:111) at org.springframework.xd.dirt.rest.XDController.deleteAll(XDController.java:114) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:749) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:689) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:938) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:870) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961) at org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:885) at javax.servlet.http.HttpServlet.service(HttpServlet.java:652) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:110) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextFilterConfiguration$1.doFilterInternal(EndpointWebMvcAutoConfiguration.java:257) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:88) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.boot.actuate.autoconfigure.MetricFilterAutoConfiguration$MetricsFilter.doFilterInternal(MetricFilterAutoConfiguration.java:85) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:683) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1720) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1679) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) # Ran the test a fourth time. Success

    Spring JIRA | 2 years ago | Glenn renfro
    org.springframework.xd.dirt.zookeeper.ZooKeeperAccessException: KeeperErrorCode = NoNode for /xd/deployments/jobs/deployundeployjob/status
  6. 0

    Re: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode

    curator-user | 2 years ago | Jordan Zimmerman
    org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /optum/acc/jobstatus/1483e1fe-14e3-4203-b139-41dc4a3b4ad3

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. org.apache.zookeeper.KeeperException$NoNodeException

      KeeperErrorCode = NoNode for /xd/deployments/streams/testCreateStream_SrcHttp_SinkFile1413234903170/status

      at org.apache.zookeeper.KeeperException.create()
    2. Zookeeper
      ZooKeeper.setData
      1. org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
      2. org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
      3. org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1270)
      3 frames
    3. Curator Framework
      SetDataBuilderImpl$4.call
      1. org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:274)
      2. org.apache.curator.framework.imps.SetDataBuilderImpl$4.call(SetDataBuilderImpl.java:270)
      2 frames
    4. Fabric8
      RetryLoop.callWithRetry
      1. org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
      1 frame
    5. Curator Framework
      SetDataBuilderImpl.forPath
      1. org.apache.curator.framework.imps.SetDataBuilderImpl.pathInForeground(SetDataBuilderImpl.java:266)
      2. org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:253)
      3. org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:41)
      3 frames
    6. org.springframework.xd
      InitialDeploymentListener$EventHandler.call
      1. org.springframework.xd.dirt.server.StreamDeploymentListener.deployStream(StreamDeploymentListener.java:179)
      2. org.springframework.xd.dirt.server.StreamDeploymentListener.onChildAdded(StreamDeploymentListener.java:100)
      3. org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:217)
      4. org.springframework.xd.dirt.server.InitialDeploymentListener$EventHandler.call(InitialDeploymentListener.java:186)
      4 frames
    7. Java RT
      Thread.run
      1. java.util.concurrent.FutureTask.run(FutureTask.java:262)
      2. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      3. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      4. java.lang.Thread.run(Thread.java:744)
      4 frames