org.apache.zookeeper.KeeperException$NodeExistsException

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • When the container which has modules deployed disconnects/reconnects to the cluster while the admin leader isn't available, following exception is thrown: This is more likely to happen in single-node scenario as there is no admin leader re-election there. In distributed mode, we can always setup HA on admins so that the leadership re-election happens. 20:03:16,307 ERROR DeploymentsPathChildrenCache-0 cache.PathChildrenCache - org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /xd/deployments/modules/allocated/53f41042-8abd-443b-abfb-ba42a24fb9fb/foo.sink.log.1/metadata at org.apache.zookeeper.KeeperException.create(KeeperException.java:119) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:676) at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:660) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:656) at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:441) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:431) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44) at org.springframework.xd.dirt.server.ContainerRegistrar.writeModuleMetadata(ContainerRegistrar.java:486) at org.springframework.xd.dirt.server.ContainerRegistrar.onChildAdded(ContainerRegistrar.java:461) at org.springframework.xd.dirt.server.ContainerRegistrar.access$8(ContainerRegistrar.java:426) at org.springframework.xd.dirt.server.ContainerRegistrar$DeploymentListener.childEvent(ContainerRegistrar.java:807) at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:509) at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:503) at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)
    via by Ilayaperumal Gopinathan,
  • When the container which has modules deployed disconnects/reconnects to the cluster while the admin leader isn't available, following exception is thrown: This is more likely to happen in single-node scenario as there is no admin leader re-election there. In distributed mode, we can always setup HA on admins so that the leadership re-election happens. 20:03:16,307 ERROR DeploymentsPathChildrenCache-0 cache.PathChildrenCache - org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /xd/deployments/modules/allocated/53f41042-8abd-443b-abfb-ba42a24fb9fb/foo.sink.log.1/metadata at org.apache.zookeeper.KeeperException.create(KeeperException.java:119) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:676) at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:660) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:656) at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:441) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:431) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44) at org.springframework.xd.dirt.server.ContainerRegistrar.writeModuleMetadata(ContainerRegistrar.java:486) at org.springframework.xd.dirt.server.ContainerRegistrar.onChildAdded(ContainerRegistrar.java:461) at org.springframework.xd.dirt.server.ContainerRegistrar.access$8(ContainerRegistrar.java:426) at org.springframework.xd.dirt.server.ContainerRegistrar$DeploymentListener.childEvent(ContainerRegistrar.java:807) at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:509) at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:503) at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)
    via by Ilayaperumal Gopinathan,
  • coordinator node exception
    via by xiaoxiaoluo,
  • As reported by Matt Stine: After closing and reopening a laptop, the following stack trace appears in the container log: {noformat} 00:47:28,226 INFO main-EventThread state.ConnectionStateManager:194 - State change: RECONNECTED 00:47:28,226 INFO ConnectionStateManager-0 zookeeper.ZooKeeperConnection:255 - >>> Curator connected event: RECONNECTED 00:47:28,322 ERROR ConnectionStateManager-0 listen.ListenerContainer:96 - Listener (org.springframework.xd.dirt.zookeeper.ZooKeeperConnection$DelegatingConnectionStateListener@6abf4158) threw an exception java.lang.RuntimeException: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /xd/containers/5a8deb7b-fd93-42a7-a393-2f15023e007a at org.springframework.xd.dirt.server.ContainerRegistrar.registerWithZooKeeper(ContainerRegistrar.java:301) at org.springframework.xd.dirt.server.ContainerRegistrar.access$100(ContainerRegistrar.java:93) at org.springframework.xd.dirt.server.ContainerRegistrar$ContainerAttributesRegisteringZooKeeperConnectionListener.onConnect(ContainerRegistrar.java:316) at org.springframework.xd.dirt.zookeeper.ZooKeeperConnection$DelegatingConnectionStateListener.stateChanged(ZooKeeperConnection.java:257) at org.apache.curator.framework.state.ConnectionStateManager$2.apply(ConnectionStateManager.java:222) at org.apache.curator.framework.state.ConnectionStateManager$2.apply(ConnectionStateManager.java:218) at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293) at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83) at org.apache.curator.framework.state.ConnectionStateManager.processEvents(ConnectionStateManager.java:215) at org.apache.curator.framework.state.ConnectionStateManager.access$000(ConnectionStateManager.java:42) at org.apache.curator.framework.state.ConnectionStateManager$1.call(ConnectionStateManager.java:110) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /xd/containers/5a8deb7b-fd93-42a7-a393-2f15023e007a at org.springframework.xd.dirt.container.store.ZooKeeperContainerAttributesRepository.save(ZooKeeperContainerAttributesRepository.java:75) at org.springframework.xd.dirt.container.store.ZooKeeperContainerAttributesRepository.save(ZooKeeperContainerAttributesRepository.java:42) at org.springframework.xd.dirt.server.ContainerRegistrar.registerWithZooKeeper(ContainerRegistrar.java:295) ... 15 more Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /xd/containers/5a8deb7b-fd93-42a7-a393-2f15023e007a at org.apache.zookeeper.KeeperException.create(KeeperException.java:119) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:676) at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:660) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:656) at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:441) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:431) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44) at org.springframework.xd.dirt.container.store.ZooKeeperContainerAttributesRepository.save(ZooKeeperContainerAttributesRepository.java:69) ... 17 more {noformat} This can occur if ZK does not remove the ephemeral node before the container creates a new one. This can be fixed in the following ways: * Remove the existing ephemeral node if it already exists * Register containers with a new UUID upon every new connection For now I'll implement the first solution.
    via by Patrick Peralta,
  • As reported by Matt Stine: After closing and reopening a laptop, the following stack trace appears in the container log: {noformat} 00:47:28,226 INFO main-EventThread state.ConnectionStateManager:194 - State change: RECONNECTED 00:47:28,226 INFO ConnectionStateManager-0 zookeeper.ZooKeeperConnection:255 - >>> Curator connected event: RECONNECTED 00:47:28,322 ERROR ConnectionStateManager-0 listen.ListenerContainer:96 - Listener (org.springframework.xd.dirt.zookeeper.ZooKeeperConnection$DelegatingConnectionStateListener@6abf4158) threw an exception java.lang.RuntimeException: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /xd/containers/5a8deb7b-fd93-42a7-a393-2f15023e007a at org.springframework.xd.dirt.server.ContainerRegistrar.registerWithZooKeeper(ContainerRegistrar.java:301) at org.springframework.xd.dirt.server.ContainerRegistrar.access$100(ContainerRegistrar.java:93) at org.springframework.xd.dirt.server.ContainerRegistrar$ContainerAttributesRegisteringZooKeeperConnectionListener.onConnect(ContainerRegistrar.java:316) at org.springframework.xd.dirt.zookeeper.ZooKeeperConnection$DelegatingConnectionStateListener.stateChanged(ZooKeeperConnection.java:257) at org.apache.curator.framework.state.ConnectionStateManager$2.apply(ConnectionStateManager.java:222) at org.apache.curator.framework.state.ConnectionStateManager$2.apply(ConnectionStateManager.java:218) at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293) at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83) at org.apache.curator.framework.state.ConnectionStateManager.processEvents(ConnectionStateManager.java:215) at org.apache.curator.framework.state.ConnectionStateManager.access$000(ConnectionStateManager.java:42) at org.apache.curator.framework.state.ConnectionStateManager$1.call(ConnectionStateManager.java:110) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /xd/containers/5a8deb7b-fd93-42a7-a393-2f15023e007a at org.springframework.xd.dirt.container.store.ZooKeeperContainerAttributesRepository.save(ZooKeeperContainerAttributesRepository.java:75) at org.springframework.xd.dirt.container.store.ZooKeeperContainerAttributesRepository.save(ZooKeeperContainerAttributesRepository.java:42) at org.springframework.xd.dirt.server.ContainerRegistrar.registerWithZooKeeper(ContainerRegistrar.java:295) ... 15 more Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /xd/containers/5a8deb7b-fd93-42a7-a393-2f15023e007a at org.apache.zookeeper.KeeperException.create(KeeperException.java:119) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:676) at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:660) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:656) at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:441) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:431) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44) at org.springframework.xd.dirt.container.store.ZooKeeperContainerAttributesRepository.save(ZooKeeperContainerAttributesRepository.java:69) ... 17 more {noformat} This can occur if ZK does not remove the ephemeral node before the container creates a new one. This can be fixed in the following ways: * Remove the existing ephemeral node if it already exists * Register containers with a new UUID upon every new connection For now I'll implement the first solution.
    via by Patrick Peralta,
  • GitHub comment 2128#166664167
    via GitHub by guobingkun
    ,
    • org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /xd/deployments/modules/allocated/53f41042-8abd-443b-abfb-ba42a24fb9fb/foo.sink.log.1/metadata at org.apache.zookeeper.KeeperException.create(KeeperException.java:119) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:676) at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:660) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:656) at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:441) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:431) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44) at org.springframework.xd.dirt.server.ContainerRegistrar.writeModuleMetadata(ContainerRegistrar.java:486) at org.springframework.xd.dirt.server.ContainerRegistrar.onChildAdded(ContainerRegistrar.java:461) at org.springframework.xd.dirt.server.ContainerRegistrar.access$8(ContainerRegistrar.java:426) at org.springframework.xd.dirt.server.ContainerRegistrar$DeploymentListener.childEvent(ContainerRegistrar.java:807) at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:509) at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:503) at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)

    Users with the same issue

    nasimknasimk
    2 times, last one,
    Unknown visitor
    Unknown visitor1 times, last one,