java.net.SocketException

If you like a tip written by other Samebug users, mark is as helpful! Marks help our algorithm provide you better solutions and also help other users.
tip

Check if you don't have a VPN running, as it might cause conflict and throw this exception.

You have a different solution? A short tip here would help you and many other users who saw this issue last week.

  • {noformat} Error Message Broken pipe Stacktrace java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:466) at java.nio.channels.Channels.writeFullyImpl(Channels.java:78) at java.nio.channels.Channels.writeFully(Channels.java:98) at java.nio.channels.Channels.access$000(Channels.java:61) at java.nio.channels.Channels$1.write(Channels.java:174) at java.io.OutputStream.write(OutputStream.java:75) at java.nio.channels.Channels$1.write(Channels.java:155) at java.io.DataOutputStream.writeInt(DataOutputStream.java:198) at org.apache.zookeeper.server.quorum.CnxManagerTest.testCnxFromFutureVersion(CnxManagerTest.java:318) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) Standard Output 2016-07-12 22:34:46,623 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testCnxFromFutureVersion 2016-07-12 22:34:46,627 [myid:] - INFO [main:PortAssignment@32] - assigning port 11221 2016-07-12 22:34:46,630 [myid:] - INFO [main:PortAssignment@32] - assigning port 11222 2016-07-12 22:34:46,631 [myid:] - INFO [main:PortAssignment@32] - assigning port 11223 2016-07-12 22:34:46,643 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:46,658 [myid:] - INFO [main:PortAssignment@32] - assigning port 11224 2016-07-12 22:34:46,658 [myid:] - INFO [main:PortAssignment@32] - assigning port 11225 2016-07-12 22:34:46,659 [myid:] - INFO [main:PortAssignment@32] - assigning port 11226 2016-07-12 22:34:46,659 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:46,659 [myid:] - INFO [main:PortAssignment@32] - assigning port 11227 2016-07-12 22:34:46,659 [myid:] - INFO [main:PortAssignment@32] - assigning port 11228 2016-07-12 22:34:46,659 [myid:] - INFO [main:PortAssignment@32] - assigning port 11229 2016-07-12 22:34:46,660 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:46,660 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@53] - RUNNING TEST METHOD testCnxFromFutureVersion 2016-07-12 22:34:46,672 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11225 2016-07-12 22:34:46,692 [myid:] - INFO [main:CnxManagerTest@301] - Election port: 11226 2016-07-12 22:34:46,692 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /0.0.0.0:11226 2016-07-12 22:34:47,696 [myid:] - INFO [/0.0.0.0:11226:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:48724 2016-07-12 22:34:49,700 [myid:] - WARN [/0.0.0.0:11226:QuorumCnxManager@274] - Exception reading or writing challenge: java.net.SocketTimeoutException: Read timed out 2016-07-12 22:34:52,700 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@74] - TEST METHOD FAILED testCnxFromFutureVersion java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:466) at java.nio.channels.Channels.writeFullyImpl(Channels.java:78) at java.nio.channels.Channels.writeFully(Channels.java:98) at java.nio.channels.Channels.access$000(Channels.java:61) at java.nio.channels.Channels$1.write(Channels.java:174) at java.io.OutputStream.write(OutputStream.java:75) at java.nio.channels.Channels$1.write(Channels.java:155) at java.io.DataOutputStream.writeInt(DataOutputStream.java:198) at org.apache.zookeeper.server.quorum.CnxManagerTest.testCnxFromFutureVersion(CnxManagerTest.java:318) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) 2016-07-12 22:34:52,705 [myid:] - INFO [main:ZKTestCase$1@65] - FAILED testCnxFromFutureVersion java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:466) at java.nio.channels.Channels.writeFullyImpl(Channels.java:78) at java.nio.channels.Channels.writeFully(Channels.java:98) at java.nio.channels.Channels.access$000(Channels.java:61) at java.nio.channels.Channels$1.write(Channels.java:174) at java.io.OutputStream.write(OutputStream.java:75) at java.nio.channels.Channels$1.write(Channels.java:155) at java.io.DataOutputStream.writeInt(DataOutputStream.java:198) at org.apache.zookeeper.server.quorum.CnxManagerTest.testCnxFromFutureVersion(CnxManagerTest.java:318) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) 2016-07-12 22:34:52,706 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testCnxFromFutureVersion 2016-07-12 22:34:52,720 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testSocketTimeout 2016-07-12 22:34:52,720 [myid:] - INFO [main:PortAssignment@32] - assigning port 11230 2016-07-12 22:34:52,720 [myid:] - INFO [main:PortAssignment@32] - assigning port 11231 2016-07-12 22:34:52,720 [myid:] - INFO [main:PortAssignment@32] - assigning port 11232 2016-07-12 22:34:52,721 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:52,721 [myid:] - INFO [main:PortAssignment@32] - assigning port 11233 2016-07-12 22:34:52,722 [myid:] - INFO [main:PortAssignment@32] - assigning port 11234 2016-07-12 22:34:52,722 [myid:] - INFO [main:PortAssignment@32] - assigning port 11235 2016-07-12 22:34:52,722 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:52,723 [myid:] - INFO [main:PortAssignment@32] - assigning port 11236 2016-07-12 22:34:52,723 [myid:] - INFO [main:PortAssignment@32] - assigning port 11237 2016-07-12 22:34:52,723 [myid:] - INFO [main:PortAssignment@32] - assigning port 11238 2016-07-12 22:34:52,724 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:52,724 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@53] - RUNNING TEST METHOD testSocketTimeout 2016-07-12 22:34:52,725 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11234 2016-07-12 22:34:52,726 [myid:] - INFO [main:CnxManagerTest@370] - Election port: 11235 2016-07-12 22:34:52,726 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /0.0.0.0:11235 2016-07-12 22:34:53,729 [myid:] - INFO [/0.0.0.0:11235:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:34496 2016-07-12 22:34:57,734 [myid:] - WARN [/0.0.0.0:11235:QuorumCnxManager@274] - Exception reading or writing challenge: java.net.SocketTimeoutException: Read timed out 2016-07-12 22:34:57,734 [myid:] - WARN [main:QuorumCnxManager@274] - Exception reading or writing challenge: java.io.EOFException 2016-07-12 22:34:57,734 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 3153 2016-07-12 22:34:57,735 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 6 2016-07-12 22:34:57,735 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testSocketTimeout 2016-07-12 22:34:57,735 [myid:] - INFO [main:ZKTestCase$1@60] - SUCCEEDED testSocketTimeout 2016-07-12 22:34:57,735 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testSocketTimeout 2016-07-12 22:34:57,735 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testWorkerThreads 2016-07-12 22:34:57,736 [myid:] - INFO [main:PortAssignment@32] - assigning port 11239 2016-07-12 22:34:57,736 [myid:] - INFO [main:PortAssignment@32] - assigning port 11240 2016-07-12 22:34:57,736 [myid:] - INFO [main:PortAssignment@32] - assigning port 11241 2016-07-12 22:34:57,736 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:57,736 [myid:] - INFO [main:PortAssignment@32] - assigning port 11242 2016-07-12 22:34:57,737 [myid:] - INFO [main:PortAssignment@32] - assigning port 11243 2016-07-12 22:34:57,737 [myid:] - INFO [main:PortAssignment@32] - assigning port 11244 2016-07-12 22:34:57,737 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:57,737 [myid:] - INFO [main:PortAssignment@32] - assigning port 11245 2016-07-12 22:34:57,737 [myid:] - INFO [main:PortAssignment@32] - assigning port 11246 2016-07-12 22:34:57,737 [myid:] - INFO [main:PortAssignment@32] - assigning port 11247 2016-07-12 22:34:57,737 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:57,738 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@53] - RUNNING TEST METHOD testWorkerThreads 2016-07-12 22:34:57,738 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11240 2016-07-12 22:34:57,784 [myid:] - INFO [main:CnxManagerTest@393] - Starting peer 0 2016-07-12 22:34:57,789 [myid:] - INFO [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-12 22:34:57,905 [myid:] - INFO [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-12 22:34:57,945 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /0.0.0.0:11241 2016-07-12 22:34:57,953 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11243 2016-07-12 22:34:57,954 [myid:] - INFO [main:CnxManagerTest@393] - Starting peer 1 2016-07-12 22:34:57,954 [myid:] - INFO [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-12 22:34:58,024 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:QuorumPeer@774] - LOOKING 2016-07-12 22:34:58,025 [myid:] - INFO [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-12 22:34:58,025 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:FastLeaderElection@818] - New election. My id = 0, proposed zxid=0x0 2016-07-12 22:34:58,028 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@600] - Notification: 1 (message format version), 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,028 [myid:] - WARN [WorkerSender[myid=0]:QuorumCnxManager@400] - Cannot open channel to 1 at election address /0.0.0.0:11244 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:204) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433) at java.lang.Thread.run(Thread.java:744) 2016-07-12 22:34:58,029 [myid:] - INFO [WorkerSender[myid=0]:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:58,030 [myid:] - WARN [WorkerSender[myid=0]:QuorumCnxManager@400] - Cannot open channel to 2 at election address /0.0.0.0:11247 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:204) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433) at java.lang.Thread.run(Thread.java:744) 2016-07-12 22:34:58,030 [myid:] - INFO [WorkerSender[myid=0]:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:58,050 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /0.0.0.0:11244 2016-07-12 22:34:58,052 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11246 2016-07-12 22:34:58,052 [myid:] - INFO [main:CnxManagerTest@393] - Starting peer 2 2016-07-12 22:34:58,053 [myid:] - INFO [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-12 22:34:58,057 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:QuorumPeer@774] - LOOKING 2016-07-12 22:34:58,058 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:FastLeaderElection@818] - New election. My id = 1, proposed zxid=0x0 2016-07-12 22:34:58,058 [myid:] - INFO [/0.0.0.0:11241:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:46871 2016-07-12 22:34:58,063 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,064 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,064 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,065 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,065 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,065 [myid:] - WARN [WorkerSender[myid=1]:QuorumCnxManager@400] - Cannot open channel to 2 at election address /0.0.0.0:11247 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:204) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433) at java.lang.Thread.run(Thread.java:744) 2016-07-12 22:34:58,065 [myid:] - WARN [WorkerSender[myid=0]:QuorumCnxManager@400] - Cannot open channel to 2 at election address /0.0.0.0:11247 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:204) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433) at java.lang.Thread.run(Thread.java:744) 2016-07-12 22:34:58,066 [myid:] - INFO [WorkerSender[myid=1]:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:58,066 [myid:] - INFO [WorkerSender[myid=0]:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:58,084 [myid:] - INFO [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-12 22:34:58,133 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /0.0.0.0:11247 2016-07-12 22:34:58,136 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:QuorumPeer@774] - LOOKING 2016-07-12 22:34:58,136 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:FastLeaderElection@818] - New election. My id = 2, proposed zxid=0x0 2016-07-12 22:34:58,137 [myid:] - INFO [/0.0.0.0:11241:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:46874 2016-07-12 22:34:58,138 [myid:] - INFO [/0.0.0.0:11244:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:33135 2016-07-12 22:34:58,141 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,143 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,143 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,143 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,144 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,144 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,144 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,144 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,144 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,145 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,145 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:58,345 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:QuorumPeer@844] - FOLLOWING 2016-07-12 22:34:58,345 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:QuorumPeer@844] - FOLLOWING 2016-07-12 22:34:58,346 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:QuorumPeer@856] - LEADING 2016-07-12 22:34:58,349 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Learner@86] - TCP NoDelay set to: true 2016-07-12 22:34:58,350 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:Leader@59] - TCP NoDelay set to: true 2016-07-12 22:34:58,354 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:zookeeper.version=3.4.9-SNAPSHOT-1752356, built on 07/12/2016 22:30 GMT 2016-07-12 22:34:58,354 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:host.name=hemera.apache.org 2016-07-12 22:34:58,354 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:java.version=1.8.0 2016-07-12 22:34:58,354 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:java.vendor=Oracle Corporation 2016-07-12 22:34:58,354 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:java.home=/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/jdk-1.8.0/jre 2016-07-12 22:34:58,354 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:java.class.path=/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/classes:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/lib/antlr-2.7.6.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/lib/checkstyle-5.0.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/lib/commons-beanutils-core-1.7.0.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/lib/commons-cli-1.0.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/lib/commons-lang-1.0.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/lib/commons-logging-1.0.3.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/lib/google-collections-0.9.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/lib/junit-4.8.1.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/lib/mockito-all-1.8.2.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/classes:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/src/java/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/latest/lib/ant.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/lib/jline-0.9.94.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/lib/log4j-1.2.16.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/lib/netty-3.10.5.Final.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/lib/slf4j-api-1.6.1.jar:/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/lib/slf4j-log4j12-1.6.1.jar:/home/jenkins/tools/clover/latest/lib/clover.jar:/x1/jenkins/tools/ant/apache-ant-1.8.2/lib/ant-launcher.jar:/home/jenkins/tools/ant/latest/lib/ant-junit.jar:/home/jenkins/tools/ant/latest/lib/ant-junit4.jar 2016-07-12 22:34:58,355 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2016-07-12 22:34:58,355 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:java.io.tmpdir=/tmp 2016-07-12 22:34:58,355 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:java.compiler=<NA> 2016-07-12 22:34:58,355 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:os.name=Linux 2016-07-12 22:34:58,355 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:os.arch=amd64 2016-07-12 22:34:58,355 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:os.version=3.2.0-104-generic 2016-07-12 22:34:58,355 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:user.name=jenkins 2016-07-12 22:34:58,355 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:user.home=/home/jenkins 2016-07-12 22:34:58,355 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Environment@100] - Server environment:user.dir=/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4 2016-07-12 22:34:58,358 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:ZooKeeperServer@170] - Created server with tickTime 1000 minSessionTimeout 2000 maxSessionTimeout 20000 datadir /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test6977370939981061612.junit.dir/version-2 snapdir /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test6977370939981061612.junit.dir/version-2 2016-07-12 22:34:58,358 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:ZooKeeperServer@170] - Created server with tickTime 1000 minSessionTimeout 2000 maxSessionTimeout 20000 datadir /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test4996523116536588259.junit.dir/version-2 snapdir /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test4996523116536588259.junit.dir/version-2 2016-07-12 22:34:58,358 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:ZooKeeperServer@170] - Created server with tickTime 1000 minSessionTimeout 2000 maxSessionTimeout 20000 datadir /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test8108741215991761367.junit.dir/version-2 snapdir /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test8108741215991761367.junit.dir/version-2 2016-07-12 22:34:58,359 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Follower@63] - FOLLOWING - LEADER ELECTION TOOK - 301 2016-07-12 22:34:58,359 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:Follower@63] - FOLLOWING - LEADER ELECTION TOOK - 334 2016-07-12 22:34:58,365 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:Leader@361] - LEADING - LEADER ELECTION TOOK - 224 2016-07-12 22:34:58,370 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:58,373 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:58,383 [myid:] - INFO [LearnerHandler-/140.211.11.27:45223:LearnerHandler@329] - Follower sid: 1 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@4e46c644 2016-07-12 22:34:58,383 [myid:] - INFO [LearnerHandler-/140.211.11.27:45222:LearnerHandler@329] - Follower sid: 0 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@23da3d 2016-07-12 22:34:58,528 [myid:] - INFO [LearnerHandler-/140.211.11.27:45222:LearnerHandler@384] - Synchronizing with Follower sid: 0 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0 2016-07-12 22:34:58,528 [myid:] - INFO [LearnerHandler-/140.211.11.27:45223:LearnerHandler@384] - Synchronizing with Follower sid: 1 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0 2016-07-12 22:34:58,529 [myid:] - INFO [LearnerHandler-/140.211.11.27:45222:LearnerHandler@393] - leader and follower are in sync, zxid=0x0 2016-07-12 22:34:58,529 [myid:] - INFO [LearnerHandler-/140.211.11.27:45223:LearnerHandler@393] - leader and follower are in sync, zxid=0x0 2016-07-12 22:34:58,529 [myid:] - INFO [LearnerHandler-/140.211.11.27:45223:LearnerHandler@458] - Sending DIFF 2016-07-12 22:34:58,529 [myid:] - INFO [LearnerHandler-/140.211.11.27:45222:LearnerHandler@458] - Sending DIFF 2016-07-12 22:34:58,530 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Learner@326] - Getting a diff from the leader 0x0 2016-07-12 22:34:58,530 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:Learner@326] - Getting a diff from the leader 0x0 2016-07-12 22:34:58,534 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:FileTxnSnapLog@240] - Snapshotting: 0x0 to /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test6977370939981061612.junit.dir/version-2/snapshot.0 2016-07-12 22:34:58,534 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:FileTxnSnapLog@240] - Snapshotting: 0x0 to /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test8108741215991761367.junit.dir/version-2/snapshot.0 2016-07-12 22:34:58,623 [myid:] - INFO [LearnerHandler-/140.211.11.27:45222:LearnerHandler@518] - Received NEWLEADER-ACK message from 0 2016-07-12 22:34:58,623 [myid:] - INFO [LearnerHandler-/140.211.11.27:45223:LearnerHandler@518] - Received NEWLEADER-ACK message from 1 2016-07-12 22:34:58,623 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:Leader@946] - Have quorum of supporters, sids: [ 0,2 ]; starting up and setting last processed zxid: 0x100000000 2016-07-12 22:34:58,634 [myid:] - INFO [main:CnxManagerTest@405] - Round 0, halting peer 0 2016-07-12 22:34:58,634 [myid:] - INFO [main:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:891) at org.apache.zookeeper.server.quorum.CnxManagerTest.testWorkerThreads(CnxManagerTest.java:407) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) 2016-07-12 22:34:58,635 [myid:] - INFO [main:FollowerZooKeeperServer@140] - Shutting down 2016-07-12 22:34:58,645 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11240:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method 2016-07-12 22:34:58,645 [myid:] - INFO [main:FollowerZooKeeperServer@140] - Shutting down 2016-07-12 22:34:58,647 [myid:] - INFO [main:ZooKeeperServer@469] - shutting down 2016-07-12 22:34:58,647 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:850) 2016-07-12 22:34:58,648 [myid:] - INFO [main:FollowerRequestProcessor@107] - Shutting down 2016-07-12 22:34:58,648 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:FollowerZooKeeperServer@140] - Shutting down 2016-07-12 22:34:58,648 [myid:] - INFO [FollowerRequestProcessor:0:FollowerRequestProcessor@97] - FollowerRequestProcessor exited loop! 2016-07-12 22:34:58,648 [myid:] - INFO [main:CommitProcessor@184] - Shutting down 2016-07-12 22:34:58,649 [myid:] - INFO [main:FinalRequestProcessor@402] - shutdown of request processor complete 2016-07-12 22:34:58,649 [myid:] - INFO [CommitProcessor:0:CommitProcessor@153] - CommitProcessor exited loop! 2016-07-12 22:34:58,650 [myid:] - INFO [main:SyncRequestProcessor@209] - Shutting down 2016-07-12 22:34:58,650 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:SyncRequestProcessor@209] - Shutting down 2016-07-12 22:34:58,651 [myid:] - INFO [SyncThread:0:SyncRequestProcessor@187] - SyncRequestProcessor exited! 2016-07-12 22:34:58,651 [myid:] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:QuorumPeer@874] - QuorumPeer main thread exited 2016-07-12 22:34:58,651 [myid:] - ERROR [/0.0.0.0:11241:QuorumCnxManager$Listener@547] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539) 2016-07-12 22:34:58,652 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:34:58,653 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:150) at java.net.SocketInputStream.read(SocketInputStream.java:121) at java.net.SocketInputStream.read(SocketInputStream.java:203) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:34:58,654 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:34:58,653 [myid:] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@810] - Connection broken for id 0, my id = 2, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:34:58,653 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:34:58,656 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:34:58,652 [myid:] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@810] - Connection broken for id 0, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:34:58,656 [myid:] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:34:58,652 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@810] - Connection broken for id 1, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:150) at java.net.SocketInputStream.read(SocketInputStream.java:121) at java.net.SocketInputStream.read(SocketInputStream.java:203) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:34:58,657 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:34:58,657 [myid:] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:34:58,658 [myid:] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:34:58,655 [myid:] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:34:58,654 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:34:58,659 [myid:] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:34:58,659 [myid:] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:34:59,154 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11240 2016-07-12 22:34:59,155 [myid:] - INFO [main:CnxManagerTest@414] - Round {}, restarting peer {}[Ljava.lang.Object;@12bc6874 2016-07-12 22:34:59,157 [myid:] - INFO [main:FileSnap@83] - Reading snapshot /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test8108741215991761367.junit.dir/version-2/snapshot.0 2016-07-12 22:34:59,159 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /0.0.0.0:11241 2016-07-12 22:34:59,161 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:QuorumPeer@774] - LOOKING 2016-07-12 22:34:59,162 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:FastLeaderElection@818] - New election. My id = 0, proposed zxid=0x0 2016-07-12 22:34:59,162 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@600] - Notification: 1 (message format version), 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x1 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:59,163 [myid:] - INFO [WorkerSender[myid=0]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (1, 0) 2016-07-12 22:34:59,163 [myid:] - INFO [/0.0.0.0:11244:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:33142 2016-07-12 22:34:59,163 [myid:] - INFO [/0.0.0.0:11247:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:44785 2016-07-12 22:34:59,163 [myid:] - INFO [WorkerSender[myid=0]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (2, 0) 2016-07-12 22:34:59,164 [myid:] - INFO [/0.0.0.0:11241:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:46884 2016-07-12 22:34:59,166 [myid:] - INFO [/0.0.0.0:11241:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:46885 2016-07-12 22:34:59,167 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x1 (n.peerEpoch) FOLLOWING (my state) 2016-07-12 22:34:59,169 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x1 (n.peerEpoch) LEADING (my state) 2016-07-12 22:34:59,169 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:59,169 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:59,169 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), FOLLOWING (n.state), 1 (n.sid), 0x1 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:59,170 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LEADING (n.state), 2 (n.sid), 0x1 (n.peerEpoch) LOOKING (my state) 2016-07-12 22:34:59,170 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:QuorumPeer@844] - FOLLOWING 2016-07-12 22:34:59,170 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:ZooKeeperServer@170] - Created server with tickTime 1000 minSessionTimeout 2000 maxSessionTimeout 20000 datadir /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test8108741215991761367.junit.dir/version-2 snapdir /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test8108741215991761367.junit.dir/version-2 2016-07-12 22:34:59,170 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:Follower@63] - FOLLOWING - LEADER ELECTION TOOK - 8 2016-07-12 22:34:59,171 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:34:59,172 [myid:] - INFO [LearnerHandler-/140.211.11.27:45228:LearnerHandler@329] - Follower sid: 0 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@23da3d 2016-07-12 22:34:59,172 [myid:] - INFO [LearnerHandler-/140.211.11.27:45228:LearnerHandler@384] - Synchronizing with Follower sid: 0 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0 2016-07-12 22:34:59,172 [myid:] - INFO [LearnerHandler-/140.211.11.27:45228:LearnerHandler@458] - Sending SNAP 2016-07-12 22:34:59,172 [myid:] - INFO [LearnerHandler-/140.211.11.27:45228:LearnerHandler@482] - Sending snapshot last zxid of peer is 0x0 zxid of leader is 0x100000000sent zxid of db as 0x100000000 2016-07-12 22:34:59,172 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:Learner@329] - Getting a snapshot from leader 2016-07-12 22:34:59,174 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:FileTxnSnapLog@240] - Snapshotting: 0x100000000 to /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test8108741215991761367.junit.dir/version-2/snapshot.100000000 2016-07-12 22:34:59,284 [myid:] - INFO [LearnerHandler-/140.211.11.27:45228:LearnerHandler@518] - Received NEWLEADER-ACK message from 0 2016-07-12 22:34:59,652 [myid:] - INFO [/0.0.0.0:11241:QuorumCnxManager$Listener@560] - Leaving listener 2016-07-12 22:34:59,660 [myid:] - INFO [main:CnxManagerTest@405] - Round 1, halting peer 0 2016-07-12 22:34:59,660 [myid:] - INFO [main:Follower@166] - shutdown called j ...[truncated 254755 chars]... Keeper_branch34_jdk8/branch-3.4/build/test/tmp/test4996523116536588259.junit.dir/version-2 snapdir /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test4996523116536588259.junit.dir/version-2 2016-07-12 22:35:13,328 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:Follower@63] - FOLLOWING - LEADER ELECTION TOOK - 3 2016-07-12 22:35:13,329 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:35:13,329 [myid:] - INFO [LearnerHandler-/140.211.11.27:32997:LearnerHandler@329] - Follower sid: 2 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@6f23fc6d 2016-07-12 22:35:13,330 [myid:] - INFO [LearnerHandler-/140.211.11.27:32997:LearnerHandler@384] - Synchronizing with Follower sid: 2 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x200000000 2016-07-12 22:35:13,330 [myid:] - INFO [LearnerHandler-/140.211.11.27:32997:LearnerHandler@393] - leader and follower are in sync, zxid=0x200000000 2016-07-12 22:35:13,330 [myid:] - INFO [LearnerHandler-/140.211.11.27:32997:LearnerHandler@458] - Sending DIFF 2016-07-12 22:35:13,330 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:Learner@326] - Getting a diff from the leader 0x200000000 2016-07-12 22:35:13,331 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:FileTxnSnapLog@240] - Snapshotting: 0x200000000 to /x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/branch-3.4/build/test/tmp/test4996523116536588259.junit.dir/version-2/snapshot.200000000 2016-07-12 22:35:13,376 [myid:] - INFO [LearnerHandler-/140.211.11.27:32997:LearnerHandler@518] - Received NEWLEADER-ACK message from 2 2016-07-12 22:35:13,821 [myid:] - INFO [/0.0.0.0:11247:QuorumCnxManager$Listener@560] - Leaving listener 2016-07-12 22:35:13,824 [myid:] - INFO [main:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:891) at org.apache.zookeeper.server.quorum.CnxManagerTest.testWorkerThreads(CnxManagerTest.java:424) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) 2016-07-12 22:35:13,824 [myid:] - INFO [main:FollowerZooKeeperServer@140] - Shutting down 2016-07-12 22:35:13,824 [myid:] - INFO [main:ZooKeeperServer@469] - shutting down 2016-07-12 22:35:13,825 [myid:] - INFO [main:FollowerRequestProcessor@107] - Shutting down 2016-07-12 22:35:13,825 [myid:] - INFO [main:CommitProcessor@184] - Shutting down 2016-07-12 22:35:13,825 [myid:] - INFO [main:FinalRequestProcessor@402] - shutdown of request processor complete 2016-07-12 22:35:13,825 [myid:] - INFO [CommitProcessor:0:CommitProcessor@153] - CommitProcessor exited loop! 2016-07-12 22:35:13,826 [myid:] - INFO [main:SyncRequestProcessor@209] - Shutting down 2016-07-12 22:35:13,825 [myid:] - INFO [FollowerRequestProcessor:0:FollowerRequestProcessor@97] - FollowerRequestProcessor exited loop! 2016-07-12 22:35:13,826 [myid:] - INFO [SyncThread:0:SyncRequestProcessor@187] - SyncRequestProcessor exited! 2016-07-12 22:35:13,827 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11240:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method 2016-07-12 22:35:13,828 [myid:] - ERROR [/0.0.0.0:11241:QuorumCnxManager$Listener@547] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539) 2016-07-12 22:35:13,829 [myid:] - INFO [main:Leader@496] - Shutting down 2016-07-12 22:35:13,828 [myid:] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@810] - Connection broken for id 0, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:35:13,829 [myid:] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:35:13,828 [myid:] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@810] - Connection broken for id 0, my id = 2, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:35:13,829 [myid:] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:35:13,828 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:35:13,830 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:35:13,828 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:35:13,830 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:35:13,828 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@810] - Connection broken for id 1, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:150) at java.net.SocketInputStream.read(SocketInputStream.java:121) at java.net.SocketInputStream.read(SocketInputStream.java:203) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:35:13,830 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:35:13,830 [myid:] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:35:13,830 [myid:] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:35:13,829 [myid:] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:35:13,831 [myid:] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:35:13,829 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:150) at java.net.SocketInputStream.read(SocketInputStream.java:121) at java.net.SocketInputStream.read(SocketInputStream.java:203) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:35:13,831 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:35:13,829 [myid:] - INFO [main:Leader@502] - Shutdown called java.lang.Exception: shutdown Leader! reason: quorum Peer shutdown at org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:502) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:888) at org.apache.zookeeper.server.quorum.CnxManagerTest.testWorkerThreads(CnxManagerTest.java:424) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) 2016-07-12 22:35:13,831 [myid:] - INFO [main:ZooKeeperServer@469] - shutting down 2016-07-12 22:35:13,831 [myid:] - INFO [main:SessionTrackerImpl@225] - Shutting down 2016-07-12 22:35:13,832 [myid:] - INFO [main:PrepRequestProcessor@765] - Shutting down 2016-07-12 22:35:13,832 [myid:] - INFO [main:ProposalRequestProcessor@88] - Shutting down 2016-07-12 22:35:13,832 [myid:] - INFO [main:CommitProcessor@184] - Shutting down 2016-07-12 22:35:13,832 [myid:] - INFO [main:Leader$ToBeAppliedRequestProcessor@661] - Shutting down 2016-07-12 22:35:13,832 [myid:] - INFO [main:FinalRequestProcessor@402] - shutdown of request processor complete 2016-07-12 22:35:13,832 [myid:] - INFO [main:SyncRequestProcessor@209] - Shutting down 2016-07-12 22:35:13,831 [myid:] - INFO [LearnerCnxAcceptor-/0.0.0.0:11242:Leader$LearnerCnxAcceptor@325] - exception while shutting down acceptor: java.net.SocketException: Socket closed 2016-07-12 22:35:13,832 [myid:] - INFO [SyncThread:1:SyncRequestProcessor@187] - SyncRequestProcessor exited! 2016-07-12 22:35:13,832 [myid:] - INFO [CommitProcessor:1:CommitProcessor@153] - CommitProcessor exited loop! 2016-07-12 22:35:13,832 [myid:] - INFO [ProcessThread(sid:1 cport:-1)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop! 2016-07-12 22:35:13,834 [myid:] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:Follower@89] - Exception when following the leader java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:153) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:85) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:846) 2016-07-12 22:35:13,833 [myid:] - WARN [LearnerHandler-/140.211.11.27:32993:LearnerHandler@644] - ******* GOODBYE /140.211.11.27:32993 ******** 2016-07-12 22:35:13,835 [myid:] - WARN [LearnerHandler-/140.211.11.27:32990:LearnerHandler@644] - ******* GOODBYE /140.211.11.27:32990 ******** 2016-07-12 22:35:13,834 [myid:] - WARN [LearnerHandler-/140.211.11.27:32980:LearnerHandler@644] - ******* GOODBYE /140.211.11.27:32980 ******** 2016-07-12 22:35:13,835 [myid:] - WARN [LearnerHandler-/140.211.11.27:32980:LearnerHandler@656] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:339) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647) 2016-07-12 22:35:13,834 [myid:] - WARN [LearnerHandler-/140.211.11.27:32997:LearnerHandler@644] - ******* GOODBYE /140.211.11.27:32997 ******** 2016-07-12 22:35:13,835 [myid:] - WARN [LearnerHandler-/140.211.11.27:32997:LearnerHandler@656] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:339) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647) 2016-07-12 22:35:13,835 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11243:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method 2016-07-12 22:35:13,835 [myid:] - WARN [LearnerHandler-/140.211.11.27:32990:LearnerHandler@656] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:339) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647) 2016-07-12 22:35:13,836 [myid:] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:QuorumPeer@862] - Unexpected exception java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:456) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:859) 2016-07-12 22:35:13,836 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:Leader@496] - Shutting down 2016-07-12 22:35:13,836 [myid:] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11243:QuorumPeer@874] - QuorumPeer main thread exited 2016-07-12 22:35:13,835 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:850) 2016-07-12 22:35:13,837 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:FollowerZooKeeperServer@140] - Shutting down 2016-07-12 22:35:13,837 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:ZooKeeperServer@469] - shutting down 2016-07-12 22:35:13,837 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:FollowerRequestProcessor@107] - Shutting down 2016-07-12 22:35:13,835 [myid:] - WARN [LearnerHandler-/140.211.11.27:32993:LearnerHandler@656] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:339) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647) 2016-07-12 22:35:13,835 [myid:] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:Follower@89] - Exception when following the leader java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:153) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:85) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:846) 2016-07-12 22:35:13,837 [myid:] - INFO [FollowerRequestProcessor:2:FollowerRequestProcessor@97] - FollowerRequestProcessor exited loop! 2016-07-12 22:35:13,838 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:850) 2016-07-12 22:35:13,837 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:CommitProcessor@184] - Shutting down 2016-07-12 22:35:13,839 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:FinalRequestProcessor@402] - shutdown of request processor complete 2016-07-12 22:35:13,839 [myid:] - INFO [CommitProcessor:2:CommitProcessor@153] - CommitProcessor exited loop! 2016-07-12 22:35:13,837 [myid:] - INFO [main:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:891) at org.apache.zookeeper.server.quorum.CnxManagerTest.testWorkerThreads(CnxManagerTest.java:424) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) 2016-07-12 22:35:13,839 [myid:] - INFO [main:FollowerZooKeeperServer@140] - Shutting down 2016-07-12 22:35:13,839 [myid:] - INFO [main:SyncRequestProcessor@209] - Shutting down 2016-07-12 22:35:13,836 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 1, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:150) at java.net.SocketInputStream.read(SocketInputStream.java:121) at java.net.SocketInputStream.read(SocketInputStream.java:203) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:35:13,840 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:35:13,836 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@810] - Connection broken for id 1, my id = 2, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:35:13,840 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:35:13,836 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:35:13,840 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:35:13,836 [myid:] - ERROR [/0.0.0.0:11244:QuorumCnxManager$Listener@547] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539) 2016-07-12 22:35:13,840 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:35:13,841 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:35:13,840 [myid:] - INFO [SyncThread:2:SyncRequestProcessor@187] - SyncRequestProcessor exited! 2016-07-12 22:35:13,839 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:SyncRequestProcessor@209] - Shutting down 2016-07-12 22:35:13,841 [myid:] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11246:QuorumPeer@874] - QuorumPeer main thread exited 2016-07-12 22:35:13,841 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11246:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method 2016-07-12 22:35:13,839 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:FollowerZooKeeperServer@140] - Shutting down 2016-07-12 22:35:13,842 [myid:] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:SyncRequestProcessor@209] - Shutting down 2016-07-12 22:35:13,842 [myid:] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11240:QuorumPeer@874] - QuorumPeer main thread exited 2016-07-12 22:35:13,842 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 11673 2016-07-12 22:35:13,842 [myid:] - ERROR [/0.0.0.0:11247:QuorumCnxManager$Listener@547] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539) 2016-07-12 22:35:13,842 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 20 2016-07-12 22:35:13,842 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testWorkerThreads 2016-07-12 22:35:13,842 [myid:] - INFO [main:ZKTestCase$1@60] - SUCCEEDED testWorkerThreads 2016-07-12 22:35:13,843 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testWorkerThreads 2016-07-12 22:35:13,843 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testCnxManager 2016-07-12 22:35:13,843 [myid:] - INFO [main:PortAssignment@32] - assigning port 11248 2016-07-12 22:35:13,843 [myid:] - INFO [main:PortAssignment@32] - assigning port 11249 2016-07-12 22:35:13,843 [myid:] - INFO [main:PortAssignment@32] - assigning port 11250 2016-07-12 22:35:13,843 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:35:13,844 [myid:] - INFO [main:PortAssignment@32] - assigning port 11251 2016-07-12 22:35:13,844 [myid:] - INFO [main:PortAssignment@32] - assigning port 11252 2016-07-12 22:35:13,844 [myid:] - INFO [main:PortAssignment@32] - assigning port 11253 2016-07-12 22:35:13,844 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:35:13,844 [myid:] - INFO [main:PortAssignment@32] - assigning port 11254 2016-07-12 22:35:13,844 [myid:] - INFO [main:PortAssignment@32] - assigning port 11255 2016-07-12 22:35:13,844 [myid:] - INFO [main:PortAssignment@32] - assigning port 11256 2016-07-12 22:35:13,844 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:35:13,844 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@53] - RUNNING TEST METHOD testCnxManager 2016-07-12 22:35:13,845 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11252 2016-07-12 22:35:13,846 [myid:] - INFO [Thread-19:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11249 2016-07-12 22:35:13,846 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /0.0.0.0:11253 2016-07-12 22:35:13,846 [myid:] - WARN [main:QuorumCnxManager@400] - Cannot open channel to 0 at election address /0.0.0.0:11250 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:204) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354) at org.apache.zookeeper.server.quorum.CnxManagerTest.testCnxManager(CnxManagerTest.java:142) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) 2016-07-12 22:35:13,847 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:35:13,847 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /0.0.0.0:11250 2016-07-12 22:35:13,847 [myid:] - INFO [/0.0.0.0:11253:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:41746 2016-07-12 22:35:13,847 [myid:] - INFO [Thread-19:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (1, 0) 2016-07-12 22:35:13,848 [myid:] - INFO [/0.0.0.0:11250:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:46883 2016-07-12 22:35:13,849 [myid:] - INFO [/0.0.0.0:11253:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:41748 2016-07-12 22:35:13,849 [myid:] - INFO [Thread-19:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (1, 0) 2016-07-12 22:35:13,849 [myid:] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:35:13,849 [myid:] - INFO [/0.0.0.0:11250:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:46885 2016-07-12 22:35:13,849 [myid:] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:35:13,849 [myid:] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@810] - Connection broken for id 0, my id = 1, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:150) at java.net.SocketInputStream.read(SocketInputStream.java:121) at java.net.SocketInputStream.read(SocketInputStream.java:203) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:35:13,850 [myid:] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:35:13,849 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@810] - Connection broken for id 1, my id = 0, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-12 22:35:13,850 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:35:13,850 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:35:13,851 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:35:13,851 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 12332 2016-07-12 22:35:13,851 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 26 2016-07-12 22:35:13,851 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testCnxManager 2016-07-12 22:35:13,851 [myid:] - INFO [main:ZKTestCase$1@60] - SUCCEEDED testCnxManager 2016-07-12 22:35:13,851 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testCnxManager 2016-07-12 22:35:13,862 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testCnxManagerTimeout 2016-07-12 22:35:13,862 [myid:] - INFO [main:PortAssignment@32] - assigning port 11257 2016-07-12 22:35:13,862 [myid:] - INFO [main:PortAssignment@32] - assigning port 11258 2016-07-12 22:35:13,862 [myid:] - INFO [main:PortAssignment@32] - assigning port 11259 2016-07-12 22:35:13,862 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:35:13,862 [myid:] - INFO [main:PortAssignment@32] - assigning port 11260 2016-07-12 22:35:13,862 [myid:] - INFO [main:PortAssignment@32] - assigning port 11261 2016-07-12 22:35:13,862 [myid:] - INFO [main:PortAssignment@32] - assigning port 11262 2016-07-12 22:35:13,862 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:35:13,863 [myid:] - INFO [main:PortAssignment@32] - assigning port 11263 2016-07-12 22:35:13,863 [myid:] - INFO [main:PortAssignment@32] - assigning port 11264 2016-07-12 22:35:13,863 [myid:] - INFO [main:PortAssignment@32] - assigning port 11265 2016-07-12 22:35:13,863 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:35:13,863 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@53] - RUNNING TEST METHOD testCnxManagerTimeout 2016-07-12 22:35:13,863 [myid:] - INFO [main:PortAssignment@32] - assigning port 11266 2016-07-12 22:35:13,863 [myid:] - INFO [main:CnxManagerTest@171] - This is the dead address I'm trying: 192.0.2.179 2016-07-12 22:35:13,863 [myid:] - INFO [main:PortAssignment@32] - assigning port 11267 2016-07-12 22:35:13,863 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 192.0.2.179 to address: /192.0.2.179 2016-07-12 22:35:13,864 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11261 2016-07-12 22:35:13,864 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /0.0.0.0:11262 2016-07-12 22:35:14,000 [myid:] - INFO [SessionTracker:SessionTrackerImpl@162] - SessionTrackerImpl exited loop! 2016-07-12 22:35:14,310 [myid:] - INFO [WorkerSender[myid=2]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down 2016-07-12 22:35:14,311 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down 2016-07-12 22:35:14,829 [myid:] - INFO [/0.0.0.0:11241:QuorumCnxManager$Listener@560] - Leaving listener 2016-07-12 22:35:14,840 [myid:] - INFO [/0.0.0.0:11244:QuorumCnxManager$Listener@560] - Leaving listener 2016-07-12 22:35:14,842 [myid:] - INFO [/0.0.0.0:11247:QuorumCnxManager$Listener@560] - Leaving listener 2016-07-12 22:35:15,319 [myid:] - INFO [WorkerSender[myid=2]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down 2016-07-12 22:35:15,320 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down 2016-07-12 22:35:16,327 [myid:] - INFO [WorkerReceiver[myid=0]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down 2016-07-12 22:35:16,327 [myid:] - INFO [WorkerSender[myid=2]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down 2016-07-12 22:35:16,327 [myid:] - INFO [WorkerSender[myid=0]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down 2016-07-12 22:35:16,328 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down 2016-07-12 22:35:16,328 [myid:] - INFO [WorkerSender[myid=1]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down 2016-07-12 22:35:16,328 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down 2016-07-12 22:35:18,869 [myid:] - WARN [main:QuorumCnxManager@400] - Cannot open channel to 2 at election address /192.0.2.179:11267 java.net.SocketTimeoutException: connect timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354) at org.apache.zookeeper.server.quorum.CnxManagerTest.testCnxManagerTimeout(CnxManagerTest.java:187) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) 2016-07-12 22:35:18,870 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 192.0.2.179 to address: /192.0.2.179 2016-07-12 22:35:18,870 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 12750 2016-07-12 22:35:18,870 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 13 2016-07-12 22:35:18,871 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testCnxManagerTimeout 2016-07-12 22:35:18,871 [myid:] - INFO [main:ZKTestCase$1@60] - SUCCEEDED testCnxManagerTimeout 2016-07-12 22:35:18,871 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testCnxManagerTimeout 2016-07-12 22:35:18,871 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testCnxManagerSpinLock 2016-07-12 22:35:18,872 [myid:] - INFO [main:PortAssignment@32] - assigning port 11268 2016-07-12 22:35:18,872 [myid:] - INFO [main:PortAssignment@32] - assigning port 11269 2016-07-12 22:35:18,872 [myid:] - INFO [main:PortAssignment@32] - assigning port 11270 2016-07-12 22:35:18,872 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:35:18,872 [myid:] - INFO [main:PortAssignment@32] - assigning port 11271 2016-07-12 22:35:18,872 [myid:] - INFO [main:PortAssignment@32] - assigning port 11272 2016-07-12 22:35:18,872 [myid:] - INFO [main:PortAssignment@32] - assigning port 11273 2016-07-12 22:35:18,873 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:35:18,873 [myid:] - INFO [main:PortAssignment@32] - assigning port 11274 2016-07-12 22:35:18,873 [myid:] - INFO [main:PortAssignment@32] - assigning port 11275 2016-07-12 22:35:18,873 [myid:] - INFO [main:PortAssignment@32] - assigning port 11276 2016-07-12 22:35:18,873 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 0.0.0.0 to address: /0.0.0.0 2016-07-12 22:35:18,874 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@53] - RUNNING TEST METHOD testCnxManagerSpinLock 2016-07-12 22:35:18,874 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11272 2016-07-12 22:35:18,875 [myid:] - INFO [main:CnxManagerTest@214] - Election port: 11273 2016-07-12 22:35:18,875 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /0.0.0.0:11273 2016-07-12 22:35:19,876 [myid:] - INFO [/0.0.0.0:11273:QuorumCnxManager$Listener@541] - Received connection request /140.211.11.27:37939 2016-07-12 22:35:19,876 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 1, error = java.io.IOException: Received packet with invalid packet: -20 at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:797) 2016-07-12 22:35:19,877 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-12 22:35:19,877 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-12 22:35:19,877 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-12 22:35:20,876 [myid:] - INFO [main:CnxManagerTest@249] - Socket has been closed as expected 2016-07-12 22:35:20,876 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 12944 2016-07-12 22:35:20,876 [myid:] - ERROR [/0.0.0.0:11273:QuorumCnxManager$Listener@547] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539) 2016-07-12 22:35:20,877 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 14 2016-07-12 22:35:20,877 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testCnxManagerSpinLock 2016-07-12 22:35:20,877 [myid:] - INFO [main:ZKTestCase$1@60] - SUCCEEDED testCnxManagerSpinLock 2016-07-12 22:35:20,877 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testCnxManagerSpinLock {noformat}
    via by Michael Han,
  • From https://builds.apache.org/job/ZooKeeper_branch34_jdk7/1156 {noformat} Error Message waiting for server up Stacktrace junit.framework.AssertionFailedError: waiting for server up at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:183) at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:113) at org.apache.zookeeper.test.QuorumZxidSyncTest.testBehindLeader(QuorumZxidSyncTest.java:67) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) Standard Output 2016-07-21 08:11:45,722 [myid:] - INFO [main:PortAssignment@32] - assigning port 11221 2016-07-21 08:11:45,729 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testBehindLeader 2016-07-21 08:11:45,729 [myid:] - INFO [main:QuorumBase@69] - QuorumBase.setup null 2016-07-21 08:11:45,926 [myid:] - INFO [main:PortAssignment@32] - assigning port 11222 2016-07-21 08:11:45,926 [myid:] - INFO [main:PortAssignment@32] - assigning port 11223 2016-07-21 08:11:45,927 [myid:] - INFO [main:PortAssignment@32] - assigning port 11224 2016-07-21 08:11:45,927 [myid:] - INFO [main:PortAssignment@32] - assigning port 11225 2016-07-21 08:11:45,927 [myid:] - INFO [main:PortAssignment@32] - assigning port 11226 2016-07-21 08:11:45,928 [myid:] - INFO [main:PortAssignment@32] - assigning port 11227 2016-07-21 08:11:45,928 [myid:] - INFO [main:PortAssignment@32] - assigning port 11228 2016-07-21 08:11:45,928 [myid:] - INFO [main:PortAssignment@32] - assigning port 11229 2016-07-21 08:11:45,928 [myid:] - INFO [main:PortAssignment@32] - assigning port 11230 2016-07-21 08:11:45,929 [myid:] - INFO [main:PortAssignment@32] - assigning port 11231 2016-07-21 08:11:45,929 [myid:] - INFO [main:QuorumBase@93] - Ports are: 127.0.0.1:11222,127.0.0.1:11223,127.0.0.1:11224,127.0.0.1:11225,127.0.0.1:11226 2016-07-21 08:11:45,946 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1 2016-07-21 08:11:45,946 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1 2016-07-21 08:11:45,948 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1 2016-07-21 08:11:45,948 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1 2016-07-21 08:11:45,949 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1 2016-07-21 08:11:45,949 [myid:] - INFO [main:QuorumBase@142] - creating QuorumPeer 1 port 11222 2016-07-21 08:11:45,961 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11222 2016-07-21 08:11:45,979 [myid:] - INFO [main:QuorumBase@145] - creating QuorumPeer 2 port 11223 2016-07-21 08:11:45,979 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11223 2016-07-21 08:11:45,980 [myid:] - INFO [main:QuorumBase@148] - creating QuorumPeer 3 port 11224 2016-07-21 08:11:45,980 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11224 2016-07-21 08:11:45,981 [myid:] - INFO [main:QuorumBase@151] - creating QuorumPeer 4 port 11225 2016-07-21 08:11:45,981 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11225 2016-07-21 08:11:45,981 [myid:] - INFO [main:QuorumBase@154] - creating QuorumPeer 5 port 11226 2016-07-21 08:11:45,982 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11226 2016-07-21 08:11:45,982 [myid:] - INFO [main:QuorumBase@163] - QuorumPeer 1 voting view: {1=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@47628981, 2=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@163198c4, 3=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1224773e, 4=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@181090c0, 5=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@678f3997} 2016-07-21 08:11:45,983 [myid:] - INFO [main:QuorumBase@164] - QuorumPeer 2 voting view: {1=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@47628981, 2=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@163198c4, 3=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1224773e, 4=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@181090c0, 5=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@678f3997} 2016-07-21 08:11:45,983 [myid:] - INFO [main:QuorumBase@165] - QuorumPeer 3 voting view: {1=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@47628981, 2=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@163198c4, 3=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1224773e, 4=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@181090c0, 5=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@678f3997} 2016-07-21 08:11:45,983 [myid:] - INFO [main:QuorumBase@166] - QuorumPeer 4 voting view: {1=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@47628981, 2=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@163198c4, 3=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1224773e, 4=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@181090c0, 5=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@678f3997} 2016-07-21 08:11:45,983 [myid:] - INFO [main:QuorumBase@167] - QuorumPeer 5 voting view: {1=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@47628981, 2=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@163198c4, 3=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1224773e, 4=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@181090c0, 5=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@678f3997} 2016-07-21 08:11:45,983 [myid:] - INFO [main:QuorumBase@169] - start QuorumPeer 1 2016-07-21 08:11:45,988 [myid:] - INFO [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-21 08:11:45,990 [myid:] - INFO [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-21 08:11:45,996 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /127.0.0.1:12227 2016-07-21 08:11:46,001 [myid:] - INFO [main:QuorumBase@171] - start QuorumPeer 2 2016-07-21 08:11:46,002 [myid:] - INFO [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-21 08:11:46,003 [myid:] - INFO [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-21 08:11:46,004 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /127.0.0.1:12228 2016-07-21 08:11:46,006 [myid:] - INFO [main:QuorumBase@173] - start QuorumPeer 3 2016-07-21 08:11:46,006 [myid:] - INFO [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-21 08:11:46,007 [myid:] - INFO [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-21 08:11:46,007 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11222:QuorumPeer@774] - LOOKING 2016-07-21 08:11:46,008 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11223:QuorumPeer@774] - LOOKING 2016-07-21 08:11:46,009 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /127.0.0.1:12229 2016-07-21 08:11:46,009 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11222:FastLeaderElection@818] - New election. My id = 1, proposed zxid=0x0 2016-07-21 08:11:46,009 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11223:FastLeaderElection@818] - New election. My id = 2, proposed zxid=0x0 2016-07-21 08:11:46,009 [myid:] - INFO [main:QuorumBase@175] - start QuorumPeer 4 2016-07-21 08:11:46,010 [myid:] - INFO [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-21 08:11:46,011 [myid:] - INFO [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-21 08:11:46,011 [myid:] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11224:QuorumPeer@774] - LOOKING 2016-07-21 08:11:46,011 [myid:] - INFO [/127.0.0.1:12228:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:38924 2016-07-21 08:11:46,011 [myid:] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11224:FastLeaderElection@818] - New election. My id = 3, proposed zxid=0x0 2016-07-21 08:11:46,011 [myid:] - INFO [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43415 2016-07-21 08:11:46,011 [myid:] - INFO [WorkerSender[myid=1]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (2, 1) 2016-07-21 08:11:46,011 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,013 [myid:] - INFO [WorkerSender[myid=1]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (3, 1) 2016-07-21 08:11:46,014 [myid:] - INFO [/127.0.0.1:12229:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:51548 2016-07-21 08:11:46,014 [myid:] - INFO [main:QuorumBase@177] - start QuorumPeer 5 2016-07-21 08:11:46,015 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /127.0.0.1:12230 2016-07-21 08:11:46,015 [myid:] - INFO [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-21 08:11:46,015 [myid:] - WARN [WorkerSender[myid=1]:QuorumCnxManager@400] - Cannot open channel to 4 at election address /127.0.0.1:12230 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433) at java.lang.Thread.run(Thread.java:745) 2016-07-21 08:11:46,018 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,018 [myid:] - INFO [/127.0.0.1:12229:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:51551 2016-07-21 08:11:46,016 [myid:] - INFO [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43417 2016-07-21 08:11:46,016 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:QuorumPeer@774] - LOOKING 2016-07-21 08:11:46,016 [myid:] - INFO [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2016-07-21 08:11:46,019 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:FastLeaderElection@818] - New election. My id = 4, proposed zxid=0x0 2016-07-21 08:11:46,019 [myid:] - INFO [/127.0.0.1:12230:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:46102 2016-07-21 08:11:46,018 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,018 [myid:] - INFO [WorkerSender[myid=2]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (3, 2) 2016-07-21 08:11:46,018 [myid:] - INFO [WorkerSender[myid=1]:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1 2016-07-21 08:11:46,018 [myid:] - INFO [/127.0.0.1:12228:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:38928 2016-07-21 08:11:46,021 [myid:] - INFO [WorkerSender[myid=2]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (4, 2) 2016-07-21 08:11:46,021 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,021 [myid:] - INFO [/127.0.0.1:12229:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:51553 2016-07-21 08:11:46,020 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,019 [myid:] - INFO [WorkerSender[myid=3]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (4, 3) 2016-07-21 08:11:46,022 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,021 [myid:] - WARN [WorkerSender[myid=2]:QuorumCnxManager@400] - Cannot open channel to 5 at election address /127.0.0.1:12231 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433) at java.lang.Thread.run(Thread.java:745) 2016-07-21 08:11:46,021 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,021 [myid:] - WARN [WorkerSender[myid=1]:QuorumCnxManager@400] - Cannot open channel to 5 at election address /127.0.0.1:12231 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433) at java.lang.Thread.run(Thread.java:745) 2016-07-21 08:11:46,023 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,023 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,023 [myid:] - INFO [WorkerSender[myid=2]:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1 2016-07-21 08:11:46,024 [myid:] - INFO [/127.0.0.1:12230:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:46104 2016-07-21 08:11:46,022 [myid:] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /127.0.0.1:12231 2016-07-21 08:11:46,022 [myid:] - WARN [WorkerSender[myid=3]:QuorumCnxManager@400] - Cannot open channel to 5 at election address /127.0.0.1:12231 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433) at java.lang.Thread.run(Thread.java:745) 2016-07-21 08:11:46,025 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,025 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,025 [myid:] - INFO [main:QuorumBase@179] - started QuorumPeer 5 2016-07-21 08:11:46,026 [myid:] - INFO [main:QuorumBase@181] - Checking ports 127.0.0.1:11222,127.0.0.1:11223,127.0.0.1:11224,127.0.0.1:11225,127.0.0.1:11226 2016-07-21 08:11:46,025 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,025 [myid:] - INFO [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43428 2016-07-21 08:11:46,027 [myid:] - INFO [/127.0.0.1:12230:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:46109 2016-07-21 08:11:46,025 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,024 [myid:] - INFO [WorkerSender[myid=1]:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1 2016-07-21 08:11:46,027 [myid:] - INFO [WorkerSender[myid=4]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 4) 2016-07-21 08:11:46,027 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,026 [myid:] - INFO [/127.0.0.1:12228:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:38938 2016-07-21 08:11:46,025 [myid:] - INFO [WorkerSender[myid=3]:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1 2016-07-21 08:11:46,025 [myid:] - INFO [WorkerSender[myid=2]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (4, 2) 2016-07-21 08:11:46,028 [myid:] - INFO [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11222 2016-07-21 08:11:46,028 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 4, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:11:46,029 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:11:46,028 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,027 [myid:] - INFO [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11226:QuorumPeer@774] - LOOKING 2016-07-21 08:11:46,027 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,027 [myid:] - INFO [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52086 2016-07-21 08:11:46,030 [myid:] - WARN [RecvWorker:4:QuorumCnxManager$RecvWorker@810] - Connection broken for id 4, my id = 2, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:11:46,030 [myid:] - WARN [SendWorker:4:QuorumCnxManager$SendWorker@732] - Exception when using channel: for id 4 my id = 2 error = java.net.SocketException: Broken pipe 2016-07-21 08:11:46,029 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,031 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11222:NIOServerCnxnFactory@192] - Accepted socket connection from /127.0.0.1:52363 2016-07-21 08:11:46,029 [myid:] - INFO [/127.0.0.1:12228:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:38940 2016-07-21 08:11:46,029 [myid:] - INFO [WorkerSender[myid=1]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 1) 2016-07-21 08:11:46,029 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,032 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,032 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,029 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@732] - Exception when using channel: for id 2 my id = 4 error = java.net.SocketException: Socket closed 2016-07-21 08:11:46,033 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:11:46,029 [myid:] - INFO [WorkerSender[myid=2]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 2) 2016-07-21 08:11:46,028 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,033 [myid:] - INFO [WorkerSender[myid=1]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 1) 2016-07-21 08:11:46,032 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,031 [myid:] - INFO [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52088 2016-07-21 08:11:46,031 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,031 [myid:] - INFO [/127.0.0.1:12230:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:46117 2016-07-21 08:11:46,030 [myid:] - WARN [SendWorker:4:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:11:46,030 [myid:] - WARN [RecvWorker:4:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:11:46,030 [myid:] - INFO [WorkerSender[myid=3]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 3) 2016-07-21 08:11:46,030 [myid:] - INFO [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11226:FastLeaderElection@818] - New election. My id = 5, proposed zxid=0x0 2016-07-21 08:11:46,030 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,034 [myid:] - INFO [/127.0.0.1:12228:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:38947 2016-07-21 08:11:46,034 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,034 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,033 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,035 [myid:] - INFO [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52089 2016-07-21 08:11:46,035 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,036 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,036 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,037 [myid:] - INFO [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43442 2016-07-21 08:11:46,036 [myid:] - INFO [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,036 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,037 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,037 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,038 [myid:] - INFO [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52091 2016-07-21 08:11:46,037 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,036 [myid:] - INFO [WorkerSender[myid=1]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 1) 2016-07-21 08:11:46,038 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,038 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,038 [myid:] - INFO [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,038 [myid:] - INFO [/127.0.0.1:12229:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:51573 2016-07-21 08:11:46,038 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,037 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,039 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,039 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,039 [myid:] - INFO [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,040 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,039 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,039 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,039 [myid:] - INFO [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52093 2016-07-21 08:11:46,040 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,040 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,040 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,040 [myid:] - INFO [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,041 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,041 [myid:] - INFO [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43444 2016-07-21 08:11:46,041 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,041 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@810] - Connection broken for id 1, my id = 5, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.DataInputStream.readFully(DataInputStream.java:195) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:805) 2016-07-21 08:11:46,042 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:11:46,041 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,041 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:11:46,041 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11222:NIOServerCnxn@827] - Processing stat command from /127.0.0.1:52363 2016-07-21 08:11:46,041 [myid:] - WARN [RecvWorker:5:QuorumCnxManager$RecvWorker@810] - Connection broken for id 5, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:11:46,043 [myid:] - WARN [RecvWorker:5:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:11:46,041 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,041 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,043 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,043 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:11:46,042 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,042 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:11:46,044 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:11:46,042 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,042 [myid:] - INFO [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52096 2016-07-21 08:11:46,041 [myid:] - INFO [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,045 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:11:46,045 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:11:46,045 [myid:] - INFO [Thread-2:NIOServerCnxn@1008] - Closed socket connection for client /127.0.0.1:52363 (no session established for client) 2016-07-21 08:11:46,044 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,044 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,044 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,046 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,045 [myid:] - INFO [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43446 2016-07-21 08:11:46,045 [myid:] - INFO [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,045 [myid:] - WARN [RecvWorker:5:QuorumCnxManager$RecvWorker@810] - Connection broken for id 5, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:11:46,047 [myid:] - WARN [RecvWorker:5:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:11:46,045 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@810] - Connection broken for id 1, my id = 5, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:11:46,047 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:11:46,047 [myid:] - INFO [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,047 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:11:46,048 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:11:46,046 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,048 [myid:] - INFO [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,048 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,049 [myid:] - INFO [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,049 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,049 [myid:] - INFO [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2016-07-21 08:11:46,245 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:QuorumPeer@844] - FOLLOWING 2016-07-21 08:11:46,246 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11223:QuorumPeer@844] - FOLLOWING 2016-07-21 08:11:46,247 [myid:] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11224:QuorumPeer@844] - FOLLOWING 2016-07-21 08:11:46,249 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11222:QuorumPeer@844] - FOLLOWING 2016-07-21 08:11:46,249 [myid:] - INFO [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11226:QuorumPeer@856] - LEADING 2016-07-21 08:11:46,251 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Learner@86] - TCP NoDelay set to: true 2016-07-21 08:11:46,255 [myid:] - INFO [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11226:Leader@59] - TCP NoDelay set to: true 2016-07-21 08:11:46,259 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:zookeeper.version=3.4.9-SNAPSHOT-1753645, built on 07/21/2016 07:46 GMT 2016-07-21 08:11:46,259 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:host.name=asf907.gq1.ygridcore.net 2016-07-21 08:11:46,260 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.version=1.7.0_80 2016-07-21 08:11:46,260 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.vendor=Oracle Corporation 2016-07-21 08:11:46,260 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.home=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre 2016-07-21 08:11:46,260 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.class.path=/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/classes:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/antlr-2.7.6.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/checkstyle-5.0.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/commons-beanutils-core-1.7.0.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/commons-cli-1.0.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/commons-collections-3.2.2.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/commons-lang-1.0.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/commons-logging-1.0.3.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/google-collections-0.9.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/junit-4.8.1.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/mockito-all-1.8.2.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/classes:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/src/java/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/latest/lib/ant.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/lib/jline-0.9.94.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/lib/log4j-1.2.16.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/lib/netty-3.10.5.Final.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/lib/slf4j-api-1.6.1.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/lib/slf4j-log4j12-1.6.1.jar:/home/jenkins/tools/ant/apache-ant-1.9.4/lib/ant-launcher.jar:/home/jenkins/tools/ant/latest/lib/ant-junit.jar:/home/jenkins/tools/ant/latest/lib/ant-junit4.jar 2016-07-21 08:11:46,260 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2016-07-21 08:11:46,260 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.io.tmpdir=/tmp 2016-07-21 08:11:46,260 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.compiler=<NA> 2016-07-21 08:11:46,260 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:os.name=Linux 2016-07-21 08:11:46,260 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:os.arch=amd64 2016-07-21 08:11:46,260 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:os.version=3.13.0-36-lowlatency 2016-07-21 08:11:46,260 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0: ...[truncated 1420555 chars]... ager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:21,678 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:21,672 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@810] - Connection broken for id 1, my id = 5, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:21,679 [myid:] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:21,672 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:21,679 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:21,679 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:21,679 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:21,678 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:21,679 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:21,677 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:21,680 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:21,675 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:21,680 [myid:] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:22,000 [myid:] - INFO [SessionTracker:SessionTrackerImpl@162] - SessionTrackerImpl exited loop! 2016-07-21 08:14:22,317 [myid:] - INFO [/127.0.0.1:12241:QuorumCnxManager$Listener@560] - Leaving listener 2016-07-21 08:14:22,320 [myid:] - INFO [/127.0.0.1:12242:QuorumCnxManager$Listener@560] - Leaving listener 2016-07-21 08:14:22,579 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11233:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:850) 2016-07-21 08:14:22,579 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11233:FollowerZooKeeperServer@140] - Shutting down 2016-07-21 08:14:22,579 [myid:] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11233:SyncRequestProcessor@209] - Shutting down 2016-07-21 08:14:22,579 [myid:] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11233:QuorumPeer@874] - QuorumPeer main thread exited 2016-07-21 08:14:22,582 [myid:] - INFO [main:QuorumBase@306] - Shutting down quorum peer QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234 2016-07-21 08:14:22,582 [myid:] - INFO [main:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:891) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:307) at org.apache.zookeeper.test.QuorumBase.shutdownServers(QuorumBase.java:298) at org.apache.zookeeper.test.QuorumBase.tearDown(QuorumBase.java:285) at org.apache.zookeeper.test.QuorumZxidSyncTest.tearDown(QuorumZxidSyncTest.java:169) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:532) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1179) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1030) 2016-07-21 08:14:22,582 [myid:] - INFO [main:FollowerZooKeeperServer@140] - Shutting down 2016-07-21 08:14:22,582 [myid:] - INFO [main:ZooKeeperServer@469] - shutting down 2016-07-21 08:14:22,582 [myid:] - INFO [main:FollowerRequestProcessor@107] - Shutting down 2016-07-21 08:14:22,582 [myid:] - INFO [main:CommitProcessor@184] - Shutting down 2016-07-21 08:14:22,582 [myid:] - INFO [FollowerRequestProcessor:2:FollowerRequestProcessor@97] - FollowerRequestProcessor exited loop! 2016-07-21 08:14:22,582 [myid:] - INFO [CommitProcessor:2:CommitProcessor@153] - CommitProcessor exited loop! 2016-07-21 08:14:22,582 [myid:] - INFO [main:FinalRequestProcessor@402] - shutdown of request processor complete 2016-07-21 08:14:22,583 [myid:] - INFO [main:SyncRequestProcessor@209] - Shutting down 2016-07-21 08:14:22,583 [myid:] - INFO [SyncThread:2:SyncRequestProcessor@187] - SyncRequestProcessor exited! 2016-07-21 08:14:22,584 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11234:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method 2016-07-21 08:14:22,585 [myid:] - WARN [SendWorker:3:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:22,585 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 4, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:22,585 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:22,585 [myid:] - WARN [RecvWorker:3:QuorumCnxManager$RecvWorker@810] - Connection broken for id 3, my id = 2, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:22,586 [myid:] - WARN [RecvWorker:3:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:22,585 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 5, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:22,586 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:22,585 [myid:] - WARN [RecvWorker:5:QuorumCnxManager$RecvWorker@810] - Connection broken for id 5, my id = 2, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:22,587 [myid:] - WARN [RecvWorker:5:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:22,585 [myid:] - WARN [SendWorker:3:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:22,585 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 3, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:22,587 [myid:] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:22,585 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:22,587 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:22,585 [myid:] - ERROR [/127.0.0.1:12239:QuorumCnxManager$Listener@547] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539) 2016-07-21 08:14:22,587 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:22,588 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:22,587 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:22,589 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:22,586 [myid:] - WARN [SendWorker:4:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:22,589 [myid:] - WARN [SendWorker:4:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:22,586 [myid:] - WARN [RecvWorker:4:QuorumCnxManager$RecvWorker@810] - Connection broken for id 4, my id = 2, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:22,589 [myid:] - WARN [RecvWorker:4:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:22,586 [myid:] - INFO [main:QuorumBase@310] - Shutting down leader election QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234 2016-07-21 08:14:22,589 [myid:] - INFO [main:QuorumBase@315] - Waiting for QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234 to exit thread 2016-07-21 08:14:22,585 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:22,590 [myid:] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:22,673 [myid:] - INFO [/127.0.0.1:12238:QuorumCnxManager$Listener@560] - Leaving listener 2016-07-21 08:14:23,079 [myid:] - INFO [WorkerSender[myid=3]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down 2016-07-21 08:14:23,080 [myid:] - INFO [WorkerSender[myid=4]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down 2016-07-21 08:14:23,084 [myid:] - INFO [WorkerSender[myid=5]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down 2016-07-21 08:14:23,088 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down 2016-07-21 08:14:23,088 [myid:] - INFO [WorkerReceiver[myid=4]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down 2016-07-21 08:14:23,089 [myid:] - INFO [WorkerReceiver[myid=5]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down 2016-07-21 08:14:23,578 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:850) 2016-07-21 08:14:23,578 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234:FollowerZooKeeperServer@140] - Shutting down 2016-07-21 08:14:23,578 [myid:] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234:SyncRequestProcessor@209] - Shutting down 2016-07-21 08:14:23,578 [myid:] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234:QuorumPeer@874] - QuorumPeer main thread exited 2016-07-21 08:14:23,579 [myid:] - INFO [main:QuorumBase@306] - Shutting down quorum peer QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235 2016-07-21 08:14:23,579 [myid:] - INFO [main:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:891) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:307) at org.apache.zookeeper.test.QuorumBase.shutdownServers(QuorumBase.java:299) at org.apache.zookeeper.test.QuorumBase.tearDown(QuorumBase.java:285) at org.apache.zookeeper.test.QuorumZxidSyncTest.tearDown(QuorumZxidSyncTest.java:169) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:532) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1179) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1030) 2016-07-21 08:14:23,579 [myid:] - INFO [main:FollowerZooKeeperServer@140] - Shutting down 2016-07-21 08:14:23,579 [myid:] - INFO [main:ZooKeeperServer@469] - shutting down 2016-07-21 08:14:23,580 [myid:] - INFO [main:FollowerRequestProcessor@107] - Shutting down 2016-07-21 08:14:23,580 [myid:] - INFO [main:CommitProcessor@184] - Shutting down 2016-07-21 08:14:23,580 [myid:] - INFO [FollowerRequestProcessor:3:FollowerRequestProcessor@97] - FollowerRequestProcessor exited loop! 2016-07-21 08:14:23,580 [myid:] - INFO [CommitProcessor:3:CommitProcessor@153] - CommitProcessor exited loop! 2016-07-21 08:14:23,580 [myid:] - INFO [main:FinalRequestProcessor@402] - shutdown of request processor complete 2016-07-21 08:14:23,581 [myid:] - INFO [main:SyncRequestProcessor@209] - Shutting down 2016-07-21 08:14:23,581 [myid:] - INFO [SyncThread:3:SyncRequestProcessor@187] - SyncRequestProcessor exited! 2016-07-21 08:14:23,582 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11235:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method 2016-07-21 08:14:23,583 [myid:] - ERROR [/127.0.0.1:12240:QuorumCnxManager$Listener@547] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539) 2016-07-21 08:14:23,583 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:23,585 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:23,584 [myid:] - WARN [RecvWorker:4:QuorumCnxManager$RecvWorker@810] - Connection broken for id 4, my id = 3, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:23,585 [myid:] - WARN [RecvWorker:4:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:23,584 [myid:] - WARN [SendWorker:4:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:23,585 [myid:] - WARN [SendWorker:4:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:23,584 [myid:] - WARN [RecvWorker:5:QuorumCnxManager$RecvWorker@810] - Connection broken for id 5, my id = 3, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:23,585 [myid:] - WARN [RecvWorker:5:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:23,584 [myid:] - WARN [RecvWorker:3:QuorumCnxManager$RecvWorker@810] - Connection broken for id 3, my id = 5, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:23,586 [myid:] - WARN [RecvWorker:3:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:23,584 [myid:] - WARN [RecvWorker:3:QuorumCnxManager$RecvWorker@810] - Connection broken for id 3, my id = 4, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:23,586 [myid:] - WARN [RecvWorker:3:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:23,584 [myid:] - INFO [main:QuorumBase@310] - Shutting down leader election QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235 2016-07-21 08:14:23,586 [myid:] - INFO [main:QuorumBase@315] - Waiting for QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235 to exit thread 2016-07-21 08:14:23,586 [myid:] - WARN [SendWorker:3:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:23,586 [myid:] - WARN [SendWorker:3:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:23,586 [myid:] - WARN [SendWorker:3:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:23,587 [myid:] - WARN [SendWorker:3:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:23,588 [myid:] - INFO [/127.0.0.1:12239:QuorumCnxManager$Listener@560] - Leaving listener 2016-07-21 08:14:24,355 [myid:] - INFO [WorkerSender[myid=2]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down 2016-07-21 08:14:24,358 [myid:] - INFO [WorkerSender[myid=1]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down 2016-07-21 08:14:24,360 [myid:] - INFO [WorkerSender[myid=3]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down 2016-07-21 08:14:24,361 [myid:] - INFO [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down 2016-07-21 08:14:24,361 [myid:] - INFO [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down 2016-07-21 08:14:24,362 [myid:] - INFO [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down 2016-07-21 08:14:24,578 [myid:] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:850) 2016-07-21 08:14:24,578 [myid:] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235:FollowerZooKeeperServer@140] - Shutting down 2016-07-21 08:14:24,578 [myid:] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235:SyncRequestProcessor@209] - Shutting down 2016-07-21 08:14:24,578 [myid:] - WARN [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235:QuorumPeer@874] - QuorumPeer main thread exited 2016-07-21 08:14:24,579 [myid:] - INFO [main:QuorumBase@306] - Shutting down quorum peer QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236 2016-07-21 08:14:24,579 [myid:] - INFO [main:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:891) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:307) at org.apache.zookeeper.test.QuorumBase.shutdownServers(QuorumBase.java:300) at org.apache.zookeeper.test.QuorumBase.tearDown(QuorumBase.java:285) at org.apache.zookeeper.test.QuorumZxidSyncTest.tearDown(QuorumZxidSyncTest.java:169) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:532) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1179) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1030) 2016-07-21 08:14:24,580 [myid:] - INFO [main:FollowerZooKeeperServer@140] - Shutting down 2016-07-21 08:14:24,581 [myid:] - INFO [main:ZooKeeperServer@469] - shutting down 2016-07-21 08:14:24,581 [myid:] - INFO [main:FollowerRequestProcessor@107] - Shutting down 2016-07-21 08:14:24,581 [myid:] - INFO [main:CommitProcessor@184] - Shutting down 2016-07-21 08:14:24,581 [myid:] - INFO [main:FinalRequestProcessor@402] - shutdown of request processor complete 2016-07-21 08:14:24,581 [myid:] - INFO [FollowerRequestProcessor:4:FollowerRequestProcessor@97] - FollowerRequestProcessor exited loop! 2016-07-21 08:14:24,581 [myid:] - INFO [main:SyncRequestProcessor@209] - Shutting down 2016-07-21 08:14:24,581 [myid:] - INFO [CommitProcessor:4:CommitProcessor@153] - CommitProcessor exited loop! 2016-07-21 08:14:24,582 [myid:] - INFO [SyncThread:4:SyncRequestProcessor@187] - SyncRequestProcessor exited! 2016-07-21 08:14:24,583 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method 2016-07-21 08:14:24,585 [myid:] - INFO [main:QuorumBase@310] - Shutting down leader election QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236 2016-07-21 08:14:24,586 [myid:] - ERROR [/127.0.0.1:12241:QuorumCnxManager$Listener@547] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539) 2016-07-21 08:14:24,586 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:24,586 [myid:] - WARN [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:24,586 [myid:] - INFO [main:QuorumBase@315] - Waiting for QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236 to exit thread 2016-07-21 08:14:24,586 [myid:] - WARN [RecvWorker:4:QuorumCnxManager$RecvWorker@810] - Connection broken for id 4, my id = 5, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:24,587 [myid:] - WARN [RecvWorker:4:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:24,585 [myid:] - INFO [/127.0.0.1:12240:QuorumCnxManager$Listener@560] - Leaving listener 2016-07-21 08:14:24,585 [myid:] - WARN [RecvWorker:5:QuorumCnxManager$RecvWorker@810] - Connection broken for id 5, my id = 4, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795) 2016-07-21 08:14:24,588 [myid:] - WARN [RecvWorker:5:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker 2016-07-21 08:14:24,588 [myid:] - WARN [SendWorker:4:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715) 2016-07-21 08:14:24,588 [myid:] - WARN [SendWorker:4:QuorumCnxManager$SendWorker@736] - Send worker leaving thread 2016-07-21 08:14:25,579 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236:Follower@166] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:850) 2016-07-21 08:14:25,579 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236:FollowerZooKeeperServer@140] - Shutting down 2016-07-21 08:14:25,579 [myid:] - INFO [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236:SyncRequestProcessor@209] - Shutting down 2016-07-21 08:14:25,579 [myid:] - WARN [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236:QuorumPeer@874] - QuorumPeer main thread exited 2016-07-21 08:14:25,579 [myid:] - INFO [main:QuorumBase@306] - Shutting down quorum peer QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237 2016-07-21 08:14:25,579 [myid:] - INFO [main:Leader@496] - Shutting down 2016-07-21 08:14:25,580 [myid:] - INFO [main:Leader@502] - Shutdown called java.lang.Exception: shutdown Leader! reason: quorum Peer shutdown at org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:502) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:888) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:307) at org.apache.zookeeper.test.QuorumBase.shutdownServers(QuorumBase.java:301) at org.apache.zookeeper.test.QuorumBase.tearDown(QuorumBase.java:285) at org.apache.zookeeper.test.QuorumZxidSyncTest.tearDown(QuorumZxidSyncTest.java:169) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:532) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1179) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1030) 2016-07-21 08:14:25,581 [myid:] - INFO [main:ZooKeeperServer@469] - shutting down 2016-07-21 08:14:25,581 [myid:] - INFO [LearnerCnxAcceptor-/127.0.0.1:12237:Leader$LearnerCnxAcceptor@325] - exception while shutting down acceptor: java.net.SocketException: Socket closed 2016-07-21 08:14:25,582 [myid:] - INFO [main:SessionTrackerImpl@225] - Shutting down 2016-07-21 08:14:25,582 [myid:] - INFO [main:PrepRequestProcessor@765] - Shutting down 2016-07-21 08:14:25,582 [myid:] - INFO [main:ProposalRequestProcessor@88] - Shutting down 2016-07-21 08:14:25,582 [myid:] - INFO [main:CommitProcessor@184] - Shutting down 2016-07-21 08:14:25,582 [myid:] - INFO [ProcessThread(sid:5 cport:-1)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop! 2016-07-21 08:14:25,583 [myid:] - INFO [CommitProcessor:5:CommitProcessor@153] - CommitProcessor exited loop! 2016-07-21 08:14:25,582 [myid:] - INFO [main:Leader$ToBeAppliedRequestProcessor@661] - Shutting down 2016-07-21 08:14:25,583 [myid:] - INFO [main:FinalRequestProcessor@402] - shutdown of request processor complete 2016-07-21 08:14:25,583 [myid:] - INFO [main:SyncRequestProcessor@209] - Shutting down 2016-07-21 08:14:25,583 [myid:] - INFO [SyncThread:5:SyncRequestProcessor@187] - SyncRequestProcessor exited! 2016-07-21 08:14:25,585 [myid:] - WARN [LearnerHandler-/127.0.0.1:43485:LearnerHandler@644] - ******* GOODBYE /127.0.0.1:43485 ******** 2016-07-21 08:14:25,585 [myid:] - WARN [LearnerHandler-/127.0.0.1:43486:LearnerHandler@644] - ******* GOODBYE /127.0.0.1:43486 ******** 2016-07-21 08:14:25,585 [myid:] - WARN [LearnerHandler-/127.0.0.1:43486:LearnerHandler@656] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647) 2016-07-21 08:14:25,585 [myid:] - WARN [LearnerHandler-/127.0.0.1:43485:LearnerHandler@656] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647) 2016-07-21 08:14:25,585 [myid:] - WARN [LearnerHandler-/127.0.0.1:43488:LearnerHandler@644] - ******* GOODBYE /127.0.0.1:43488 ******** 2016-07-21 08:14:25,586 [myid:] - INFO [/127.0.0.1:12241:QuorumCnxManager$Listener@560] - Leaving listener 2016-07-21 08:14:25,586 [myid:] - WARN [LearnerHandler-/127.0.0.1:43487:LearnerHandler@644] - ******* GOODBYE /127.0.0.1:43487 ******** 2016-07-21 08:14:25,586 [myid:] - WARN [LearnerHandler-/127.0.0.1:43487:LearnerHandler@656] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647) 2016-07-21 08:14:25,586 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11237:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method 2016-07-21 08:14:25,586 [myid:] - WARN [LearnerHandler-/127.0.0.1:43488:LearnerHandler@656] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647) 2016-07-21 08:14:25,587 [myid:] - WARN [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237:QuorumPeer@862] - Unexpected exception java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:456) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:859) 2016-07-21 08:14:25,588 [myid:] - INFO [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237:Leader@496] - Shutting down 2016-07-21 08:14:25,588 [myid:] - INFO [main:QuorumBase@310] - Shutting down leader election QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237 2016-07-21 08:14:25,588 [myid:] - ERROR [/127.0.0.1:12242:QuorumCnxManager$Listener@547] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539) 2016-07-21 08:14:25,588 [myid:] - INFO [main:QuorumBase@315] - Waiting for QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237 to exit thread 2016-07-21 08:14:25,588 [myid:] - WARN [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237:QuorumPeer@874] - QuorumPeer main thread exited 2016-07-21 08:14:25,589 [myid:] - INFO [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11233 2016-07-21 08:14:25,589 [myid:] - INFO [main:QuorumBase@291] - 127.0.0.1:11233 is no longer accepting client connections 2016-07-21 08:14:25,589 [myid:] - INFO [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11234 2016-07-21 08:14:25,589 [myid:] - INFO [main:QuorumBase@291] - 127.0.0.1:11234 is no longer accepting client connections 2016-07-21 08:14:25,590 [myid:] - INFO [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11235 2016-07-21 08:14:25,590 [myid:] - INFO [main:QuorumBase@291] - 127.0.0.1:11235 is no longer accepting client connections 2016-07-21 08:14:25,590 [myid:] - INFO [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11236 2016-07-21 08:14:25,590 [myid:] - INFO [main:QuorumBase@291] - 127.0.0.1:11236 is no longer accepting client connections 2016-07-21 08:14:25,590 [myid:] - INFO [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11237 2016-07-21 08:14:25,590 [myid:] - INFO [main:QuorumBase@291] - 127.0.0.1:11237 is no longer accepting client connections 2016-07-21 08:14:25,592 [myid:] - INFO [main:ZKTestCase$1@60] - SUCCEEDED testLateLogs 2016-07-21 08:14:25,592 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testLateLogs {noformat}
    via by Michael Han,
  • This flaky test was introduced as part of ZOOKEEPER-1045, which fails regularly both in our internal test bot and upstream apache build bot. {noformat} Error Message waiting for server1being up Stacktrace junit.framework.AssertionFailedError: waiting for server1being up at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.restartServer(QuorumAuthUpgradeTest.java:232) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.testRollingUpgrade(QuorumAuthUpgradeTest.java:204) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52) Standard Output 2017-01-24 22:12:19,965 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testAuthLearnerAgainstNoAuthRequiredServer 2017-01-24 22:12:19,971 [myid:] - INFO [Thread-0:JUnit4ZKTestRunner$LoggedInvokeMethod@50] - RUNNING TEST METHOD testAuthLearnerAgainstNoAuthRequiredServer 2017-01-24 22:12:19,972 [myid:] - INFO [Thread-0:PortAssignment@32] - assigning port 11221 2017-01-24 22:12:19,973 [myid:] - INFO [Thread-0:PortAssignment@32] - assigning port 11222 2017-01-24 22:12:19,973 [myid:] - INFO [Thread-0:PortAssignment@32] - assigning port 11223 2017-01-24 22:12:19,973 [myid:] - INFO [Thread-0:PortAssignment@32] - assigning port 11224 2017-01-24 22:12:19,973 [myid:] - INFO [Thread-0:PortAssignment@32] - assigning port 11225 2017-01-24 22:12:19,973 [myid:] - INFO [Thread-0:PortAssignment@32] - assigning port 11226 2017-01-24 22:12:19,974 [myid:] - INFO [Thread-0:QuorumPeerTestBase$MainThread@81] - id = 0 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8717551953335097579.junit.dir clientPort = 11221 2017-01-24 22:12:19,981 [myid:] - INFO [Thread-0:QuorumPeerTestBase$MainThread@81] - id = 1 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4158500655544466289.junit.dir clientPort = 11224 2017-01-24 22:12:20,048 [myid:] - INFO [Thread-0:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 2017-01-24 22:12:20,048 [myid:] - INFO [Thread-2:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4158500655544466289.junit.dir/zoo.cfg 2017-01-24 22:12:20,050 [myid:] - INFO [Thread-1:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8717551953335097579.junit.dir/zoo.cfg 2017-01-24 22:12:20,054 [myid:] - WARN [Thread-2:QuorumPeerConfig@327] - No server failure will be tolerated. You need at least 3 servers. 2017-01-24 22:12:20,056 [myid:] - INFO [Thread-2:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:12:20,059 [myid:] - WARN [Thread-1:QuorumPeerConfig@327] - No server failure will be tolerated. You need at least 3 servers. 2017-01-24 22:12:20,064 [myid:] - INFO [Thread-0:ClientBase@246] - server 127.0.0.1:11221 not up java.net.ConnectException: Connection refused 2017-01-24 22:12:20,070 [myid:] - INFO [Thread-1:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:12:20,071 [myid:1] - INFO [Thread-2:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:12:20,071 [myid:1] - INFO [Thread-2:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:12:20,071 [myid:1] - INFO [Thread-2:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:12:20,075 [myid:0] - INFO [Thread-1:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:12:20,075 [myid:0] - INFO [Thread-1:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:12:20,075 [myid:0] - INFO [Thread-1:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:12:20,210 [myid:1] - WARN [Thread-2:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:20,213 [myid:1] - INFO [Thread-2:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:12:20,222 [myid:0] - INFO [Thread-1:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:12:20,237 [myid:0] - INFO [Thread-1:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11221 2017-01-24 22:12:20,241 [myid:1] - INFO [Thread-2:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11224 2017-01-24 22:12:20,275 [myid:0] - INFO [Thread-1:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:12:20,275 [myid:1] - INFO [Thread-2:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:12:20,275 [myid:0] - INFO [Thread-1:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:12:20,275 [myid:1] - INFO [Thread-2:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:12:20,277 [myid:0] - INFO [Thread-1:QuorumPeer@1279] - quorum.auth.enableSasl set to true 2017-01-24 22:12:20,277 [myid:1] - INFO [Thread-2:QuorumPeer@1279] - quorum.auth.enableSasl set to true 2017-01-24 22:12:20,277 [myid:0] - INFO [Thread-1:QuorumPeer@1264] - quorum.auth.serverRequireSasl set to false 2017-01-24 22:12:20,277 [myid:1] - INFO [Thread-2:QuorumPeer@1264] - quorum.auth.serverRequireSasl set to false 2017-01-24 22:12:20,277 [myid:0] - INFO [Thread-1:QuorumPeer@1270] - quorum.auth.learnerRequireSasl set to false 2017-01-24 22:12:20,278 [myid:1] - INFO [Thread-2:QuorumPeer@1270] - quorum.auth.learnerRequireSasl set to false 2017-01-24 22:12:20,278 [myid:0] - INFO [Thread-1:QuorumPeer@1286] - quorum.auth.kerberos.servicePrincipal set to zkquorum/localhost 2017-01-24 22:12:20,278 [myid:1] - INFO [Thread-2:QuorumPeer@1286] - quorum.auth.kerberos.servicePrincipal set to zkquorum/localhost 2017-01-24 22:12:20,278 [myid:0] - INFO [Thread-1:QuorumPeer@1298] - quorum.auth.server.saslLoginContext set to QuorumServer 2017-01-24 22:12:20,278 [myid:1] - INFO [Thread-2:QuorumPeer@1298] - quorum.auth.server.saslLoginContext set to QuorumServer 2017-01-24 22:12:20,278 [myid:0] - INFO [Thread-1:QuorumPeer@1292] - quorum.auth.learner.saslLoginContext set to QuorumLearner 2017-01-24 22:12:20,278 [myid:1] - INFO [Thread-2:QuorumPeer@1292] - quorum.auth.learner.saslLoginContext set to QuorumLearner 2017-01-24 22:12:20,278 [myid:0] - INFO [Thread-1:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:12:20,279 [myid:1] - INFO [Thread-2:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:12:20,288 [myid:0] - INFO [Thread-1:Login@294] - QuorumServer successfully logged in. 2017-01-24 22:12:20,288 [myid:1] - INFO [Thread-2:Login@294] - QuorumServer successfully logged in. 2017-01-24 22:12:20,292 [myid:0] - INFO [Thread-1:Login@294] - QuorumLearner successfully logged in. 2017-01-24 22:12:20,292 [myid:1] - INFO [Thread-2:Login@294] - QuorumLearner successfully logged in. 2017-01-24 22:12:20,299 [myid:0] - INFO [Thread-1:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:20,299 [myid:1] - INFO [Thread-2:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:20,302 [myid:0] - INFO [Thread-1:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:20,302 [myid:1] - INFO [Thread-2:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:20,312 [myid:1] - INFO [Thread-4:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11226 2017-01-24 22:12:20,313 [myid:0] - INFO [Thread-5:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11223 2017-01-24 22:12:20,320 [myid:] - INFO [Thread-0:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 2017-01-24 22:12:20,322 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:58467 2017-01-24 22:12:20,338 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:58467 2017-01-24 22:12:20,349 [myid:0] - INFO [Thread-6:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:58467 (no session established for client) 2017-01-24 22:12:20,345 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:QuorumPeer@781] - LOOKING 2017-01-24 22:12:20,345 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:QuorumPeer@781] - LOOKING 2017-01-24 22:12:20,352 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FastLeaderElection@744] - New election. My id = 1, proposed zxid=0x0 2017-01-24 22:12:20,354 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:FastLeaderElection@744] - New election. My id = 0, proposed zxid=0x0 2017-01-24 22:12:20,357 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:20,357 [myid:1] - INFO [localhost/127.0.0.1:11226:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:48700 2017-01-24 22:12:20,356 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-1:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:20,354 [myid:0] - INFO [localhost/127.0.0.1:11223:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:51927 2017-01-24 22:12:20,358 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:20,357 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:20,359 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:QuorumCnxManager@331] - Have smaller server identifier, so dropping the connection: (1, 0) 2017-01-24 22:12:20,363 [myid:1] - WARN [QuorumConnectionThread-[myid=1]-2:SaslQuorumAuthServer@135] - Failed to authenticate using SASL java.io.EOFException at java.io.DataInputStream.readFully(DataInputStream.java:197) at java.io.DataInputStream.readLong(DataInputStream.java:416) at org.apache.jute.BinaryInputArchive.readLong(BinaryInputArchive.java:67) at org.apache.zookeeper.server.quorum.auth.QuorumAuth.nextPacketIsAuth(QuorumAuth.java:91) at org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthServer.authenticate(SaslQuorumAuthServer.java:79) at org.apache.zookeeper.server.quorum.QuorumCnxManager.handleConnection(QuorumCnxManager.java:435) at org.apache.zookeeper.server.quorum.QuorumCnxManager.receiveConnection(QuorumCnxManager.java:373) at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReceiverThread.run(QuorumCnxManager.java:409) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:20,363 [myid:1] - WARN [QuorumConnectionThread-[myid=1]-2:SaslQuorumAuthServer@136] - Maintaining learner connection despite SASL authentication failure. server addr: /127.0.0.1:48700, quorum.auth.serverRequireSasl: false 2017-01-24 22:12:20,364 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:20,364 [myid:0] - INFO [localhost/127.0.0.1:11223:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:51929 2017-01-24 22:12:20,364 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:20,364 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:20,365 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@901] - Exception when using channel: for id 1 my id = 0 error = java.net.SocketException: Broken pipe 2017-01-24 22:12:20,365 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:20,366 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:20,366 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-3:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:20,378 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:20,379 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:20,380 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:20,380 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:20,581 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:20,581 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:QuorumPeer@861] - LEADING 2017-01-24 22:12:20,585 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:Leader@62] - TCP NoDelay set to: true 2017-01-24 22:12:20,588 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Learner@85] - TCP NoDelay set to: true 2017-01-24 22:12:20,593 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:zookeeper.version=3.4.5--1, built on 01/25/2017 06:01 GMT 2017-01-24 22:12:20,593 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:host.name=ec2-unittest-ub12-m3-large-0e11.vpc.cloudera.com 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:java.version=1.7.0_75 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:java.vendor=Oracle Corporation 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:java.home=/data/jenkins/tools/hudson.model.JDK/Java_7/jre 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:java.class.path=/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/classes:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/antlr-2.7.6.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/checkstyle-5.0.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-beanutils-core-1.7.0.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-cli-1.0.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-collections-3.2.2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-io-2.4.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-lang-1.0.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-logging-1.0.3.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/google-collections-0.9.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/junit-4.8.1.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-admin-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-client-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-common-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-core-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-crypto-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-identity-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-server-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-simplekdc-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-util-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerby-asn1-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerby-config-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerby-pkix-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerby-util-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/log4j-1.2.17.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/mockito-all-1.8.2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/slf4j-api-1.7.14.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/slf4j-log4j12-1.7.14.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/classes:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/src/java/lib/ivy-2.2.0.jar:/data/jenkins/tools/hudson.tasks.Ant_AntInstallation/Ant_1.8.2/apache-ant-1.9.6/lib/ant.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/lib/jline-2.11.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/lib/log4j-1.2.16.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/lib/netty-3.10.5.Final.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/lib/slf4j-api-1.7.5.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/lib/slf4j-log4j12-1.7.5.jar:/mnt/toolchain/clover-ant-4.0.3/lib/clover.jar:/data/jenkins/tools/hudson.tasks.Ant_AntInstallation/Ant_1.8.2/apache-ant-1.9.6/lib/ant-launcher.jar:/data/jenkins/tools/hudson.tasks.Ant_AntInstallation/Ant_1.8.2/apache-ant-1.9.6/lib/ant-junit.jar:/data/jenkins/tools/hudson.tasks.Ant_AntInstallation/Ant_1.8.2/apache-ant-1.9.6/lib/ant-junit4.jar 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:java.io.tmpdir=/tmp 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:java.compiler=<NA> 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:os.name=Linux 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:os.arch=amd64 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:os.version=3.2.0-33-virtual 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:user.name=jenkins 2017-01-24 22:12:20,594 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:user.home=/var/lib/jenkins 2017-01-24 22:12:20,595 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Environment@100] - Server environment:user.dir=/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7 2017-01-24 22:12:20,597 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4158500655544466289.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4158500655544466289.junit.dir/data/version-2 2017-01-24 22:12:20,597 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8717551953335097579.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8717551953335097579.junit.dir/data/version-2 2017-01-24 22:12:20,599 [myid:] - INFO [Thread-0:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 2017-01-24 22:12:20,599 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:Leader@356] - LEADING - LEADER ELECTION TOOK - 247 2017-01-24 22:12:20,600 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:58471 2017-01-24 22:12:20,600 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:58471 2017-01-24 22:12:20,601 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 247 2017-01-24 22:12:20,604 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4158500655544466289.junit.dir/data/version-2/snapshot.0 2017-01-24 22:12:20,608 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:20,609 [myid:0] - INFO [Thread-7:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:58471 (no session established for client) 2017-01-24 22:12:20,619 [myid:1] - INFO [LearnerHandler-/127.0.0.1:57759:LearnerHandler@287] - Follower sid: 0 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@2b6c7d59 2017-01-24 22:12:20,621 [myid:1] - INFO [LearnerHandler-/127.0.0.1:57759:LearnerHandler@342] - Synchronizing with Follower sid: 0 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0 2017-01-24 22:12:20,621 [myid:1] - INFO [LearnerHandler-/127.0.0.1:57759:LearnerHandler@441] - Sending snapshot last zxid of peer is 0x0 zxid of leader is 0x100000000sent zxid of db as 0x0 2017-01-24 22:12:20,622 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Learner@329] - Getting a snapshot from leader 2017-01-24 22:12:20,626 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8717551953335097579.junit.dir/data/version-2/snapshot.0 2017-01-24 22:12:20,627 [myid:1] - INFO [LearnerHandler-/127.0.0.1:57759:LearnerHandler@477] - Received NEWLEADER-ACK message from 0 2017-01-24 22:12:20,627 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:Leader@934] - Have quorum of supporters, sids: [ 0,1 ]; starting up and setting last processed zxid: 0x100000000 2017-01-24 22:12:20,860 [myid:] - INFO [Thread-0:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 2017-01-24 22:12:20,860 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:58473 2017-01-24 22:12:20,861 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:58473 2017-01-24 22:12:20,862 [myid:0] - INFO [Thread-10:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:12:20,863 [myid:0] - INFO [Thread-10:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:58473 (no session established for client) 2017-01-24 22:12:20,863 [myid:] - INFO [Thread-0:FourLetterWordMain@43] - connecting to 127.0.0.1 11224 2017-01-24 22:12:20,864 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11224:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:36288 2017-01-24 22:12:20,864 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11224:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:36288 2017-01-24 22:12:20,864 [myid:1] - INFO [Thread-11:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:12:20,865 [myid:1] - INFO [Thread-11:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:36288 (no session established for client) 2017-01-24 22:12:20,869 [myid:] - INFO [Thread-0:Environment@100] - Client environment:zookeeper.version=3.4.5--1, built on 01/25/2017 06:01 GMT 2017-01-24 22:12:20,869 [myid:] - INFO [Thread-0:Environment@100] - Client environment:host.name=ec2-unittest-ub12-m3-large-0e11.vpc.cloudera.com 2017-01-24 22:12:20,869 [myid:] - INFO [Thread-0:Environment@100] - Client environment:java.version=1.7.0_75 2017-01-24 22:12:20,870 [myid:] - INFO [Thread-0:Environment@100] - Client environment:java.vendor=Oracle Corporation 2017-01-24 22:12:20,870 [myid:] - INFO [Thread-0:Environment@100] - Client environment:java.home=/data/jenkins/tools/hudson.model.JDK/Java_7/jre 2017-01-24 22:12:20,870 [myid:] - INFO [Thread-0:Environment@100] - Client environment:java.class.path=/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/classes:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/antlr-2.7.6.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/checkstyle-5.0.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-beanutils-core-1.7.0.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-cli-1.0.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-collections-3.2.2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-io-2.4.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-lang-1.0.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/commons-logging-1.0.3.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/google-collections-0.9.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/junit-4.8.1.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-admin-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-client-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-common-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-core-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-crypto-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-identity-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-server-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-simplekdc-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerb-util-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerby-asn1-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerby-config-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerby-pkix-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/kerby-util-1.0.0-RC2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/log4j-1.2.17.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/mockito-all-1.8.2.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/slf4j-api-1.7.14.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/lib/slf4j-log4j12-1.7.14.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/classes:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/src/java/lib/ivy-2.2.0.jar:/data/jenkins/tools/hudson.tasks.Ant_AntInstallation/Ant_1.8.2/apache-ant-1.9.6/lib/ant.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/lib/jline-2.11.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/lib/log4j-1.2.16.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/lib/netty-3.10.5.Final.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/lib/slf4j-api-1.7.5.jar:/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/lib/slf4j-log4j12-1.7.5.jar:/mnt/toolchain/clover-ant-4.0.3/lib/clover.jar:/data/jenkins/tools/hudson.tasks.Ant_AntInstallation/Ant_1.8.2/apache-ant-1.9.6/lib/ant-launcher.jar:/data/jenkins/tools/hudson.tasks.Ant_AntInstallation/Ant_1.8.2/apache-ant-1.9.6/lib/ant-junit.jar:/data/jenkins/tools/hudson.tasks.Ant_AntInstallation/Ant_1.8.2/apache-ant-1.9.6/lib/ant-junit4.jar 2017-01-24 22:12:20,870 [myid:] - INFO [Thread-0:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2017-01-24 22:12:20,870 [myid:] - INFO [Thread-0:Environment@100] - Client environment:java.io.tmpdir=/tmp 2017-01-24 22:12:20,870 [myid:] - INFO [Thread-0:Environment@100] - Client environment:java.compiler=<NA> 2017-01-24 22:12:20,870 [myid:] - INFO [Thread-0:Environment@100] - Client environment:os.name=Linux 2017-01-24 22:12:20,870 [myid:] - INFO [Thread-0:Environment@100] - Client environment:os.arch=amd64 2017-01-24 22:12:20,870 [myid:] - INFO [Thread-0:Environment@100] - Client environment:os.version=3.2.0-33-virtual 2017-01-24 22:12:20,870 [myid:] - INFO [Thread-0:Environment@100] - Client environment:user.name=jenkins 2017-01-24 22:12:20,870 [myid:] - INFO [Thread-0:Environment@100] - Client environment:user.home=/var/lib/jenkins 2017-01-24 22:12:20,871 [myid:] - INFO [Thread-0:Environment@100] - Client environment:user.dir=/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7 2017-01-24 22:12:20,871 [myid:] - INFO [Thread-0:ZooKeeper@438] - Initiating client connection, connectString=127.0.0.1:11221,127.0.0.1:11224 sessionTimeout=30000 watcher=org.apache.zookeeper.test.ClientBase$CountdownWatcher@693a2c6a 2017-01-24 22:12:20,889 [myid:] - WARN [Thread-0-SendThread(localhost:11221):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:20,891 [myid:] - INFO [Thread-0-SendThread(localhost:11221):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11221 2017-01-24 22:12:20,891 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:58475 2017-01-24 22:12:20,891 [myid:] - INFO [Thread-0-SendThread(localhost:11221):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:58475, server: localhost/127.0.0.1:11221 2017-01-24 22:12:20,893 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:ZooKeeperServer@839] - Client attempting to establish new session at /127.0.0.1:58475 2017-01-24 22:12:20,896 [myid:1] - INFO [SyncThread:1:FileTxnLog@199] - Creating new log file: log.100000001 2017-01-24 22:12:20,897 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Follower@119] - Got zxid 0x100000001 expected 0x1 2017-01-24 22:12:20,897 [myid:0] - INFO [SyncThread:0:FileTxnLog@199] - Creating new log file: log.100000001 2017-01-24 22:12:20,906 [myid:0] - INFO [CommitProcessor:0:ZooKeeperServer@595] - Established session 0x59d440e8120000 with negotiated timeout 30000 for client /127.0.0.1:58475 2017-01-24 22:12:20,906 [myid:] - INFO [Thread-0-SendThread(localhost:11221):ClientCnxn$SendThread@1235] - Session establishment complete on server localhost/127.0.0.1:11221, sessionid = 0x59d440e8120000, negotiated timeout = 30000 2017-01-24 22:12:20,922 [myid:1] - INFO [ProcessThread(sid:1 cport:-1)::PrepRequestProcessor@494] - Processed session termination for sessionid: 0x59d440e8120000 2017-01-24 22:12:20,925 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:58475 which had sessionid 0x59d440e8120000 2017-01-24 22:12:20,925 [myid:] - INFO [Thread-0:ZooKeeper@684] - Session: 0x59d440e8120000 closed 2017-01-24 22:12:20,926 [myid:] - INFO [Thread-0:JUnit4ZKTestRunner$LoggedInvokeMethod@55] - Memory used 14639 2017-01-24 22:12:20,926 [myid:] - INFO [Thread-0:JUnit4ZKTestRunner$LoggedInvokeMethod@60] - Number of threads 39 2017-01-24 22:12:20,926 [myid:] - INFO [Thread-0:JUnit4ZKTestRunner$LoggedInvokeMethod@65] - FINISHED TEST METHOD testAuthLearnerAgainstNoAuthRequiredServer 2017-01-24 22:12:20,926 [myid:] - INFO [Thread-0-EventThread:ClientCnxn$EventThread@512] - EventThread shut down 2017-01-24 22:12:20,927 [myid:] - INFO [main:QuorumBase@314] - Shutting down quorum peer QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221 2017-01-24 22:12:20,927 [myid:] - INFO [main:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:896) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:315) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$TestQPMain.shutdown(QuorumPeerTestBase.java:59) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.shutdown(QuorumPeerTestBase.java:152) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdown(QuorumAuthTestBase.java:138) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdownAll(QuorumAuthTestBase.java:131) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.tearDown(QuorumAuthUpgradeTest.java:68) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033) 2017-01-24 22:12:20,928 [myid:] - INFO [main:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:20,928 [myid:] - INFO [main:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:20,928 [myid:] - INFO [main:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:12:20,928 [myid:] - INFO [main:CommitProcessor@181] - Shutting down 2017-01-24 22:12:20,928 [myid:0] - INFO [FollowerRequestProcessor:0:FollowerRequestProcessor@95] - FollowerRequestProcessor exited loop! 2017-01-24 22:12:20,929 [myid:] - INFO [main:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:20,929 [myid:0] - INFO [CommitProcessor:0:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:20,929 [myid:] - INFO [main:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:20,929 [myid:0] - INFO [SyncThread:0:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:20,930 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method 2017-01-24 22:12:20,930 [myid:0] - ERROR [localhost/127.0.0.1:11223:QuorumCnxManager$Listener@715] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:696) 2017-01-24 22:12:20,930 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:20,930 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:20,931 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:20,931 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:20,931 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:20,932 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:20,931 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:20,932 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:20,933 [myid:] - INFO [main:QuorumBase@318] - Shutting down leader election QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221 2017-01-24 22:12:20,933 [myid:] - INFO [main:QuorumBase@323] - Waiting for QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221 to exit thread 2017-01-24 22:12:21,930 [myid:0] - INFO [localhost/127.0.0.1:11223:QuorumCnxManager$Listener@728] - Leaving listener 2017-01-24 22:12:22,648 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:12:22,648 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:22,648 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:22,648 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:12:22,648 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:CommitProcessor@181] - Shutting down 2017-01-24 22:12:22,648 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:22,648 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:22,648 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11221:QuorumPeer@879] - QuorumPeer main thread exited 2017-01-24 22:12:22,651 [myid:] - INFO [main:ZKTestCase$1@60] - SUCCEEDED testAuthLearnerAgainstNoAuthRequiredServer 2017-01-24 22:12:22,651 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testAuthLearnerAgainstNoAuthRequiredServer 2017-01-24 22:12:22,652 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testRollingUpgrade 2017-01-24 22:12:22,653 [myid:] - INFO [Thread-12:JUnit4ZKTestRunner$LoggedInvokeMethod@50] - RUNNING TEST METHOD testRollingUpgrade 2017-01-24 22:12:22,653 [myid:] - INFO [Thread-12:PortAssignment@32] - assigning port 11227 2017-01-24 22:12:22,653 [myid:] - INFO [Thread-12:PortAssignment@32] - assigning port 11228 2017-01-24 22:12:22,653 [myid:] - INFO [Thread-12:PortAssignment@32] - assigning port 11229 2017-01-24 22:12:22,653 [myid:] - INFO [Thread-12:PortAssignment@32] - assigning port 11230 2017-01-24 22:12:22,653 [myid:] - INFO [Thread-12:PortAssignment@32] - assigning port 11231 2017-01-24 22:12:22,653 [myid:] - INFO [Thread-12:PortAssignment@32] - assigning port 11232 2017-01-24 22:12:22,653 [myid:] - INFO [Thread-12:PortAssignment@32] - assigning port 11233 2017-01-24 22:12:22,654 [myid:] - INFO [Thread-12:PortAssignment@32] - assigning port 11234 2017-01-24 22:12:22,654 [myid:] - INFO [Thread-12:PortAssignment@32] - assigning port 11235 2017-01-24 22:12:22,654 [myid:] - INFO [Thread-12:QuorumPeerTestBase$MainThread@81] - id = 0 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test1594648493062198412.junit.dir clientPort = 11227 2017-01-24 22:12:22,655 [myid:] - INFO [Thread-13:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test1594648493062198412.junit.dir/zoo.cfg 2017-01-24 22:12:22,655 [myid:] - INFO [Thread-12:QuorumPeerTestBase$MainThread@81] - id = 1 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8525006967275020093.junit.dir clientPort = 11230 2017-01-24 22:12:22,655 [myid:] - INFO [Thread-13:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:12:22,656 [myid:0] - INFO [Thread-13:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:12:22,656 [myid:0] - INFO [Thread-13:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:12:22,656 [myid:0] - INFO [Thread-13:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:12:22,656 [myid:] - INFO [Thread-12:QuorumPeerTestBase$MainThread@81] - id = 2 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2781541844609161589.junit.dir clientPort = 11233 2017-01-24 22:12:22,656 [myid:0] - WARN [Thread-13:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:22,657 [myid:0] - INFO [Thread-13:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:12:22,657 [myid:] - INFO [Thread-15:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2781541844609161589.junit.dir/zoo.cfg 2017-01-24 22:12:22,657 [myid:0] - INFO [Thread-13:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11227 2017-01-24 22:12:22,657 [myid:] - INFO [Thread-15:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:12:22,658 [myid:0] - INFO [Thread-13:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:12:22,658 [myid:0] - INFO [Thread-13:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:12:22,656 [myid:] - INFO [Thread-14:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8525006967275020093.junit.dir/zoo.cfg 2017-01-24 22:12:22,658 [myid:0] - INFO [Thread-13:QuorumPeer@1277] - QuorumPeer communication is not secured! 2017-01-24 22:12:22,658 [myid:0] - INFO [Thread-13:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:12:22,658 [myid:] - INFO [Thread-14:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:12:22,659 [myid:0] - INFO [Thread-13:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:22,659 [myid:1] - INFO [Thread-14:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:12:22,659 [myid:1] - INFO [Thread-14:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:12:22,659 [myid:1] - INFO [Thread-14:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:12:22,659 [myid:1] - WARN [Thread-14:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:22,660 [myid:1] - INFO [Thread-14:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:12:22,660 [myid:1] - INFO [Thread-14:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11230 2017-01-24 22:12:22,660 [myid:1] - INFO [Thread-14:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:12:22,660 [myid:1] - INFO [Thread-14:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:12:22,660 [myid:1] - INFO [Thread-14:QuorumPeer@1277] - QuorumPeer communication is not secured! 2017-01-24 22:12:22,661 [myid:1] - INFO [Thread-14:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:12:22,661 [myid:1] - INFO [Thread-14:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:22,658 [myid:2] - INFO [Thread-15:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:12:22,661 [myid:2] - INFO [Thread-15:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:12:22,661 [myid:2] - INFO [Thread-15:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:12:22,657 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11227 2017-01-24 22:12:22,662 [myid:2] - WARN [Thread-15:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:22,662 [myid:2] - INFO [Thread-15:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:12:22,660 [myid:0] - INFO [Thread-13:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:22,663 [myid:2] - INFO [Thread-15:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11233 2017-01-24 22:12:22,663 [myid:2] - INFO [Thread-15:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:12:22,663 [myid:2] - INFO [Thread-15:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:12:22,663 [myid:2] - INFO [Thread-15:QuorumPeer@1277] - QuorumPeer communication is not secured! 2017-01-24 22:12:22,663 [myid:2] - INFO [Thread-15:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:12:22,664 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42807 2017-01-24 22:12:22,664 [myid:2] - INFO [Thread-15:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:22,664 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:42807 2017-01-24 22:12:22,665 [myid:0] - INFO [Thread-16:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11229 2017-01-24 22:12:22,665 [myid:1] - INFO [Thread-14:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:22,666 [myid:0] - INFO [Thread-17:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42807 (no session established for client) 2017-01-24 22:12:22,666 [myid:2] - INFO [Thread-15:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:22,667 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@781] - LOOKING 2017-01-24 22:12:22,667 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FastLeaderElection@744] - New election. My id = 0, proposed zxid=0x0 2017-01-24 22:12:22,668 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,668 [myid:0] - WARN [WorkerSender[myid=0]:QuorumCnxManager@559] - Cannot open channel to 1 at election address localhost/127.0.0.1:11232 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:22,669 [myid:0] - WARN [WorkerSender[myid=0]:QuorumCnxManager@559] - Cannot open channel to 2 at election address localhost/127.0.0.1:11235 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:22,670 [myid:1] - INFO [Thread-18:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11232 2017-01-24 22:12:22,670 [myid:2] - INFO [Thread-19:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11235 2017-01-24 22:12:22,672 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@781] - LOOKING 2017-01-24 22:12:22,672 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@781] - LOOKING 2017-01-24 22:12:22,672 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FastLeaderElection@744] - New election. My id = 1, proposed zxid=0x0 2017-01-24 22:12:22,673 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FastLeaderElection@744] - New election. My id = 2, proposed zxid=0x0 2017-01-24 22:12:22,674 [myid:0] - INFO [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:51824 2017-01-24 22:12:22,680 [myid:1] - INFO [localhost/127.0.0.1:11232:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:45173 2017-01-24 22:12:22,681 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,682 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,683 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,683 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,683 [myid:0] - INFO [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:51825 2017-01-24 22:12:22,683 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,684 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,684 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,685 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,685 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,685 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,686 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,686 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,686 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,686 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,687 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:22,887 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@861] - LEADING 2017-01-24 22:12:22,887 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:22,887 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test1594648493062198412.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test1594648493062198412.junit.dir/data/version-2 2017-01-24 22:12:22,887 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2781541844609161589.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2781541844609161589.junit.dir/data/version-2 2017-01-24 22:12:22,887 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 220 2017-01-24 22:12:22,887 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Leader@356] - LEADING - LEADER ELECTION TOOK - 214 2017-01-24 22:12:22,888 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2781541844609161589.junit.dir/data/version-2/snapshot.0 2017-01-24 22:12:22,887 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:22,889 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8525006967275020093.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8525006967275020093.junit.dir/data/version-2 2017-01-24 22:12:22,889 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 217 2017-01-24 22:12:22,890 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42553:LearnerHandler@287] - Follower sid: 0 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@559c4c8e 2017-01-24 22:12:22,890 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42554:LearnerHandler@287] - Follower sid: 1 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@4f1e5ed9 2017-01-24 22:12:22,892 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42553:LearnerHandler@342] - Synchronizing with Follower sid: 0 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0 2017-01-24 22:12:22,892 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42554:LearnerHandler@342] - Synchronizing with Follower sid: 1 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0 2017-01-24 22:12:22,892 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42553:LearnerHandler@441] - Sending snapshot last zxid of peer is 0x0 zxid of leader is 0x100000000sent zxid of db as 0x0 2017-01-24 22:12:22,892 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Learner@329] - Getting a snapshot from leader 2017-01-24 22:12:22,892 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Learner@329] - Getting a snapshot from leader 2017-01-24 22:12:22,893 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42554:LearnerHandler@441] - Sending snapshot last zxid of peer is 0x0 zxid of leader is 0x100000000sent zxid of db as 0x0 2017-01-24 22:12:22,894 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test1594648493062198412.junit.dir/data/version-2/snapshot.0 2017-01-24 22:12:22,894 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8525006967275020093.junit.dir/data/version-2/snapshot.0 2017-01-24 22:12:22,895 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42553:LearnerHandler@477] - Received NEWLEADER-ACK message from 0 2017-01-24 22:12:22,895 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Leader@934] - Have quorum of supporters, sids: [ 0,2 ]; starting up and setting last processed zxid: 0x100000000 2017-01-24 22:12:22,895 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42554:LearnerHandler@477] - Received NEWLEADER-ACK message from 1 2017-01-24 22:12:22,917 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11227 2017-01-24 22:12:22,918 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42815 2017-01-24 22:12:22,918 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:42815 2017-01-24 22:12:22,918 [myid:0] - INFO [Thread-23:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:12:22,919 [myid:0] - INFO [Thread-23:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42815 (no session established for client) 2017-01-24 22:12:22,919 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11230 2017-01-24 22:12:22,920 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52317 2017-01-24 22:12:22,920 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:52317 2017-01-24 22:12:22,921 [myid:1] - INFO [Thread-24:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:12:22,921 [myid:1] - INFO [Thread-24:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52317 (no session established for client) 2017-01-24 22:12:22,921 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:22,922 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45787 2017-01-24 22:12:22,923 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45787 2017-01-24 22:12:22,923 [myid:2] - INFO [Thread-25:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:12:22,924 [myid:2] - INFO [Thread-25:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45787 (no session established for client) 2017-01-24 22:12:22,924 [myid:] - INFO [Thread-12:ZooKeeper@438] - Initiating client connection, connectString=127.0.0.1:11227,127.0.0.1:11230,127.0.0.1:11233 sessionTimeout=30000 watcher=org.apache.zookeeper.test.ClientBase$CountdownWatcher@36c0ae96 2017-01-24 22:12:22,925 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:22,925 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:22,926 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52319 2017-01-24 22:12:22,926 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52319, server: localhost/127.0.0.1:11230 2017-01-24 22:12:22,927 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:ZooKeeperServer@839] - Client attempting to establish new session at /127.0.0.1:52319 2017-01-24 22:12:22,930 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@119] - Got zxid 0x100000001 expected 0x1 2017-01-24 22:12:22,930 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@119] - Got zxid 0x100000001 expected 0x1 2017-01-24 22:12:22,930 [myid:2] - INFO [SyncThread:2:FileTxnLog@199] - Creating new log file: log.100000001 2017-01-24 22:12:22,931 [myid:1] - INFO [SyncThread:1:FileTxnLog@199] - Creating new log file: log.100000001 2017-01-24 22:12:22,931 [myid:0] - INFO [SyncThread:0:FileTxnLog@199] - Creating new log file: log.100000001 2017-01-24 22:12:22,933 [myid:1] - INFO [CommitProcessor:1:ZooKeeperServer@595] - Established session 0x159d440f0ed0000 with negotiated timeout 30000 for client /127.0.0.1:52319 2017-01-24 22:12:22,933 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1235] - Session establishment complete on server localhost/127.0.0.1:11230, sessionid = 0x159d440f0ed0000, negotiated timeout = 30000 2017-01-24 22:12:22,937 [myid:] - INFO [Thread-12:QuorumAuthUpgradeTest@229] - Restarting server myid=0 2017-01-24 22:12:22,937 [myid:] - INFO [Thread-12:QuorumBase@314] - Shutting down quorum peer QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227 2017-01-24 22:12:22,937 [myid:] - INFO [Thread-12:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:896) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:315) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$TestQPMain.shutdown(QuorumPeerTestBase.java:59) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.shutdown(QuorumPeerTestBase.java:152) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdown(QuorumAuthTestBase.java:138) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.restartServer(QuorumAuthUpgradeTest.java:230) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.testRollingUpgrade(QuorumAuthUpgradeTest.java:194) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52) at org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28) 2017-01-24 22:12:22,938 [myid:] - INFO [Thread-12:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:22,938 [myid:] - INFO [Thread-12:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:22,938 [myid:] - INFO [Thread-12:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:12:22,938 [myid:] - INFO [Thread-12:CommitProcessor@181] - Shutting down 2017-01-24 22:12:22,938 [myid:] - INFO [Thread-12:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:22,938 [myid:0] - INFO [FollowerRequestProcessor:0:FollowerRequestProcessor@95] - FollowerRequestProcessor exited loop! 2017-01-24 22:12:22,938 [myid:0] - INFO [CommitProcessor:0:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:22,939 [myid:] - INFO [Thread-12:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:22,939 [myid:0] - INFO [SyncThread:0:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:22,939 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method 2017-01-24 22:12:22,939 [myid:0] - ERROR [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@715] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:696) 2017-01-24 22:12:22,939 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:22,940 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:22,940 [myid:2] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 2, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:22,940 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:22,941 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:22,940 [myid:0] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:22,941 [myid:0] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:22,940 [myid:0] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@980] - Connection broken for id 2, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:22,941 [myid:0] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:22,941 [myid:] - INFO [Thread-12:QuorumBase@318] - Shutting down leader election QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227 2017-01-24 22:12:22,941 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:22,942 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:22,940 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:22,942 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:22,940 [myid:2] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:22,942 [myid:] - INFO [Thread-12:QuorumBase@323] - Waiting for QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227 to exit thread 2017-01-24 22:12:22,943 [myid:2] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:22,943 [myid:2] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:23,380 [myid:0] - INFO [WorkerSender[myid=0]:FastLeaderElection$Messenger$WorkerSender@370] - WorkerSender is down 2017-01-24 22:12:23,380 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection$Messenger$WorkerReceiver@340] - WorkerReceiver is down 2017-01-24 22:12:23,940 [myid:0] - INFO [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@728] - Leaving listener 2017-01-24 22:12:24,897 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:12:24,897 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:24,897 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:24,898 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:12:24,898 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:CommitProcessor@181] - Shutting down 2017-01-24 22:12:24,898 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:24,898 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:24,898 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@879] - QuorumPeer main thread exited 2017-01-24 22:12:24,901 [myid:] - INFO [Thread-12:QuorumPeerTestBase$MainThread@81] - id = 0 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir clientPort = 11227 2017-01-24 22:12:24,901 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11227 2017-01-24 22:12:24,901 [myid:] - INFO [Thread-26:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/zoo.cfg 2017-01-24 22:12:24,902 [myid:] - INFO [Thread-12:ClientBase@246] - server 127.0.0.1:11227 not up java.net.ConnectException: Connection refused 2017-01-24 22:12:24,902 [myid:] - INFO [Thread-26:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:12:24,902 [myid:0] - INFO [Thread-26:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:12:24,902 [myid:0] - INFO [Thread-26:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:12:24,902 [myid:0] - INFO [Thread-26:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:12:24,903 [myid:0] - WARN [Thread-26:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:24,903 [myid:0] - INFO [Thread-26:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:12:24,903 [myid:0] - INFO [Thread-26:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11227 2017-01-24 22:12:24,904 [myid:0] - INFO [Thread-26:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:12:24,904 [myid:0] - INFO [Thread-26:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:12:24,904 [myid:0] - INFO [Thread-26:QuorumPeer@1279] - quorum.auth.enableSasl set to true 2017-01-24 22:12:24,904 [myid:0] - INFO [Thread-26:QuorumPeer@1264] - quorum.auth.serverRequireSasl set to false 2017-01-24 22:12:24,904 [myid:0] - INFO [Thread-26:QuorumPeer@1270] - quorum.auth.learnerRequireSasl set to false 2017-01-24 22:12:24,904 [myid:0] - INFO [Thread-26:QuorumPeer@1286] - quorum.auth.kerberos.servicePrincipal set to zkquorum/localhost 2017-01-24 22:12:24,904 [myid:0] - INFO [Thread-26:QuorumPeer@1298] - quorum.auth.server.saslLoginContext set to QuorumServer 2017-01-24 22:12:24,904 [myid:0] - INFO [Thread-26:QuorumPeer@1292] - quorum.auth.learner.saslLoginContext set to QuorumLearner 2017-01-24 22:12:24,904 [myid:0] - INFO [Thread-26:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:12:24,905 [myid:0] - INFO [Thread-26:Login@294] - QuorumServer successfully logged in. 2017-01-24 22:12:24,905 [myid:0] - INFO [Thread-26:Login@294] - QuorumLearner successfully logged in. 2017-01-24 22:12:24,905 [myid:0] - INFO [Thread-26:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:24,906 [myid:0] - INFO [Thread-26:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:24,911 [myid:0] - INFO [Thread-27:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11229 2017-01-24 22:12:24,916 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@781] - LOOKING 2017-01-24 22:12:24,916 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FastLeaderElection@744] - New election. My id = 0, proposed zxid=0x0 2017-01-24 22:12:24,917 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:24,917 [myid:1] - INFO [localhost/127.0.0.1:11232:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:45181 2017-01-24 22:12:24,919 [myid:2] - INFO [localhost/127.0.0.1:11235:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:50340 2017-01-24 22:12:24,919 [myid:0] - INFO [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:51836 2017-01-24 22:12:24,920 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:24,921 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-2:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:24,921 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-2:QuorumCnxManager@331] - Have smaller server identifier, so dropping the connection: (2, 0) 2017-01-24 22:12:24,922 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:QuorumCnxManager@331] - Have smaller server identifier, so dropping the connection: (1, 0) 2017-01-24 22:12:24,926 [myid:0] - INFO [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:51837 2017-01-24 22:12:24,933 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state) 2017-01-24 22:12:24,933 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:24,934 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LEADING (my state) 2017-01-24 22:12:24,934 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:24,934 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:24,934 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LEADING (my state) 2017-01-24 22:12:24,935 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state) 2017-01-24 22:12:24,937 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), FOLLOWING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:24,937 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), FOLLOWING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:24,937 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LEADING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:24,937 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LEADING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,138 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:25,138 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2 2017-01-24 22:12:25,138 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 222 2017-01-24 22:12:25,139 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:25,139 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42564:LearnerHandler@287] - Follower sid: 0 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@559c4c8e 2017-01-24 22:12:25,141 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42564:LearnerHandler@342] - Synchronizing with Follower sid: 0 maxCommittedLog=0x100000002 minCommittedLog=0x100000001 peerLastZxid=0x0 2017-01-24 22:12:25,141 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42564:LearnerHandler@405] - Unhandled proposal scenario 2017-01-24 22:12:25,141 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42564:LearnerHandler@441] - Sending snapshot last zxid of peer is 0x0 zxid of leader is 0x100000002sent zxid of db as 0x100000002 2017-01-24 22:12:25,141 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Learner@329] - Getting a snapshot from leader 2017-01-24 22:12:25,143 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FileTxnSnapLog@281] - Snapshotting: 0x100000002 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2/snapshot.100000002 2017-01-24 22:12:25,145 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42564:LearnerHandler@477] - Received NEWLEADER-ACK message from 0 2017-01-24 22:12:25,152 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11227 2017-01-24 22:12:25,153 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42825 2017-01-24 22:12:25,153 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:42825 2017-01-24 22:12:25,153 [myid:0] - INFO [Thread-29:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:12:25,156 [myid:0] - INFO [Thread-29:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42825 (no session established for client) 2017-01-24 22:12:25,159 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@119] - Got zxid 0x100000003 expected 0x1 2017-01-24 22:12:25,159 [myid:0] - INFO [SyncThread:0:FileTxnLog@199] - Creating new log file: log.100000003 2017-01-24 22:12:25,160 [myid:] - INFO [Thread-12:QuorumAuthUpgradeTest@229] - Restarting server myid=1 2017-01-24 22:12:25,161 [myid:] - INFO [Thread-12:QuorumBase@314] - Shutting down quorum peer QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 2017-01-24 22:12:25,161 [myid:] - INFO [Thread-12:Leader@491] - Shutting down 2017-01-24 22:12:25,161 [myid:] - INFO [Thread-12:Leader@497] - Shutdown called java.lang.Exception: shutdown Leader! reason: quorum Peer shutdown at org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:497) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:893) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:315) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$TestQPMain.shutdown(QuorumPeerTestBase.java:59) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.shutdown(QuorumPeerTestBase.java:152) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdown(QuorumAuthTestBase.java:138) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.restartServer(QuorumAuthUpgradeTest.java:230) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.testRollingUpgrade(QuorumAuthUpgradeTest.java:195) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52) at org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28) 2017-01-24 22:12:25,161 [myid:] - INFO [Thread-12:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:25,161 [myid:2] - INFO [Thread-20:Leader$LearnerCnxAcceptor@318] - exception while shutting down acceptor: java.net.SocketException: Socket closed 2017-01-24 22:12:25,162 [myid:] - INFO [Thread-12:SessionTrackerImpl@225] - Shutting down 2017-01-24 22:12:25,162 [myid:] - INFO [Thread-12:PrepRequestProcessor@761] - Shutting down 2017-01-24 22:12:25,163 [myid:] - INFO [Thread-12:ProposalRequestProcessor@88] - Shutting down 2017-01-24 22:12:25,163 [myid:] - INFO [Thread-12:CommitProcessor@181] - Shutting down 2017-01-24 22:12:25,163 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop! 2017-01-24 22:12:25,163 [myid:] - INFO [Thread-12:Leader$ToBeAppliedRequestProcessor@656] - Shutting down 2017-01-24 22:12:25,163 [myid:] - INFO [Thread-12:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:25,163 [myid:2] - INFO [CommitProcessor:2:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:25,163 [myid:] - INFO [Thread-12:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:25,164 [myid:2] - INFO [SyncThread:2:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:25,164 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@90] - Exception when following the leader java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:152) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:86) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:851) 2017-01-24 22:12:25,164 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@90] - Exception when following the leader java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:152) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:86) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:851) 2017-01-24 22:12:25,165 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:12:25,165 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52319 which had sessionid 0x159d440f0ed0000 2017-01-24 22:12:25,165 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:25,166 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:25,166 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:12:25,166 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:CommitProcessor@181] - Shutting down 2017-01-24 22:12:25,166 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:25,166 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:25,166 [myid:1] - INFO [FollowerRequestProcessor:1:FollowerRequestProcessor@95] - FollowerRequestProcessor exited loop! 2017-01-24 22:12:25,164 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42564:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:42564 ******** 2017-01-24 22:12:25,166 [myid:1] - INFO [SyncThread:1:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:25,166 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method 2017-01-24 22:12:25,166 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42553:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:42553 ******** 2017-01-24 22:12:25,166 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:25,167 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@867] - Unexpected exception java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:451) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:864) 2017-01-24 22:12:25,167 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Leader@491] - Shutting down 2017-01-24 22:12:25,168 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@879] - QuorumPeer main thread exited 2017-01-24 22:12:25,165 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42554:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:42554 ******** 2017-01-24 22:12:25,165 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:12:25,168 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42554:LearnerHandler@610] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:608) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:601) 2017-01-24 22:12:25,167 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42553:LearnerHandler@610] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:608) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:601) 2017-01-24 22:12:25,167 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@781] - LOOKING 2017-01-24 22:12:25,171 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8525006967275020093.junit.dir/data/version-2/snapshot.0 2017-01-24 22:12:25,173 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FastLeaderElection@744] - New election. My id = 1, proposed zxid=0x100000003 2017-01-24 22:12:25,174 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,167 [myid:1] - INFO [CommitProcessor:1:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:25,174 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,167 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42564:LearnerHandler@610] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:608) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:601) 2017-01-24 22:12:25,174 [myid:2] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:25,175 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:25,176 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:25,174 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x1 (n.peerEPoch), FOLLOWING (my state) 2017-01-24 22:12:25,169 [myid:2] - ERROR [localhost/127.0.0.1:11235:QuorumCnxManager$Listener@715] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:696) 2017-01-24 22:12:25,169 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:25,176 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:25,177 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:12:25,177 [myid:2] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 2, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:25,177 [myid:2] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:25,175 [myid:0] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@980] - Connection broken for id 2, my id = 0, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:25,177 [myid:0] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:25,175 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@980] - Connection broken for id 2, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:25,178 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:25,178 [myid:] - INFO [Thread-12:QuorumBase@318] - Shutting down leader election QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 2017-01-24 22:12:25,178 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), FOLLOWING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,178 [myid:2] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:25,179 [myid:2] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:25,179 [myid:] - INFO [Thread-12:QuorumBase@323] - Waiting for QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 to exit thread 2017-01-24 22:12:25,180 [myid:] - INFO [Thread-12:QuorumPeerTestBase$MainThread@81] - id = 2 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4813485841779601602.junit.dir clientPort = 11233 2017-01-24 22:12:25,179 [myid:0] - INFO [FollowerRequestProcessor:0:FollowerRequestProcessor@95] - FollowerRequestProcessor exited loop! 2017-01-24 22:12:25,179 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:CommitProcessor@181] - Shutting down 2017-01-24 22:12:25,178 [myid:0] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:25,183 [myid:0] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:25,178 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:25,183 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:25,183 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:25,183 [myid:0] - INFO [CommitProcessor:0:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:25,182 [myid:] - INFO [Thread-30:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4813485841779601602.junit.dir/zoo.cfg 2017-01-24 22:12:25,182 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:25,183 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:25,184 [myid:0] - INFO [SyncThread:0:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:25,185 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@781] - LOOKING 2017-01-24 22:12:25,185 [myid:] - INFO [Thread-30:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:12:25,186 [myid:2] - INFO [Thread-30:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:12:25,186 [myid:2] - INFO [Thread-30:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:12:25,186 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2/snapshot.100000002 2017-01-24 22:12:25,186 [myid:2] - INFO [Thread-30:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:12:25,186 [myid:2] - WARN [Thread-30:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:25,187 [myid:2] - INFO [Thread-30:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:12:25,187 [myid:2] - INFO [Thread-30:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11233 2017-01-24 22:12:25,188 [myid:] - INFO [Thread-12:ClientBase@246] - server 127.0.0.1:11233 not up java.net.ConnectException: Connection refused 2017-01-24 22:12:25,187 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FastLeaderElection@744] - New election. My id = 0, proposed zxid=0x100000003 2017-01-24 22:12:25,188 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 0 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,188 [myid:0] - WARN [WorkerSender[myid=0]:QuorumCnxManager@559] - Cannot open channel to 2 at election address localhost/127.0.0.1:11235 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:25,189 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 0 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,189 [myid:2] - INFO [Thread-30:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:12:25,189 [myid:2] - INFO [Thread-30:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:12:25,189 [myid:2] - INFO [Thread-30:QuorumPeer@1279] - quorum.auth.enableSasl set to true 2017-01-24 22:12:25,189 [myid:2] - INFO [Thread-30:QuorumPeer@1264] - quorum.auth.serverRequireSasl set to false 2017-01-24 22:12:25,189 [myid:2] - INFO [Thread-30:QuorumPeer@1270] - quorum.auth.learnerRequireSasl set to false 2017-01-24 22:12:25,190 [myid:2] - INFO [Thread-30:QuorumPeer@1286] - quorum.auth.kerberos.servicePrincipal set to zkquorum/localhost 2017-01-24 22:12:25,190 [myid:2] - INFO [Thread-30:QuorumPeer@1298] - quorum.auth.server.saslLoginContext set to QuorumServer 2017-01-24 22:12:25,190 [myid:2] - INFO [Thread-30:QuorumPeer@1292] - quorum.auth.learner.saslLoginContext set to QuorumLearner 2017-01-24 22:12:25,190 [myid:2] - INFO [Thread-30:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:12:25,190 [myid:2] - INFO [Thread-30:Login@294] - QuorumServer successfully logged in. 2017-01-24 22:12:25,190 [myid:2] - INFO [Thread-30:Login@294] - QuorumLearner successfully logged in. 2017-01-24 22:12:25,191 [myid:2] - INFO [Thread-30:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:25,192 [myid:2] - INFO [Thread-30:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:25,193 [myid:2] - INFO [Thread-31:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11235 2017-01-24 22:12:25,196 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@781] - LOOKING 2017-01-24 22:12:25,196 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FastLeaderElection@744] - New election. My id = 2, proposed zxid=0x0 2017-01-24 22:12:25,197 [myid:0] - INFO [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:51842 2017-01-24 22:12:25,197 [myid:1] - INFO [localhost/127.0.0.1:11232:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:45190 2017-01-24 22:12:25,198 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,202 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-2:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:25,204 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-1:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:25,210 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,210 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,210 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,212 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 0 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,212 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 2 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,212 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 2 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,213 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,213 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 2 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,214 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 0 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,214 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 0 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,214 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 0 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,214 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 0 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:25,414 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@861] - LEADING 2017-01-24 22:12:25,414 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8525006967275020093.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8525006967275020093.junit.dir/data/version-2 2017-01-24 22:12:25,415 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Leader@356] - LEADING - LEADER ELECTION TOOK - 245 2017-01-24 22:12:25,415 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8525006967275020093.junit.dir/data/version-2/snapshot.0 2017-01-24 22:12:25,416 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:25,416 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2 2017-01-24 22:12:25,416 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 231 2017-01-24 22:12:25,417 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:25,417 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FileTxnSnapLog@281] - Snapshotting: 0x100000003 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8525006967275020093.junit.dir/data/version-2/snapshot.100000003 2017-01-24 22:12:25,415 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:25,417 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4813485841779601602.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4813485841779601602.junit.dir/data/version-2 2017-01-24 22:12:25,417 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 221 2017-01-24 22:12:25,418 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:25,422 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50169:LearnerHandler@287] - Follower sid: 0 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@752a0b2f 2017-01-24 22:12:25,422 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50170:LearnerHandler@287] - Follower sid: 2 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1b4476c1 2017-01-24 22:12:25,424 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50169:LearnerHandler@342] - Synchronizing with Follower sid: 0 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x100000003 2017-01-24 22:12:25,424 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50169:LearnerHandler@441] - Sending snapshot last zxid of peer is 0x100000003 zxid of leader is 0x200000000sent zxid of db as 0x100000003 2017-01-24 22:12:25,424 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Learner@329] - Getting a snapshot from leader 2017-01-24 22:12:25,425 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50170:LearnerHandler@342] - Synchronizing with Follower sid: 2 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0 2017-01-24 22:12:25,426 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Learner@329] - Getting a snapshot from leader 2017-01-24 22:12:25,426 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50170:LearnerHandler@441] - Sending snapshot last zxid of peer is 0x0 zxid of leader is 0x200000000sent zxid of db as 0x100000003 2017-01-24 22:12:25,429 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FileTxnSnapLog@281] - Snapshotting: 0x100000003 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4813485841779601602.junit.dir/data/version-2/snapshot.100000003 2017-01-24 22:12:25,429 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FileTxnSnapLog@281] - Snapshotting: 0x100000003 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2/snapshot.100000003 2017-01-24 22:12:25,432 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50170:LearnerHandler@477] - Received NEWLEADER-ACK message from 2 2017-01-24 22:12:25,432 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Leader@934] - Have quorum of supporters, sids: [ 1,2 ]; starting up and setting last processed zxid: 0x200000000 2017-01-24 22:12:25,433 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50169:LearnerHandler@477] - Received NEWLEADER-ACK message from 0 2017-01-24 22:12:25,446 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:25,446 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45802 2017-01-24 22:12:25,447 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45802 2017-01-24 22:12:25,447 [myid:2] - INFO [Thread-35:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:12:25,448 [myid:2] - INFO [Thread-35:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45802 (no session established for client) 2017-01-24 22:12:25,686 [myid:0] - INFO [WorkerSender[myid=0]:FastLeaderElection$Messenger$WorkerSender@370] - WorkerSender is down 2017-01-24 22:12:25,687 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection$Messenger$WorkerReceiver@340] - WorkerReceiver is down 2017-01-24 22:12:26,176 [myid:2] - INFO [localhost/127.0.0.1:11235:QuorumCnxManager$Listener@728] - Leaving listener 2017-01-24 22:12:26,212 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:26,212 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:26,212 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45803 2017-01-24 22:12:26,212 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45803, server: localhost/127.0.0.1:11233 2017-01-24 22:12:26,213 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:ZooKeeperServer@832] - Client attempting to renew session 0x159d440f0ed0000 at /127.0.0.1:45803 2017-01-24 22:12:26,213 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:Learner@107] - Revalidating client: 0x159d440f0ed0000 2017-01-24 22:12:26,215 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:ZooKeeperServer@595] - Established session 0x159d440f0ed0000 with negotiated timeout 30000 for client /127.0.0.1:45803 2017-01-24 22:12:26,215 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1235] - Session establishment complete on server localhost/127.0.0.1:11233, sessionid = 0x159d440f0ed0000, negotiated timeout = 30000 2017-01-24 22:12:26,216 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Follower@119] - Got zxid 0x200000001 expected 0x1 2017-01-24 22:12:26,217 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@119] - Got zxid 0x200000001 expected 0x1 2017-01-24 22:12:26,217 [myid:2] - INFO [SyncThread:2:FileTxnLog@199] - Creating new log file: log.200000001 2017-01-24 22:12:26,219 [myid:] - INFO [Thread-12:QuorumAuthUpgradeTest@229] - Restarting server myid=2 2017-01-24 22:12:26,220 [myid:] - INFO [Thread-12:QuorumBase@314] - Shutting down quorum peer QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 2017-01-24 22:12:26,220 [myid:] - INFO [Thread-12:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:896) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:315) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$TestQPMain.shutdown(QuorumPeerTestBase.java:59) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.shutdown(QuorumPeerTestBase.java:152) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdown(QuorumAuthTestBase.java:138) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.restartServer(QuorumAuthUpgradeTest.java:230) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.testRollingUpgrade(QuorumAuthUpgradeTest.java:196) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52) at org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28) 2017-01-24 22:12:26,220 [myid:] - INFO [Thread-12:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45803 which had sessionid 0x159d440f0ed0000 2017-01-24 22:12:26,221 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:26,221 [myid:] - INFO [Thread-12:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:26,221 [myid:] - INFO [Thread-12:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:26,221 [myid:] - INFO [Thread-12:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:12:26,221 [myid:] - INFO [Thread-12:CommitProcessor@181] - Shutting down 2017-01-24 22:12:26,221 [myid:2] - INFO [FollowerRequestProcessor:2:FollowerRequestProcessor@95] - FollowerRequestProcessor exited loop! 2017-01-24 22:12:26,221 [myid:2] - INFO [CommitProcessor:2:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:26,221 [myid:] - INFO [Thread-12:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:26,222 [myid:] - INFO [Thread-12:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:26,222 [myid:2] - INFO [SyncThread:2:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:26,223 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method 2017-01-24 22:12:26,223 [myid:2] - ERROR [localhost/127.0.0.1:11235:QuorumCnxManager$Listener@715] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:696) 2017-01-24 22:12:26,223 [myid:0] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@980] - Connection broken for id 2, my id = 0, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:26,224 [myid:0] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:26,224 [myid:0] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:26,224 [myid:2] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 2, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:26,225 [myid:2] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:26,225 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:26,226 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:26,225 [myid:0] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:26,224 [myid:2] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:26,226 [myid:2] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:26,225 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@980] - Connection broken for id 2, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:26,227 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:26,225 [myid:2] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 2, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:26,227 [myid:2] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:26,227 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:26,228 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:26,228 [myid:] - INFO [Thread-12:QuorumBase@318] - Shutting down leader election QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 2017-01-24 22:12:26,229 [myid:] - INFO [Thread-12:QuorumBase@323] - Waiting for QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 to exit thread 2017-01-24 22:12:26,611 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:26,611 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:26,612 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42834 2017-01-24 22:12:26,612 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:42834, server: localhost/127.0.0.1:11227 2017-01-24 22:12:26,612 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:ZooKeeperServer@832] - Client attempting to renew session 0x159d440f0ed0000 at /127.0.0.1:42834 2017-01-24 22:12:26,613 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:Learner@107] - Revalidating client: 0x159d440f0ed0000 2017-01-24 22:12:26,614 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:ZooKeeperServer@595] - Established session 0x159d440f0ed0000 with negotiated timeout 30000 for client /127.0.0.1:42834 2017-01-24 22:12:26,614 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1235] - Session establishment complete on server localhost/127.0.0.1:11227, sessionid = 0x159d440f0ed0000, negotiated timeout = 30000 2017-01-24 22:12:27,224 [myid:2] - INFO [localhost/127.0.0.1:11235:QuorumCnxManager$Listener@728] - Leaving listener 2017-01-24 22:12:27,435 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:12:27,435 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:27,435 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:27,435 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:12:27,436 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:CommitProcessor@181] - Shutting down 2017-01-24 22:12:27,436 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:27,436 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:27,436 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@879] - QuorumPeer main thread exited 2017-01-24 22:12:27,447 [myid:] - INFO [Thread-12:QuorumPeerTestBase$MainThread@81] - id = 2 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2397521463065956250.junit.dir clientPort = 11233 2017-01-24 22:12:27,452 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:27,453 [myid:] - INFO [Thread-12:ClientBase@246] - server 127.0.0.1:11233 not up java.net.ConnectException: Connection refused 2017-01-24 22:12:27,453 [myid:] - INFO [Thread-36:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2397521463065956250.junit.dir/zoo.cfg 2017-01-24 22:12:27,454 [myid:] - INFO [Thread-36:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:12:27,454 [myid:2] - INFO [Thread-36:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:12:27,454 [myid:2] - INFO [Thread-36:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:12:27,454 [myid:2] - INFO [Thread-36:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:12:27,455 [myid:2] - WARN [Thread-36:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:27,455 [myid:2] - INFO [Thread-36:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:12:27,456 [myid:2] - INFO [Thread-36:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11233 2017-01-24 22:12:27,456 [myid:2] - INFO [Thread-36:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:12:27,456 [myid:2] - INFO [Thread-36:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:12:27,457 [myid:2] - INFO [Thread-36:QuorumPeer@1279] - quorum.auth.enableSasl set to true 2017-01-24 22:12:27,457 [myid:2] - INFO [Thread-36:QuorumPeer@1264] - quorum.auth.serverRequireSasl set to false 2017-01-24 22:12:27,458 [myid:2] - INFO [Thread-36:QuorumPeer@1270] - quorum.auth.learnerRequireSasl set to false 2017-01-24 22:12:27,458 [myid:2] - INFO [Thread-36:QuorumPeer@1286] - quorum.auth.kerberos.servicePrincipal set to zkquorum/localhost 2017-01-24 22:12:27,458 [myid:2] - INFO [Thread-36:QuorumPeer@1298] - quorum.auth.server.saslLoginContext set to QuorumServer 2017-01-24 22:12:27,459 [myid:2] - INFO [Thread-36:QuorumPeer@1292] - quorum.auth.learner.saslLoginContext set to QuorumLearner 2017-01-24 22:12:27,459 [myid:2] - INFO [Thread-36:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:12:27,459 [myid:2] - INFO [Thread-36:Login@294] - QuorumServer successfully logged in. 2017-01-24 22:12:27,459 [myid:2] - INFO [Thread-36:Login@294] - QuorumLearner successfully logged in. 2017-01-24 22:12:27,460 [myid:2] - INFO [Thread-36:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:27,461 [myid:2] - INFO [Thread-36:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:27,479 [myid:2] - INFO [Thread-37:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11235 2017-01-24 22:12:27,481 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@781] - LOOKING 2017-01-24 22:12:27,481 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FastLeaderElection@744] - New election. My id = 2, proposed zxid=0x0 2017-01-24 22:12:27,482 [myid:0] - INFO [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:51850 2017-01-24 22:12:27,483 [myid:1] - INFO [localhost/127.0.0.1:11232:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:45198 2017-01-24 22:12:27,483 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,488 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-1:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:27,493 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-2:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:27,496 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,497 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LEADING (my state) 2017-01-24 22:12:27,501 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LEADING (n.state), 1 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,501 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 2 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,502 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 2 (n.sid), 0x1 (n.peerEPoch), LEADING (my state) 2017-01-24 22:12:27,502 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 0 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,502 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LEADING (n.state), 1 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,502 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state) 2017-01-24 22:12:27,502 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 2 (n.sid), 0x1 (n.peerEPoch), FOLLOWING (my state) 2017-01-24 22:12:27,503 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), FOLLOWING (n.state), 0 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,503 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), FOLLOWING (n.state), 0 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,703 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:27,704 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:27,704 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2397521463065956250.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2397521463065956250.junit.dir/data/version-2 2017-01-24 22:12:27,704 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45808 2017-01-24 22:12:27,704 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 223 2017-01-24 22:12:27,705 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45808 2017-01-24 22:12:27,705 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:27,706 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50178:LearnerHandler@287] - Follower sid: 2 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1b4476c1 2017-01-24 22:12:27,709 [myid:2] - INFO [Thread-38:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45808 (no session established for client) 2017-01-24 22:12:27,710 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50178:LearnerHandler@342] - Synchronizing with Follower sid: 2 maxCommittedLog=0x200000001 minCommittedLog=0x200000001 peerLastZxid=0x0 2017-01-24 22:12:27,710 [myid:1] - WARN [LearnerHandler-/127.0.0.1:50178:LearnerHandler@405] - Unhandled proposal scenario 2017-01-24 22:12:27,710 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50178:LearnerHandler@441] - Sending snapshot last zxid of peer is 0x0 zxid of leader is 0x200000001sent zxid of db as 0x200000001 2017-01-24 22:12:27,710 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Learner@329] - Getting a snapshot from leader 2017-01-24 22:12:27,712 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FileTxnSnapLog@281] - Snapshotting: 0x200000001 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2397521463065956250.junit.dir/data/version-2/snapshot.200000001 2017-01-24 22:12:27,713 [myid:1] - INFO [LearnerHandler-/127.0.0.1:50178:LearnerHandler@477] - Received NEWLEADER-ACK message from 2 2017-01-24 22:12:27,935 [myid:2] - INFO [WorkerSender[myid=2]:FastLeaderElection$Messenger$WorkerSender@370] - WorkerSender is down 2017-01-24 22:12:27,959 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:27,960 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45810 2017-01-24 22:12:27,960 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45810 2017-01-24 22:12:27,961 [myid:2] - INFO [Thread-40:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:12:27,962 [myid:2] - INFO [Thread-40:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45810 (no session established for client) 2017-01-24 22:12:27,964 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Follower@119] - Got zxid 0x200000002 expected 0x1 2017-01-24 22:12:27,964 [myid:2] - INFO [SyncThread:2:FileTxnLog@199] - Creating new log file: log.200000002 2017-01-24 22:12:27,967 [myid:] - INFO [Thread-12:QuorumAuthUpgradeTest@229] - Restarting server myid=0 2017-01-24 22:12:27,967 [myid:] - INFO [Thread-12:QuorumBase@314] - Shutting down quorum peer QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230 2017-01-24 22:12:27,967 [myid:] - INFO [Thread-12:Leader@491] - Shutting down 2017-01-24 22:12:27,967 [myid:] - INFO [Thread-12:Leader@497] - Shutdown called java.lang.Exception: shutdown Leader! reason: quorum Peer shutdown at org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:497) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:893) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:315) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$TestQPMain.shutdown(QuorumPeerTestBase.java:59) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.shutdown(QuorumPeerTestBase.java:152) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdown(QuorumAuthTestBase.java:138) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.restartServer(QuorumAuthUpgradeTest.java:230) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.testRollingUpgrade(QuorumAuthUpgradeTest.java:203) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52) at org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28) 2017-01-24 22:12:27,968 [myid:] - INFO [Thread-12:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:27,968 [myid:] - INFO [Thread-12:SessionTrackerImpl@225] - Shutting down 2017-01-24 22:12:27,968 [myid:] - INFO [Thread-12:PrepRequestProcessor@761] - Shutting down 2017-01-24 22:12:27,968 [myid:] - INFO [Thread-12:ProposalRequestProcessor@88] - Shutting down 2017-01-24 22:12:27,968 [myid:] - INFO [Thread-12:CommitProcessor@181] - Shutting down 2017-01-24 22:12:27,969 [myid:1] - INFO [CommitProcessor:1:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:27,969 [myid:] - INFO [Thread-12:Leader$ToBeAppliedRequestProcessor@656] - Shutting down 2017-01-24 22:12:27,969 [myid:] - INFO [Thread-12:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:27,969 [myid:] - INFO [Thread-12:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:27,968 [myid:1] - INFO [Thread-32:Leader$LearnerCnxAcceptor@318] - exception while shutting down acceptor: java.net.SocketException: Socket closed 2017-01-24 22:12:27,971 [myid:1] - INFO [ProcessThread(sid:1 cport:-1)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop! 2017-01-24 22:12:27,971 [myid:1] - INFO [SyncThread:1:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:27,972 [myid:1] - WARN [LearnerHandler-/127.0.0.1:50170:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:50170 ******** 2017-01-24 22:12:27,972 [myid:1] - WARN [LearnerHandler-/127.0.0.1:50170:LearnerHandler@610] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:608) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:601) 2017-01-24 22:12:27,973 [myid:1] - WARN [LearnerHandler-/127.0.0.1:50178:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:50178 ******** 2017-01-24 22:12:27,973 [myid:1] - WARN [LearnerHandler-/127.0.0.1:50178:LearnerHandler@610] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:608) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:601) 2017-01-24 22:12:27,973 [myid:1] - WARN [LearnerHandler-/127.0.0.1:50169:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:50169 ******** 2017-01-24 22:12:27,974 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@90] - Exception when following the leader java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:152) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:86) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:851) 2017-01-24 22:12:27,974 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:12:27,974 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method 2017-01-24 22:12:27,975 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Follower@90] - Exception when following the leader java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:152) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:86) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:851) 2017-01-24 22:12:27,975 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@867] - Unexpected exception java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:451) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:864) 2017-01-24 22:12:27,975 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Leader@491] - Shutting down 2017-01-24 22:12:27,975 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@879] - QuorumPeer main thread exited 2017-01-24 22:12:27,975 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:12:27,975 [myid:1] - ERROR [localhost/127.0.0.1:11232:QuorumCnxManager$Listener@715] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:696) 2017-01-24 22:12:27,976 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:27,977 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:27,977 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:12:27,978 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:CommitProcessor@181] - Shutting down 2017-01-24 22:12:27,978 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:27,977 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42834 which had sessionid 0x159d440f0ed0000 2017-01-24 22:12:27,977 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 1, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:27,979 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:27,977 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 0, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:27,977 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:27,980 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:27,979 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:27,980 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:27,980 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:12:27,980 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:CommitProcessor@181] - Shutting down 2017-01-24 22:12:27,980 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:27,980 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:27,979 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:27,979 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@980] - Connection broken for id 2, my id = 1, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:27,981 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:27,978 [myid:] - INFO [Thread-12:QuorumBase@318] - Shutting down leader election QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230 2017-01-24 22:12:27,978 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:27,981 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:27,978 [myid:2] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 2, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:27,982 [myid:2] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:27,978 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:27,978 [myid:2] - INFO [CommitProcessor:2:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:27,978 [myid:2] - INFO [FollowerRequestProcessor:2:FollowerRequestProcessor@95] - FollowerRequestProcessor exited loop! 2017-01-24 22:12:27,982 [myid:2] - INFO [SyncThread:2:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:27,982 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:27,986 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:27,981 [myid:] - INFO [Thread-12:QuorumBase@323] - Waiting for QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230 to exit thread 2017-01-24 22:12:27,980 [myid:0] - INFO [SyncThread:0:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:27,980 [myid:0] - INFO [CommitProcessor:0:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:27,980 [myid:0] - INFO [FollowerRequestProcessor:0:FollowerRequestProcessor@95] - FollowerRequestProcessor exited loop! 2017-01-24 22:12:27,979 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:27,987 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:27,986 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@781] - LOOKING 2017-01-24 22:12:27,985 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@781] - LOOKING 2017-01-24 22:12:27,988 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2397521463065956250.junit.dir/data/version-2/snapshot.200000001 2017-01-24 22:12:27,988 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2/snapshot.100000003 2017-01-24 22:12:27,990 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FastLeaderElection@744] - New election. My id = 2, proposed zxid=0x200000002 2017-01-24 22:12:27,990 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:27,991 [myid:] - INFO [Thread-12:QuorumPeerTestBase$MainThread@81] - id = 1 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir clientPort = 11230 2017-01-24 22:12:27,991 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 2 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,992 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FastLeaderElection@744] - New election. My id = 0, proposed zxid=0x200000002 2017-01-24 22:12:27,992 [myid:2] - WARN [WorkerSender[myid=2]:QuorumCnxManager@559] - Cannot open channel to 1 at election address localhost/127.0.0.1:11232 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:27,992 [myid:] - INFO [Thread-41:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir/zoo.cfg 2017-01-24 22:12:27,992 [myid:0] - WARN [WorkerSender[myid=0]:QuorumCnxManager@559] - Cannot open channel to 1 at election address localhost/127.0.0.1:11232 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:27,992 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,994 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,994 [myid:0] - WARN [WorkerSender[myid=0]:QuorumCnxManager@559] - Cannot open channel to 1 at election address localhost/127.0.0.1:11232 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:27,993 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 2 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,994 [myid:] - INFO [Thread-41:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:12:27,995 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11230 2017-01-24 22:12:27,995 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:27,995 [myid:1] - INFO [Thread-41:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:12:27,995 [myid:1] - INFO [Thread-41:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:12:27,995 [myid:1] - INFO [Thread-41:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:12:27,996 [myid:1] - WARN [Thread-41:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:28,001 [myid:1] - INFO [Thread-41:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:12:28,001 [myid:1] - INFO [Thread-41:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11230 2017-01-24 22:12:28,002 [myid:1] - INFO [Thread-41:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:12:28,002 [myid:1] - INFO [Thread-41:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:12:28,002 [myid:1] - INFO [Thread-41:QuorumPeer@1279] - quorum.auth.enableSasl set to true 2017-01-24 22:12:28,002 [myid:1] - INFO [Thread-41:QuorumPeer@1264] - quorum.auth.serverRequireSasl set to false 2017-01-24 22:12:28,002 [myid:1] - INFO [Thread-41:QuorumPeer@1270] - quorum.auth.learnerRequireSasl set to true 2017-01-24 22:12:28,002 [myid:1] - INFO [Thread-41:QuorumPeer@1286] - quorum.auth.kerberos.servicePrincipal set to zkquorum/localhost 2017-01-24 22:12:28,002 [myid:1] - INFO [Thread-41:QuorumPeer@1298] - quorum.auth.server.saslLoginContext set to QuorumServer 2017-01-24 22:12:28,003 [myid:1] - INFO [Thread-41:QuorumPeer@1292] - quorum.auth.learner.saslLoginContext set to QuorumLearner 2017-01-24 22:12:28,003 [myid:1] - INFO [Thread-41:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:12:28,003 [myid:1] - INFO [Thread-41:Login@294] - QuorumServer successfully logged in. 2017-01-24 22:12:28,003 [myid:1] - INFO [Thread-41:Login@294] - QuorumLearner successfully logged in. 2017-01-24 22:12:28,004 [myid:1] - INFO [Thread-41:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:27,995 [myid:] - INFO [Thread-12:ClientBase@246] - server 127.0.0.1:11230 not up java.net.ConnectException: Connection refused 2017-01-24 22:12:28,000 [myid:2] - INFO [SessionTracker:SessionTrackerImpl@162] - SessionTrackerImpl exited loop! 2017-01-24 22:12:28,000 [myid:1] - INFO [SessionTracker:SessionTrackerImpl@162] - SessionTrackerImpl exited loop! 2017-01-24 22:12:28,006 [myid:1] - INFO [Thread-41:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:28,012 [myid:1] - INFO [Thread-42:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11232 2017-01-24 22:12:28,013 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@781] - LOOKING 2017-01-24 22:12:28,013 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FastLeaderElection@744] - New election. My id = 1, proposed zxid=0x0 2017-01-24 22:12:28,014 [myid:0] - INFO [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:51859 2017-01-24 22:12:28,014 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,014 [myid:2] - INFO [localhost/127.0.0.1:11235:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:50365 2017-01-24 22:12:28,021 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-2:SecurityUtils@68] - QuorumLearner will use DIGEST-MD5 as SASL mechanism. 2017-01-24 22:12:28,021 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-1:SecurityUtils@68] - QuorumLearner will use DIGEST-MD5 as SASL mechanism. 2017-01-24 22:12:28,148 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:SaslQuorumServerCallbackHandler@143] - Successfully authenticated learner: authenticationID=test; authorizationID=test. 2017-01-24 22:12:28,148 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-3:SaslQuorumServerCallbackHandler@143] - Successfully authenticated learner: authenticationID=test; authorizationID=test. 2017-01-24 22:12:28,149 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:SaslQuorumAuthServer@114] - Successfully completed the authentication using SASL. learner addr: /127.0.0.1:51859 2017-01-24 22:12:28,149 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-3:SaslQuorumAuthServer@114] - Successfully completed the authentication using SASL. learner addr: /127.0.0.1:50365 2017-01-24 22:12:28,150 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-2:SaslQuorumAuthLearner@151] - Successfully completed the authentication using SASL. server addr: localhost/127.0.0.1:11235, status: SUCCESS 2017-01-24 22:12:28,150 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-2:QuorumCnxManager@331] - Have smaller server identifier, so dropping the connection: (2, 1) 2017-01-24 22:12:28,152 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-1:SaslQuorumAuthLearner@151] - Successfully completed the authentication using SASL. server addr: localhost/127.0.0.1:11229, status: SUCCESS 2017-01-24 22:12:28,157 [myid:1] - INFO [localhost/127.0.0.1:11232:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:45208 2017-01-24 22:12:28,157 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,158 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,158 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-2:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:28,159 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 1 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,160 [myid:2] - INFO [localhost/127.0.0.1:11235:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:50367 2017-01-24 22:12:28,160 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,160 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 1 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,160 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-1:SecurityUtils@68] - QuorumLearner will use DIGEST-MD5 as SASL mechanism. 2017-01-24 22:12:28,167 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 1 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,167 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 2 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,179 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@340] - WorkerReceiver is down 2017-01-24 22:12:28,180 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-2:SaslQuorumServerCallbackHandler@143] - Successfully authenticated learner: authenticationID=test; authorizationID=test. 2017-01-24 22:12:28,180 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-2:SaslQuorumAuthServer@114] - Successfully completed the authentication using SASL. learner addr: /127.0.0.1:50367 2017-01-24 22:12:28,181 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@980] - Connection broken for id 2, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:28,181 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:28,181 [myid:2] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 2, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:28,182 [myid:2] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:28,183 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-3:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:28,181 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:28,184 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:28,182 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:28,184 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:28,182 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-1:SaslQuorumAuthLearner@151] - Successfully completed the authentication using SASL. server addr: localhost/127.0.0.1:11235, status: SUCCESS 2017-01-24 22:12:28,182 [myid:1] - INFO [localhost/127.0.0.1:11232:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:45210 2017-01-24 22:12:28,184 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-1:QuorumCnxManager@331] - Have smaller server identifier, so dropping the connection: (2, 1) 2017-01-24 22:12:28,189 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 1 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,189 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 2 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,193 [myid:1] - ERROR [LearnerHandler-/127.0.0.1:57759:LearnerHandler@585] - Unexpected exception causing shutdown while sock still open java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:196) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:499) 2017-01-24 22:12:28,193 [myid:1] - WARN [LearnerHandler-/127.0.0.1:57759:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:57759 ******** 2017-01-24 22:12:28,212 [myid:2] - INFO [WorkerSender[myid=2]:FastLeaderElection$Messenger$WorkerSender@370] - WorkerSender is down 2017-01-24 22:12:28,215 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@340] - WorkerReceiver is down 2017-01-24 22:12:28,256 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11230 2017-01-24 22:12:28,257 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52351 2017-01-24 22:12:28,257 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:52351 2017-01-24 22:12:28,258 [myid:1] - INFO [Thread-43:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52351 (no session established for client) 2017-01-24 22:12:28,360 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:28,361 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2 2017-01-24 22:12:28,361 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 373 2017-01-24 22:12:28,363 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Learner@233] - Unexpected exception, tries=0, connecting to localhost/127.0.0.1:11234 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.Learner.connectToLeader(Learner.java:225) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:72) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:851) 2017-01-24 22:12:28,390 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@861] - LEADING 2017-01-24 22:12:28,390 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2397521463065956250.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2397521463065956250.junit.dir/data/version-2 2017-01-24 22:12:28,390 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Leader@356] - LEADING - LEADER ELECTION TOOK - 402 2017-01-24 22:12:28,390 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:28,390 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir/data/version-2 2017-01-24 22:12:28,390 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 377 2017-01-24 22:12:28,392 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:SecurityUtils@68] - QuorumLearner will use DIGEST-MD5 as SASL mechanism. 2017-01-24 22:12:28,392 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2397521463065956250.junit.dir/data/version-2/snapshot.200000001 2017-01-24 22:12:28,393 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FileTxnSnapLog@281] - Snapshotting: 0x200000002 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2397521463065956250.junit.dir/data/version-2/snapshot.200000002 2017-01-24 22:12:28,396 [myid:2] - INFO [Thread-44:SaslQuorumServerCallbackHandler@143] - Successfully authenticated learner: authenticationID=test; authorizationID=test. 2017-01-24 22:12:28,396 [myid:2] - INFO [Thread-44:SaslQuorumAuthServer@114] - Successfully completed the authentication using SASL. learner addr: /127.0.0.1:42592 2017-01-24 22:12:28,398 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:SaslQuorumAuthLearner@151] - Successfully completed the authentication using SASL. server addr: localhost/127.0.0.1:11234, status: SUCCESS 2017-01-24 22:12:28,398 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42592:LearnerHandler@287] - Follower sid: 1 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@b6804a5 2017-01-24 22:12:28,404 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42592:LearnerHandler@342] - Synchronizing with Follower sid: 1 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0 2017-01-24 22:12:28,404 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42592:LearnerHandler@441] - Sending snapshot last zxid of peer is 0x0 zxid of leader is 0x300000000sent zxid of db as 0x200000002 2017-01-24 22:12:28,405 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Learner@329] - Getting a snapshot from leader 2017-01-24 22:12:28,411 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FileTxnSnapLog@281] - Snapshotting: 0x200000002 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir/data/version-2/snapshot.200000002 2017-01-24 22:12:28,413 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42592:LearnerHandler@477] - Received NEWLEADER-ACK message from 1 2017-01-24 22:12:28,413 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Leader@934] - Have quorum of supporters, sids: [ 1,2 ]; starting up and setting last processed zxid: 0x300000000 2017-01-24 22:12:28,508 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11230 2017-01-24 22:12:28,509 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52354 2017-01-24 22:12:28,509 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:52354 2017-01-24 22:12:28,509 [myid:1] - INFO [Thread-46:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:12:28,510 [myid:1] - INFO [Thread-46:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52354 (no session established for client) 2017-01-24 22:12:28,648 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:Leader@491] - Shutting down 2017-01-24 22:12:28,648 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:Leader@497] - Shutdown called java.lang.Exception: shutdown Leader! reason: Not sufficient followers synced, only synced with sids: [ 1 ] at org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:497) at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:472) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:864) 2017-01-24 22:12:28,648 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:28,648 [myid:1] - INFO [Thread-8:Leader$LearnerCnxAcceptor@318] - exception while shutting down acceptor: java.net.SocketException: Socket closed 2017-01-24 22:12:28,648 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:SessionTrackerImpl@225] - Shutting down 2017-01-24 22:12:28,649 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:PrepRequestProcessor@761] - Shutting down 2017-01-24 22:12:28,649 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:ProposalRequestProcessor@88] - Shutting down 2017-01-24 22:12:28,649 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:CommitProcessor@181] - Shutting down 2017-01-24 22:12:28,649 [myid:1] - INFO [ProcessThread(sid:1 cport:-1)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop! 2017-01-24 22:12:28,649 [myid:1] - INFO [CommitProcessor:1:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:28,649 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:Leader$ToBeAppliedRequestProcessor@656] - Shutting down 2017-01-24 22:12:28,649 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:28,649 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:28,649 [myid:1] - INFO [SyncThread:1:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:28,650 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:QuorumPeer@781] - LOOKING 2017-01-24 22:12:28,650 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4158500655544466289.junit.dir/data/version-2/snapshot.0 2017-01-24 22:12:28,651 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FastLeaderElection@744] - New election. My id = 1, proposed zxid=0x100000003 2017-01-24 22:12:28,652 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11223 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:28,652 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x100000003 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x1 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,768 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:28,768 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:28,769 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52356 2017-01-24 22:12:28,769 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52356, server: localhost/127.0.0.1:11230 2017-01-24 22:12:28,769 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:ZooKeeperServer@832] - Client attempting to renew session 0x159d440f0ed0000 at /127.0.0.1:52356 2017-01-24 22:12:28,769 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:Learner@107] - Revalidating client: 0x159d440f0ed0000 2017-01-24 22:12:28,772 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:ZooKeeperServer@595] - Established session 0x159d440f0ed0000 with negotiated timeout 30000 for client /127.0.0.1:52356 2017-01-24 22:12:28,772 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1235] - Session establishment complete on server localhost/127.0.0.1:11230, sessionid = 0x159d440f0ed0000, negotiated timeout = 30000 2017-01-24 22:12:28,774 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@119] - Got zxid 0x300000001 expected 0x1 2017-01-24 22:12:28,775 [myid:1] - INFO [SyncThread:1:FileTxnLog@199] - Creating new log file: log.300000001 2017-01-24 22:12:28,777 [myid:] - INFO [Thread-12:QuorumAuthUpgradeTest@229] - Restarting server myid=1 2017-01-24 22:12:28,777 [myid:] - INFO [Thread-12:QuorumBase@314] - Shutting down quorum peer QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 2017-01-24 22:12:28,777 [myid:] - INFO [Thread-12:Leader@491] - Shutting down 2017-01-24 22:12:28,777 [myid:] - INFO [Thread-12:Leader@497] - Shutdown called java.lang.Exception: shutdown Leader! reason: quorum Peer shutdown at org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:497) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:893) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:315) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$TestQPMain.shutdown(QuorumPeerTestBase.java:59) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.shutdown(QuorumPeerTestBase.java:152) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdown(QuorumAuthTestBase.java:138) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.restartServer(QuorumAuthUpgradeTest.java:230) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.testRollingUpgrade(QuorumAuthUpgradeTest.java:204) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52) at org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28) 2017-01-24 22:12:28,778 [myid:] - INFO [Thread-12:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:28,778 [myid:2] - INFO [Thread-44:Leader$LearnerCnxAcceptor@318] - exception while shutting down acceptor: java.net.SocketException: Socket closed 2017-01-24 22:12:28,778 [myid:] - INFO [Thread-12:SessionTrackerImpl@225] - Shutting down 2017-01-24 22:12:28,778 [myid:] - INFO [Thread-12:PrepRequestProcessor@761] - Shutting down 2017-01-24 22:12:28,779 [myid:] - INFO [Thread-12:ProposalRequestProcessor@88] - Shutting down 2017-01-24 22:12:28,779 [myid:] - INFO [Thread-12:CommitProcessor@181] - Shutting down 2017-01-24 22:12:28,779 [myid:] - INFO [Thread-12:Leader$ToBeAppliedRequestProcessor@656] - Shutting down 2017-01-24 22:12:28,779 [myid:] - INFO [Thread-12:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:28,779 [myid:] - INFO [Thread-12:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:28,779 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop! 2017-01-24 22:12:28,779 [myid:2] - INFO [SyncThread:2:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:28,779 [myid:2] - INFO [CommitProcessor:2:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:28,782 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@90] - Exception when following the leader java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:152) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:86) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:851) 2017-01-24 22:12:28,784 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method 2017-01-24 22:12:28,782 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42592:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:42592 ******** 2017-01-24 22:12:28,784 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42592:LearnerHandler@610] - Ignoring unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338) at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:608) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:601) 2017-01-24 22:12:28,785 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:12:28,785 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52356 which had sessionid 0x159d440f0ed0000 2017-01-24 22:12:28,785 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:28,785 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:28,786 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:28,786 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:12:28,786 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:CommitProcessor@181] - Shutting down 2017-01-24 22:12:28,786 [myid:1] - INFO [FollowerRequestProcessor:1:FollowerRequestProcessor@95] - FollowerRequestProcessor exited loop! 2017-01-24 22:12:28,786 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@867] - Unexpected exception java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:451) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:864) 2017-01-24 22:12:28,786 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Leader@491] - Shutting down 2017-01-24 22:12:28,786 [myid:2] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:28,788 [myid:2] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:28,786 [myid:2] - ERROR [localhost/127.0.0.1:11235:QuorumCnxManager$Listener@715] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:696) 2017-01-24 22:12:28,788 [myid:] - INFO [Thread-12:QuorumBase@318] - Shutting down leader election QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 2017-01-24 22:12:28,788 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@980] - Connection broken for id 2, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:28,788 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:28,788 [myid:2] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 2, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:28,789 [myid:2] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:28,787 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@879] - QuorumPeer main thread exited 2017-01-24 22:12:28,789 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:28,787 [myid:2] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 2, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:28,790 [myid:2] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:28,790 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:28,787 [myid:1] - INFO [CommitProcessor:1:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:12:28,787 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:28,791 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:28,787 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:12:28,787 [myid:0] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@980] - Connection broken for id 2, my id = 0, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:28,791 [myid:0] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:28,791 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:12:28,788 [myid:] - INFO [Thread-12:QuorumBase@323] - Waiting for QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 to exit thread 2017-01-24 22:12:28,792 [myid:1] - INFO [SyncThread:1:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:12:28,792 [myid:0] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:28,792 [myid:0] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:28,793 [myid:] - INFO [Thread-12:QuorumPeerTestBase$MainThread@81] - id = 2 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6343054086158567637.junit.dir clientPort = 11233 2017-01-24 22:12:28,794 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@781] - LOOKING 2017-01-24 22:12:28,794 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir/data/version-2/snapshot.200000002 2017-01-24 22:12:28,795 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:28,795 [myid:] - INFO [Thread-12:ClientBase@246] - server 127.0.0.1:11233 not up java.net.ConnectException: Connection refused 2017-01-24 22:12:28,795 [myid:] - INFO [Thread-47:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6343054086158567637.junit.dir/zoo.cfg 2017-01-24 22:12:28,796 [myid:] - INFO [Thread-47:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:12:28,796 [myid:2] - INFO [Thread-47:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:12:28,796 [myid:2] - INFO [Thread-47:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:12:28,796 [myid:2] - INFO [Thread-47:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:12:28,797 [myid:2] - WARN [Thread-47:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:28,797 [myid:2] - INFO [Thread-47:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:12:28,797 [myid:2] - INFO [Thread-47:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11233 2017-01-24 22:12:28,798 [myid:2] - INFO [Thread-47:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:12:28,798 [myid:2] - INFO [Thread-47:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:12:28,798 [myid:2] - INFO [Thread-47:QuorumPeer@1279] - quorum.auth.enableSasl set to true 2017-01-24 22:12:28,798 [myid:2] - INFO [Thread-47:QuorumPeer@1264] - quorum.auth.serverRequireSasl set to false 2017-01-24 22:12:28,799 [myid:2] - INFO [Thread-47:QuorumPeer@1270] - quorum.auth.learnerRequireSasl set to true 2017-01-24 22:12:28,799 [myid:2] - INFO [Thread-47:QuorumPeer@1286] - quorum.auth.kerberos.servicePrincipal set to zkquorum/localhost 2017-01-24 22:12:28,799 [myid:2] - INFO [Thread-47:QuorumPeer@1298] - quorum.auth.server.saslLoginContext set to QuorumServer 2017-01-24 22:12:28,799 [myid:2] - INFO [Thread-47:QuorumPeer@1292] - quorum.auth.learner.saslLoginContext set to QuorumLearner 2017-01-24 22:12:28,799 [myid:2] - INFO [Thread-47:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:12:28,799 [myid:2] - INFO [Thread-47:Login@294] - QuorumServer successfully logged in. 2017-01-24 22:12:28,800 [myid:2] - INFO [Thread-47:Login@294] - QuorumLearner successfully logged in. 2017-01-24 22:12:28,800 [myid:2] - INFO [Thread-47:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:28,801 [myid:2] - INFO [Thread-47:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:12:28,802 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FastLeaderElection@744] - New election. My id = 1, proposed zxid=0x300000001 2017-01-24 22:12:28,802 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@559] - Cannot open channel to 2 at election address localhost/127.0.0.1:11235 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:12:28,803 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x300000001 (n.zxid), 0x4 (n.round), LOOKING (n.state), 1 (n.sid), 0x3 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,803 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x300000001 (n.zxid), 0x4 (n.round), LOOKING (n.state), 1 (n.sid), 0x3 (n.peerEPoch), FOLLOWING (my state) 2017-01-24 22:12:28,803 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), FOLLOWING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,804 [myid:2] - INFO [Thread-48:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11235 2017-01-24 22:12:28,812 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@781] - LOOKING 2017-01-24 22:12:28,812 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FastLeaderElection@744] - New election. My id = 2, proposed zxid=0x0 2017-01-24 22:12:28,813 [myid:0] - INFO [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:51872 2017-01-24 22:12:28,821 [myid:1] - INFO [localhost/127.0.0.1:11232:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:45220 2017-01-24 22:12:28,821 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,826 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-1:SecurityUtils@68] - QuorumLearner will use DIGEST-MD5 as SASL mechanism. 2017-01-24 22:12:28,828 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-2:SecurityUtils@68] - QuorumLearner will use DIGEST-MD5 as SASL mechanism. 2017-01-24 22:12:28,835 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:SaslQuorumServerCallbackHandler@143] - Successfully authenticated learner: authenticationID=test; authorizationID=test. 2017-01-24 22:12:28,836 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:SaslQuorumAuthServer@114] - Successfully completed the authentication using SASL. learner addr: /127.0.0.1:51872 2017-01-24 22:12:28,837 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-1:SaslQuorumAuthLearner@151] - Successfully completed the authentication using SASL. server addr: localhost/127.0.0.1:11229, status: SUCCESS 2017-01-24 22:12:28,838 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-1:SaslQuorumServerCallbackHandler@143] - Successfully authenticated learner: authenticationID=test; authorizationID=test. 2017-01-24 22:12:28,838 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-1:SaslQuorumAuthServer@114] - Successfully completed the authentication using SASL. learner addr: /127.0.0.1:45220 2017-01-24 22:12:28,839 [myid:2] - INFO [QuorumConnectionThread-[myid=2]-2:SaslQuorumAuthLearner@151] - Successfully completed the authentication using SASL. server addr: localhost/127.0.0.1:11232, status: SUCCESS 2017-01-24 22:12:28,844 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state) 2017-01-24 22:12:28,853 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11223 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:12:28,853 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FastLeaderElection@778] - Notification time out: 400 2017-01-24 22:12:28,854 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,854 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), FOLLOWING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,854 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 2 (n.sid), 0x2 (n.peerEPoch), FOLLOWING (my state) 2017-01-24 22:12:28,854 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 2 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,855 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x300000001 (n.zxid), 0x4 (n.round), LOOKING (n.state), 1 (n.sid), 0x3 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,855 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), FOLLOWING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,855 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@861] - LEADING 2017-01-24 22:12:28,855 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6343054086158567637.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6343054086158567637.junit.dir/data/version-2 2017-01-24 22:12:28,855 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Leader@356] - LEADING - LEADER ELECTION TOOK - 43 2017-01-24 22:12:28,856 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6343054086158567637.junit.dir/data/version-2/snapshot.0 2017-01-24 22:12:28,856 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,858 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LOOKING (n.state), 2 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,859 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x300000001 (n.zxid), 0x4 (n.round), LOOKING (n.state), 1 (n.sid), 0x3 (n.peerEPoch), LEADING (my state) 2017-01-24 22:12:28,859 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x300000001 (n.zxid), 0x4 (n.round), LOOKING (n.state), 1 (n.sid), 0x3 (n.peerEPoch), LEADING (my state) 2017-01-24 22:12:28,859 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LEADING (n.state), 2 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,859 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LEADING (n.state), 2 (n.sid), 0x2 (n.peerEPoch), FOLLOWING (my state) 2017-01-24 22:12:28,859 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:28,860 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir/data/version-2 2017-01-24 22:12:28,860 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 66 2017-01-24 22:12:28,860 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:SecurityUtils@68] - QuorumLearner will use DIGEST-MD5 as SASL mechanism. 2017-01-24 22:12:28,862 [myid:2] - INFO [Thread-49:SaslQuorumServerCallbackHandler@143] - Successfully authenticated learner: authenticationID=test; authorizationID=test. 2017-01-24 22:12:28,863 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:SaslQuorumAuthLearner@151] - Successfully completed the authentication using SASL. server addr: localhost/127.0.0.1:11234, status: SUCCESS 2017-01-24 22:12:28,864 [myid:2] - INFO [Thread-49:SaslQuorumAuthServer@114] - Successfully completed the authentication using SASL. learner addr: /127.0.0.1:42601 2017-01-24 22:12:28,864 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42601:LearnerHandler@287] - Follower sid: 1 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@2cdbf42e 2017-01-24 22:12:28,867 [myid:2] - ERROR [LearnerHandler-/127.0.0.1:42601:LearnerHandler@585] - Unexpected exception causing shutdown while sock still open java.io.IOException: Follower is ahead of the leader at org.apache.zookeeper.server.quorum.Leader.waitForEpochAck(Leader.java:889) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:322) 2017-01-24 22:12:28,867 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@90] - Exception when following the leader java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:152) at org.apache.zookeeper.server.quorum.Learner.syncWithLeader(Learner.java:321) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:83) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:851) 2017-01-24 22:12:28,868 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:12:28,868 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:28,868 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:28,868 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@781] - LOOKING 2017-01-24 22:12:28,869 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir/data/version-2/snapshot.200000002 2017-01-24 22:12:28,870 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FastLeaderElection@744] - New election. My id = 1, proposed zxid=0x300000001 2017-01-24 22:12:28,871 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x300000001 (n.zxid), 0x4 (n.round), LOOKING (n.state), 1 (n.sid), 0x3 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,871 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x300000001 (n.zxid), 0x4 (n.round), LOOKING (n.state), 1 (n.sid), 0x3 (n.peerEPoch), FOLLOWING (my state) 2017-01-24 22:12:28,871 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42601:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:42601 ******** 2017-01-24 22:12:28,872 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), FOLLOWING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,871 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x300000001 (n.zxid), 0x4 (n.round), LOOKING (n.state), 1 (n.sid), 0x3 (n.peerEPoch), LEADING (my state) 2017-01-24 22:12:28,872 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LEADING (n.state), 2 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:28,873 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:28,873 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir/data/version-2 2017-01-24 22:12:28,873 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 5 2017-01-24 22:12:28,874 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:SecurityUtils@68] - QuorumLearner will use DIGEST-MD5 as SASL mechanism. 2017-01-24 22:12:28,875 [myid:2] - INFO [Thread-49:SaslQuorumServerCallbackHandler@143] - Successfully authenticated learner: authenticationID=test; authorizationID=test. 2017-01-24 22:12:28,876 [myid:2] - INFO [Thread-49:SaslQuorumAuthServer@114] - Successfully completed the authentication using SASL. learner addr: /127.0.0.1:42602 2017-01-24 22:12:28,876 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:SaslQuorumAuthLearner@151] - Successfully completed the authentication using SASL. server addr: localhost/127.0.0.1:11234, status: SUCCESS 2017-01-24 22:12:28,877 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42602:LearnerHandler@287] - Follower sid: 1 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@2cdbf42e 2017-01-24 22:12:28,976 [myid:1] - INFO [localhost/127.0.0.1:11232:QuorumCnxManager$Listener@728] - Leaving listener 2017-01-24 22:12:29,045 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:29,046 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45833 2017-01-24 22:12:29,046 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45833 2017-01-24 22:12:29,047 [myid:2] - INFO [Thread-50:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45833 (no session established for client) 2017-01-24 22:12:29,184 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:29,184 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:29,185 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45834 2017-01-24 22:12:29,185 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45834, server: localhost/127.0.0.1:11233 2017-01-24 22:12:29,185 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:29,185 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45834 (no session established for client) 2017-01-24 22:12:29,185 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:29,254 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11223 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:12:29,254 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FastLeaderElection@778] - Notification time out: 800 2017-01-24 22:12:29,297 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:29,298 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45836 2017-01-24 22:12:29,298 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45836 2017-01-24 22:12:29,299 [myid:2] - INFO [Thread-51:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45836 (no session established for client) 2017-01-24 22:12:29,363 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:29,365 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42607:LearnerHandler@287] - Follower sid: 0 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@76415e70 2017-01-24 22:12:29,366 [myid:2] - ERROR [LearnerHandler-/127.0.0.1:42607:LearnerHandler@585] - Unexpected exception causing shutdown while sock still open java.io.IOException: Follower is ahead of the leader at org.apache.zookeeper.server.quorum.Leader.waitForEpochAck(Leader.java:889) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:322) 2017-01-24 22:12:29,366 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42607:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:42607 ******** 2017-01-24 22:12:29,366 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@90] - Exception when following the leader java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:152) at org.apache.zookeeper.server.quorum.Learner.syncWithLeader(Learner.java:321) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:83) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:851) 2017-01-24 22:12:29,367 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:12:29,367 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:29,367 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:29,367 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@781] - LOOKING 2017-01-24 22:12:29,368 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2/snapshot.100000003 2017-01-24 22:12:29,369 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FastLeaderElection@744] - New election. My id = 0, proposed zxid=0x200000002 2017-01-24 22:12:29,370 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x200000002 (n.zxid), 0x4 (n.round), LOOKING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:29,370 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x200000002 (n.zxid), 0x4 (n.round), LOOKING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), LEADING (my state) 2017-01-24 22:12:29,370 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x200000002 (n.zxid), 0x4 (n.round), LOOKING (n.state), 0 (n.sid), 0x2 (n.peerEPoch), FOLLOWING (my state) 2017-01-24 22:12:29,371 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), LEADING (n.state), 2 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:29,371 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 2 (n.leader), 0x200000002 (n.zxid), 0x3 (n.round), FOLLOWING (n.state), 1 (n.sid), 0x2 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:12:29,371 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@849] - FOLLOWING 2017-01-24 22:12:29,371 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6261823493417515862.junit.dir/data/version-2 2017-01-24 22:12:29,371 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 3 2017-01-24 22:12:29,372 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:12:29,372 [myid:2] - INFO [LearnerHandler-/127.0.0.1:42608:LearnerHandler@287] - Follower sid: 0 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@76415e70 2017-01-24 22:12:29,386 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:29,386 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:29,386 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42869 2017-01-24 22:12:29,386 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:42869, server: localhost/127.0.0.1:11227 2017-01-24 22:12:29,387 [myid:0] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:29,387 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42869 (no session established for client) 2017-01-24 22:12:29,387 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:29,549 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:29,550 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45840 2017-01-24 22:12:29,550 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45840 2017-01-24 22:12:29,551 [myid:2] - INFO [Thread-52:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45840 (no session established for client) 2017-01-24 22:12:29,788 [myid:2] - INFO [localhost/127.0.0.1:11235:QuorumCnxManager$Listener@728] - Leaving listener 2017-01-24 22:12:29,801 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:29,801 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45841 2017-01-24 22:12:29,802 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45841 2017-01-24 22:12:29,802 [myid:2] - INFO [Thread-53:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45841 (no session established for client) 2017-01-24 22:12:30,052 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:30,053 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45842 2017-01-24 22:12:30,053 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45842 2017-01-24 22:12:30,054 [myid:2] - INFO [Thread-54:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45842 (no session established for client) 2017-01-24 22:12:30,055 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11223 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:12:30,055 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FastLeaderElection@778] - Notification time out: 1600 2017-01-24 22:12:30,304 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:30,305 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45844 2017-01-24 22:12:30,305 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45844 2017-01-24 22:12:30,306 [myid:2] - INFO [Thread-55:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45844 (no session established for client) 2017-01-24 22:12:30,502 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@340] - WorkerReceiver is down 2017-01-24 22:12:30,502 [myid:1] - INFO [WorkerSender[myid=1]:FastLeaderElection$Messenger$WorkerSender@370] - WorkerSender is down 2017-01-24 22:12:30,556 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:30,557 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45845 2017-01-24 22:12:30,557 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45845 2017-01-24 22:12:30,557 [myid:2] - INFO [Thread-56:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45845 (no session established for client) 2017-01-24 22:12:30,808 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:30,808 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45846 2017-01-24 22:12:30,809 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45846 2017-01-24 22:12:30,809 [myid:2] - INFO [Thread-57:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45846 (no session established for client) 2017-01-24 22:12:30,993 [myid:2] - INFO [WorkerSender[myid=2]:FastLeaderElection$Messenger$WorkerSender@370] - WorkerSender is down 2017-01-24 22:12:31,059 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:31,060 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45847 2017-01-24 22:12:31,060 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45847 2017-01-24 22:12:31,061 [myid:2] - INFO [Thread-58:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45847 (no session established for client) 2017-01-24 22:12:31,189 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@340] - WorkerReceiver is down 2017-01-24 22:12:31,311 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:31,312 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45848 2017-01-24 22:12:31,312 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45848 2017-01-24 22:12:31,313 [myid:2] - INFO [Thread-59:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45848 (no session established for client) 2017-01-24 22:12:31,365 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:31,365 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:31,365 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52380 2017-01-24 22:12:31,365 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52380, server: localhost/127.0.0.1:11230 2017-01-24 22:12:31,366 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:31,366 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52380 (no session established for client) 2017-01-24 22:12:31,366 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:31,563 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:31,563 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45850 2017-01-24 22:12:31,564 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45850 2017-01-24 22:12:31,564 [myid:2] - INFO [Thread-60:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45850 (no session established for client) 2017-01-24 22:12:31,656 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11223 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:12:31,656 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FastLeaderElection@778] - Notification time out: 3200 2017-01-24 22:12:31,815 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:31,815 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45852 2017-01-24 22:12:31,815 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45852 2017-01-24 22:12:31,816 [myid:2] - INFO [Thread-61:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45852 (no session established for client) 2017-01-24 22:12:32,000 [myid:2] - INFO [SessionTracker:SessionTrackerImpl@162] - SessionTrackerImpl exited loop! 2017-01-24 22:12:32,000 [myid:1] - INFO [SessionTracker:SessionTrackerImpl@162] - SessionTrackerImpl exited loop! 2017-01-24 22:12:32,046 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:32,046 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:32,046 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45853 2017-01-24 22:12:32,046 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45853, server: localhost/127.0.0.1:11233 2017-01-24 22:12:32,047 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:32,047 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45853 (no session established for client) 2017-01-24 22:12:32,047 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:32,066 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:32,067 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45854 2017-01-24 22:12:32,067 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45854 2017-01-24 22:12:32,068 [myid:2] - INFO [Thread-62:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45854 (no session established for client) 2017-01-24 22:12:32,318 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:32,319 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45855 2017-01-24 22:12:32,319 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45855 2017-01-24 22:12:32,320 [myid:2] - INFO [Thread-63:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45855 (no session established for client) 2017-01-24 22:12:32,570 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:32,570 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45856 2017-01-24 22:12:32,571 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45856 2017-01-24 22:12:32,571 [myid:2] - INFO [Thread-64:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45856 (no session established for client) 2017-01-24 22:12:32,710 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:32,711 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:32,711 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42887 2017-01-24 22:12:32,711 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:42887, server: localhost/127.0.0.1:11227 2017-01-24 22:12:32,711 [myid:0] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:32,712 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42887 (no session established for client) 2017-01-24 22:12:32,712 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:32,822 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:32,822 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45858 2017-01-24 22:12:32,822 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45858 2017-01-24 22:12:32,823 [myid:2] - INFO [Thread-65:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45858 (no session established for client) 2017-01-24 22:12:33,073 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:33,074 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45859 2017-01-24 22:12:33,074 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45859 2017-01-24 22:12:33,075 [myid:2] - INFO [Thread-66:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45859 (no session established for client) 2017-01-24 22:12:33,325 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:33,326 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45860 2017-01-24 22:12:33,326 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45860 2017-01-24 22:12:33,326 [myid:2] - INFO [Thread-67:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45860 (no session established for client) 2017-01-24 22:12:33,577 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:33,577 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45861 2017-01-24 22:12:33,578 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45861 2017-01-24 22:12:33,578 [myid:2] - INFO [Thread-68:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45861 (no session established for client) 2017-01-24 22:12:33,828 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:33,829 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45862 2017-01-24 22:12:33,829 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45862 2017-01-24 22:12:33,830 [myid:2] - INFO [Thread-69:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45862 (no session established for client) 2017-01-24 22:12:34,080 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:34,081 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45863 2017-01-24 22:12:34,081 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45863 2017-01-24 22:12:34,082 [myid:2] - INFO [Thread-70:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45863 (no session established for client) 2017-01-24 22:12:34,332 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:34,332 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45864 2017-01-24 22:12:34,333 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45864 2017-01-24 22:12:34,333 [myid:2] - INFO [Thread-71:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45864 (no session established for client) 2017-01-24 22:12:34,421 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:34,421 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:34,421 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52396 2017-01-24 22:12:34,421 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52396, server: localhost/127.0.0.1:11230 2017-01-24 22:12:34,422 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:34,422 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52396 (no session established for client) 2017-01-24 22:12:34,422 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:34,584 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:34,584 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45866 2017-01-24 22:12:34,584 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45866 2017-01-24 22:12:34,585 [myid:2] - INFO [Thread-72:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45866 (no session established for client) 2017-01-24 22:12:34,835 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:34,836 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45867 2017-01-24 22:12:34,836 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45867 2017-01-24 22:12:34,837 [myid:2] - INFO [Thread-73:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45867 (no session established for client) 2017-01-24 22:12:34,857 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11223 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:12:34,857 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FastLeaderElection@778] - Notification time out: 6400 2017-01-24 22:12:35,087 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:35,088 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45869 2017-01-24 22:12:35,088 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45869 2017-01-24 22:12:35,088 [myid:2] - INFO [Thread-74:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45869 (no session established for client) 2017-01-24 22:12:35,339 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:35,339 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45870 2017-01-24 22:12:35,340 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45870 2017-01-24 22:12:35,340 [myid:2] - INFO [Thread-75:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45870 (no session established for client) 2017-01-24 22:12:35,424 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:35,425 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:35,425 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45871 2017-01-24 22:12:35,425 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45871, server: localhost/127.0.0.1:11233 2017-01-24 22:12:35,425 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:35,426 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45871 (no session established for client) 2017-01-24 22:12:35,426 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:35,590 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:35,591 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45872 2017-01-24 22:12:35,591 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45872 2017-01-24 22:12:35,592 [myid:2] - INFO [Thread-76:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45872 (no session established for client) 2017-01-24 22:12:35,842 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:35,843 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45873 2017-01-24 22:12:35,843 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45873 2017-01-24 22:12:35,844 [myid:2] - INFO [Thread-77:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45873 (no session established for client) 2017-01-24 22:12:36,094 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:36,095 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45874 2017-01-24 22:12:36,095 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45874 2017-01-24 22:12:36,095 [myid:2] - INFO [Thread-78:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45874 (no session established for client) 2017-01-24 22:12:36,205 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:36,206 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:36,206 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42905 2017-01-24 22:12:36,206 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:42905, server: localhost/127.0.0.1:11227 2017-01-24 22:12:36,207 [myid:0] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:36,207 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42905 (no session established for client) 2017-01-24 22:12:36,207 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:36,346 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:36,346 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45876 2017-01-24 22:12:36,347 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45876 2017-01-24 22:12:36,347 [myid:2] - INFO [Thread-79:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45876 (no session established for client) 2017-01-24 22:12:36,597 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:36,598 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45877 2017-01-24 22:12:36,598 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45877 2017-01-24 22:12:36,599 [myid:2] - INFO [Thread-80:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45877 (no session established for client) 2017-01-24 22:12:36,849 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:36,850 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45878 2017-01-24 22:12:36,850 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45878 2017-01-24 22:12:36,850 [myid:2] - INFO [Thread-81:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45878 (no session established for client) 2017-01-24 22:12:37,101 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:37,101 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45879 2017-01-24 22:12:37,102 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45879 2017-01-24 22:12:37,102 [myid:2] - INFO [Thread-82:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45879 (no session established for client) 2017-01-24 22:12:37,352 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:37,353 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45880 2017-01-24 22:12:37,353 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45880 2017-01-24 22:12:37,354 [myid:2] - INFO [Thread-83:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45880 (no session established for client) 2017-01-24 22:12:37,604 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:37,605 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45881 2017-01-24 22:12:37,605 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45881 2017-01-24 22:12:37,606 [myid:2] - INFO [Thread-84:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45881 (no session established for client) 2017-01-24 22:12:37,856 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:37,857 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45882 2017-01-24 22:12:37,857 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45882 2017-01-24 22:12:37,857 [myid:2] - INFO [Thread-85:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45882 (no session established for client) 2017-01-24 22:12:38,108 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:38,108 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45883 2017-01-24 22:12:38,109 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45883 2017-01-24 22:12:38,109 [myid:2] - INFO [Thread-86:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45883 (no session established for client) 2017-01-24 22:12:38,282 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:38,282 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:38,282 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52415 2017-01-24 22:12:38,282 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52415, server: localhost/127.0.0.1:11230 2017-01-24 22:12:38,283 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:38,283 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52415 (no session established for client) 2017-01-24 22:12:38,283 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:38,360 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:38,360 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45885 2017-01-24 22:12:38,360 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45885 2017-01-24 22:12:38,361 [myid:2] - INFO [Thread-87:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45885 (no session established for client) 2017-01-24 22:12:38,611 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:38,612 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45886 2017-01-24 22:12:38,612 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45886 2017-01-24 22:12:38,613 [myid:2] - INFO [Thread-88:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45886 (no session established for client) 2017-01-24 22:12:38,863 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:38,864 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45887 2017-01-24 22:12:38,864 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45887 2017-01-24 22:12:38,864 [myid:2] - INFO [Thread-89:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45887 (no session established for client) 2017-01-24 22:12:39,115 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:39,115 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45888 2017-01-24 22:12:39,116 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45888 2017-01-24 22:12:39,116 [myid:2] - INFO [Thread-90:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45888 (no session established for client) 2017-01-24 22:12:39,296 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:39,297 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:39,297 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45889 2017-01-24 22:12:39,297 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45889, server: localhost/127.0.0.1:11233 2017-01-24 22:12:39,298 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:39,298 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45889 (no session established for client) 2017-01-24 22:12:39,298 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:39,367 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:39,367 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45890 2017-01-24 22:12:39,367 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45890 2017-01-24 22:12:39,368 [myid:2] - INFO [Thread-91:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45890 (no session established for client) 2017-01-24 22:12:39,618 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:39,619 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45891 2017-01-24 22:12:39,619 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45891 2017-01-24 22:12:39,620 [myid:2] - INFO [Thread-92:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45891 (no session established for client) 2017-01-24 22:12:39,870 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:39,871 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45892 2017-01-24 22:12:39,871 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45892 2017-01-24 22:12:39,871 [myid:2] - INFO [Thread-93:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45892 (no session established for client) 2017-01-24 22:12:39,948 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:39,949 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:39,949 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42923 2017-01-24 22:12:39,949 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:42923, server: localhost/127.0.0.1:11227 2017-01-24 22:12:39,949 [myid:0] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:39,950 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42923 (no session established for client) 2017-01-24 22:12:39,950 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:40,122 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:40,122 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45894 2017-01-24 22:12:40,123 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45894 2017-01-24 22:12:40,123 [myid:2] - INFO [Thread-94:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45894 (no session established for client) 2017-01-24 22:12:40,374 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:40,375 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45895 2017-01-24 22:12:40,375 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45895 2017-01-24 22:12:40,375 [myid:2] - INFO [Thread-95:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45895 (no session established for client) 2017-01-24 22:12:40,626 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:40,626 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45896 2017-01-24 22:12:40,627 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45896 2017-01-24 22:12:40,627 [myid:2] - INFO [Thread-96:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45896 (no session established for client) 2017-01-24 22:12:40,877 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:40,878 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45897 2017-01-24 22:12:40,878 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45897 2017-01-24 22:12:40,879 [myid:2] - INFO [Thread-97:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45897 (no session established for client) 2017-01-24 22:12:41,129 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:41,130 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45898 2017-01-24 22:12:41,130 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45898 2017-01-24 22:12:41,131 [myid:2] - INFO [Thread-98:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45898 (no session established for client) 2017-01-24 22:12:41,205 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:41,206 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:41,206 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52430 2017-01-24 22:12:41,206 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52430, server: localhost/127.0.0.1:11230 2017-01-24 22:12:41,207 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:41,207 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52430 (no session established for client) 2017-01-24 22:12:41,207 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:41,257 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11223 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:12:41,258 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FastLeaderElection@778] - Notification time out: 12800 2017-01-24 22:12:41,381 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:41,381 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45901 2017-01-24 22:12:41,382 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45901 2017-01-24 22:12:41,382 [myid:2] - INFO [Thread-99:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45901 (no session established for client) 2017-01-24 22:12:41,633 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:41,633 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45902 2017-01-24 22:12:41,633 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45902 2017-01-24 22:12:41,634 [myid:2] - INFO [Thread-100:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45902 (no session established for client) 2017-01-24 22:12:41,741 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:41,742 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:41,742 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45903 2017-01-24 22:12:41,742 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45903, server: localhost/127.0.0.1:11233 2017-01-24 22:12:41,742 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:41,742 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45903 (no session established for client) 2017-01-24 22:12:41,743 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:41,884 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:41,885 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45904 2017-01-24 22:12:41,885 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45904 2017-01-24 22:12:41,886 [myid:2] - INFO [Thread-101:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45904 (no session established for client) 2017-01-24 22:12:42,136 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:42,137 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45905 2017-01-24 22:12:42,137 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45905 2017-01-24 22:12:42,137 [myid:2] - INFO [Thread-102:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45905 (no session established for client) 2017-01-24 22:12:42,388 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:42,388 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45906 2017-01-24 22:12:42,389 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45906 2017-01-24 22:12:42,389 [myid:2] - INFO [Thread-103:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45906 (no session established for client) 2017-01-24 22:12:42,564 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:42,565 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:42,565 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42937 2017-01-24 22:12:42,565 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:42937, server: localhost/127.0.0.1:11227 2017-01-24 22:12:42,565 [myid:0] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:42,566 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42937 (no session established for client) 2017-01-24 22:12:42,566 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:42,639 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:42,640 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45908 2017-01-24 22:12:42,640 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45908 2017-01-24 22:12:42,641 [myid:2] - INFO [Thread-104:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45908 (no session established for client) 2017-01-24 22:12:42,891 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:42,892 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45909 2017-01-24 22:12:42,892 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45909 2017-01-24 22:12:42,893 [myid:2] - INFO [Thread-105:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45909 (no session established for client) 2017-01-24 22:12:43,143 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:43,143 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45910 2017-01-24 22:12:43,144 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45910 2017-01-24 22:12:43,145 [myid:2] - INFO [Thread-106:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45910 (no session established for client) 2017-01-24 22:12:43,395 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:43,396 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45911 2017-01-24 22:12:43,396 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45911 2017-01-24 22:12:43,396 [myid:2] - INFO [Thread-107:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45911 (no session established for client) 2017-01-24 22:12:43,647 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:43,647 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45912 2017-01-24 22:12:43,647 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45912 2017-01-24 22:12:43,648 [myid:2] - INFO [Thread-108:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45912 (no session established for client) 2017-01-24 22:12:43,667 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:43,667 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:43,668 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52444 2017-01-24 22:12:43,668 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52444, server: localhost/127.0.0.1:11230 2017-01-24 22:12:43,668 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:43,668 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52444 (no session established for client) 2017-01-24 22:12:43,668 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:43,898 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:43,899 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45914 2017-01-24 22:12:43,899 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45914 2017-01-24 22:12:43,900 [myid:2] - INFO [Thread-109:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45914 (no session established for client) 2017-01-24 22:12:44,150 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:44,151 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45915 2017-01-24 22:12:44,151 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45915 2017-01-24 22:12:44,151 [myid:2] - INFO [Thread-110:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45915 (no session established for client) 2017-01-24 22:12:44,402 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:44,403 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45916 2017-01-24 22:12:44,403 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45916 2017-01-24 22:12:44,404 [myid:2] - INFO [Thread-111:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45916 (no session established for client) 2017-01-24 22:12:44,525 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:44,525 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:44,526 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45917 2017-01-24 22:12:44,526 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45917, server: localhost/127.0.0.1:11233 2017-01-24 22:12:44,526 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:44,526 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45917 (no session established for client) 2017-01-24 22:12:44,526 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:44,654 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:44,655 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45918 2017-01-24 22:12:44,655 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45918 2017-01-24 22:12:44,655 [myid:2] - INFO [Thread-112:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45918 (no session established for client) 2017-01-24 22:12:44,906 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:44,906 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45919 2017-01-24 22:12:44,907 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45919 2017-01-24 22:12:44,907 [myid:2] - INFO [Thread-113:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45919 (no session established for client) 2017-01-24 22:12:45,158 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:45,158 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45920 2017-01-24 22:12:45,158 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45920 2017-01-24 22:12:45,159 [myid:2] - INFO [Thread-114:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45920 (no session established for client) 2017-01-24 22:12:45,409 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:45,410 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45921 2017-01-24 22:12:45,410 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45921 2017-01-24 22:12:45,411 [myid:2] - INFO [Thread-115:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45921 (no session established for client) 2017-01-24 22:12:45,556 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:45,556 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:45,557 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42952 2017-01-24 22:12:45,557 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:42952, server: localhost/127.0.0.1:11227 2017-01-24 22:12:45,557 [myid:0] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:45,557 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42952 (no session established for client) 2017-01-24 22:12:45,560 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:45,661 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:45,661 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45923 2017-01-24 22:12:45,662 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45923 2017-01-24 22:12:45,662 [myid:2] - INFO [Thread-116:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45923 (no session established for client) 2017-01-24 22:12:45,913 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:45,913 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45924 2017-01-24 22:12:45,913 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45924 2017-01-24 22:12:45,914 [myid:2] - INFO [Thread-117:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45924 (no session established for client) 2017-01-24 22:12:46,164 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:46,165 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45925 2017-01-24 22:12:46,165 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45925 2017-01-24 22:12:46,166 [myid:2] - INFO [Thread-118:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45925 (no session established for client) 2017-01-24 22:12:46,416 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:46,417 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45926 2017-01-24 22:12:46,417 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45926 2017-01-24 22:12:46,417 [myid:2] - INFO [Thread-119:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45926 (no session established for client) 2017-01-24 22:12:46,668 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:46,668 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45927 2017-01-24 22:12:46,669 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45927 2017-01-24 22:12:46,669 [myid:2] - INFO [Thread-120:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45927 (no session established for client) 2017-01-24 22:12:46,919 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:46,920 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45928 2017-01-24 22:12:46,920 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45928 2017-01-24 22:12:46,921 [myid:2] - INFO [Thread-121:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45928 (no session established for client) 2017-01-24 22:12:46,935 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:46,935 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:46,935 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52460 2017-01-24 22:12:46,935 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52460, server: localhost/127.0.0.1:11230 2017-01-24 22:12:46,936 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:46,936 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52460 (no session established for client) 2017-01-24 22:12:46,936 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:47,171 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:47,172 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45930 2017-01-24 22:12:47,172 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45930 2017-01-24 22:12:47,172 [myid:2] - INFO [Thread-122:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45930 (no session established for client) 2017-01-24 22:12:47,423 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:47,423 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45931 2017-01-24 22:12:47,424 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45931 2017-01-24 22:12:47,424 [myid:2] - INFO [Thread-123:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45931 (no session established for client) 2017-01-24 22:12:47,470 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:47,470 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:47,470 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45932 2017-01-24 22:12:47,470 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45932, server: localhost/127.0.0.1:11233 2017-01-24 22:12:47,477 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:47,477 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45932 (no session established for client) 2017-01-24 22:12:47,477 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:47,675 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:47,675 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45933 2017-01-24 22:12:47,675 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45933 2017-01-24 22:12:47,676 [myid:2] - INFO [Thread-124:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45933 (no session established for client) 2017-01-24 22:12:47,926 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:47,927 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45934 2017-01-24 22:12:47,927 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45934 2017-01-24 22:12:47,928 [myid:2] - INFO [Thread-125:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45934 (no session established for client) 2017-01-24 22:12:48,178 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:48,179 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45935 2017-01-24 22:12:48,179 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45935 2017-01-24 22:12:48,179 [myid:2] - INFO [Thread-126:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45935 (no session established for client) 2017-01-24 22:12:48,430 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:48,430 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45936 2017-01-24 22:12:48,430 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45936 2017-01-24 22:12:48,431 [myid:2] - INFO [Thread-127:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45936 (no session established for client) 2017-01-24 22:12:48,469 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:48,469 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:48,469 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42967 2017-01-24 22:12:48,469 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:42967, server: localhost/127.0.0.1:11227 2017-01-24 22:12:48,470 [myid:0] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:48,470 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42967 (no session established for client) 2017-01-24 22:12:48,470 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:48,681 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:48,682 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45938 2017-01-24 22:12:48,682 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45938 2017-01-24 22:12:48,683 [myid:2] - INFO [Thread-128:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45938 (no session established for client) 2017-01-24 22:12:48,933 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:48,934 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45939 2017-01-24 22:12:48,934 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45939 2017-01-24 22:12:48,935 [myid:2] - INFO [Thread-129:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45939 (no session established for client) 2017-01-24 22:12:49,185 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:49,186 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45940 2017-01-24 22:12:49,186 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45940 2017-01-24 22:12:49,186 [myid:2] - INFO [Thread-130:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45940 (no session established for client) 2017-01-24 22:12:49,437 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:49,437 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45941 2017-01-24 22:12:49,437 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45941 2017-01-24 22:12:49,438 [myid:2] - INFO [Thread-131:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45941 (no session established for client) 2017-01-24 22:12:49,688 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:49,689 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45942 2017-01-24 22:12:49,689 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45942 2017-01-24 22:12:49,690 [myid:2] - INFO [Thread-132:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45942 (no session established for client) 2017-01-24 22:12:49,940 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:49,940 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45943 2017-01-24 22:12:49,941 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45943 2017-01-24 22:12:49,941 [myid:2] - INFO [Thread-133:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45943 (no session established for client) 2017-01-24 22:12:50,192 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:50,192 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45944 2017-01-24 22:12:50,192 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45944 2017-01-24 22:12:50,193 [myid:2] - INFO [Thread-134:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45944 (no session established for client) 2017-01-24 22:12:50,213 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:50,213 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:50,213 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52476 2017-01-24 22:12:50,213 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52476, server: localhost/127.0.0.1:11230 2017-01-24 22:12:50,214 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:50,214 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52476 (no session established for client) 2017-01-24 22:12:50,214 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:50,314 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:50,314 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:50,315 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45946 2017-01-24 22:12:50,315 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45946, server: localhost/127.0.0.1:11233 2017-01-24 22:12:50,315 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:50,315 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45946 (no session established for client) 2017-01-24 22:12:50,316 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:50,443 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:50,444 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45947 2017-01-24 22:12:50,444 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45947 2017-01-24 22:12:50,445 [myid:2] - INFO [Thread-135:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45947 (no session established for client) 2017-01-24 22:12:50,695 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:50,696 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45948 2017-01-24 22:12:50,696 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45948 2017-01-24 22:12:50,696 [myid:2] - INFO [Thread-136:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45948 (no session established for client) 2017-01-24 22:12:50,947 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:50,947 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45949 2017-01-24 22:12:50,947 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45949 2017-01-24 22:12:50,948 [myid:2] - INFO [Thread-137:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45949 (no session established for client) 2017-01-24 22:12:50,968 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:50,968 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:50,969 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42980 2017-01-24 22:12:50,969 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:42980, server: localhost/127.0.0.1:11227 2017-01-24 22:12:50,969 [myid:0] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:50,969 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42980 (no session established for client) 2017-01-24 22:12:50,970 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:51,198 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:51,199 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45951 2017-01-24 22:12:51,199 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45951 2017-01-24 22:12:51,200 [myid:2] - INFO [Thread-138:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45951 (no session established for client) 2017-01-24 22:12:51,450 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:51,451 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45952 2017-01-24 22:12:51,451 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45952 2017-01-24 22:12:51,452 [myid:2] - INFO [Thread-139:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45952 (no session established for client) 2017-01-24 22:12:51,702 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:51,703 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45953 2017-01-24 22:12:51,703 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45953 2017-01-24 22:12:51,704 [myid:2] - INFO [Thread-140:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45953 (no session established for client) 2017-01-24 22:12:51,954 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:51,955 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45954 2017-01-24 22:12:51,955 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45954 2017-01-24 22:12:51,955 [myid:2] - INFO [Thread-141:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45954 (no session established for client) 2017-01-24 22:12:52,206 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:52,206 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45955 2017-01-24 22:12:52,207 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45955 2017-01-24 22:12:52,207 [myid:2] - INFO [Thread-142:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45955 (no session established for client) 2017-01-24 22:12:52,457 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:52,458 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45956 2017-01-24 22:12:52,459 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45956 2017-01-24 22:12:52,459 [myid:2] - INFO [Thread-143:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45956 (no session established for client) 2017-01-24 22:12:52,477 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:52,478 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:52,478 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52488 2017-01-24 22:12:52,478 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52488, server: localhost/127.0.0.1:11230 2017-01-24 22:12:52,478 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:52,478 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52488 (no session established for client) 2017-01-24 22:12:52,479 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:52,710 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:52,710 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45958 2017-01-24 22:12:52,711 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45958 2017-01-24 22:12:52,711 [myid:2] - INFO [Thread-144:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45958 (no session established for client) 2017-01-24 22:12:52,948 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:52,948 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:52,949 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45959 2017-01-24 22:12:52,949 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45959, server: localhost/127.0.0.1:11233 2017-01-24 22:12:52,949 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:52,949 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45959 (no session established for client) 2017-01-24 22:12:52,949 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:52,961 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:52,962 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45960 2017-01-24 22:12:52,962 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45960 2017-01-24 22:12:52,962 [myid:2] - INFO [Thread-145:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45960 (no session established for client) 2017-01-24 22:12:53,213 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:53,213 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45961 2017-01-24 22:12:53,214 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45961 2017-01-24 22:12:53,214 [myid:2] - INFO [Thread-146:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45961 (no session established for client) 2017-01-24 22:12:53,464 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:53,465 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45962 2017-01-24 22:12:53,465 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45962 2017-01-24 22:12:53,466 [myid:2] - INFO [Thread-147:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45962 (no session established for client) 2017-01-24 22:12:53,716 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:53,717 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45963 2017-01-24 22:12:53,717 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45963 2017-01-24 22:12:53,718 [myid:2] - INFO [Thread-148:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45963 (no session established for client) 2017-01-24 22:12:53,878 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:53,878 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:53,879 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:42994 2017-01-24 22:12:53,879 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:42994, server: localhost/127.0.0.1:11227 2017-01-24 22:12:53,879 [myid:0] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:53,879 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:42994 (no session established for client) 2017-01-24 22:12:53,879 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:53,968 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:53,968 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45965 2017-01-24 22:12:53,969 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45965 2017-01-24 22:12:53,969 [myid:2] - INFO [Thread-149:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45965 (no session established for client) 2017-01-24 22:12:54,058 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11223 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:12:54,059 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11224:FastLeaderElection@778] - Notification time out: 25600 2017-01-24 22:12:54,220 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:54,220 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45967 2017-01-24 22:12:54,220 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45967 2017-01-24 22:12:54,221 [myid:2] - INFO [Thread-150:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45967 (no session established for client) 2017-01-24 22:12:54,471 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:54,472 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45968 2017-01-24 22:12:54,472 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45968 2017-01-24 22:12:54,473 [myid:2] - INFO [Thread-151:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45968 (no session established for client) 2017-01-24 22:12:54,723 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:54,723 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45969 2017-01-24 22:12:54,724 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45969 2017-01-24 22:12:54,724 [myid:2] - INFO [Thread-152:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45969 (no session established for client) 2017-01-24 22:12:54,975 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:54,975 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45970 2017-01-24 22:12:54,975 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45970 2017-01-24 22:12:54,976 [myid:2] - INFO [Thread-153:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45970 (no session established for client) 2017-01-24 22:12:55,110 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:55,110 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:55,111 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52502 2017-01-24 22:12:55,111 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52502, server: localhost/127.0.0.1:11230 2017-01-24 22:12:55,111 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:55,111 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52502 (no session established for client) 2017-01-24 22:12:55,111 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:55,226 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:55,227 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45972 2017-01-24 22:12:55,227 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45972 2017-01-24 22:12:55,229 [myid:2] - INFO [Thread-154:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45972 (no session established for client) 2017-01-24 22:12:55,479 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:55,480 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45973 2017-01-24 22:12:55,480 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45973 2017-01-24 22:12:55,481 [myid:2] - INFO [Thread-155:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45973 (no session established for client) 2017-01-24 22:12:55,718 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:55,718 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:55,718 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45974 2017-01-24 22:12:55,719 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45974, server: localhost/127.0.0.1:11233 2017-01-24 22:12:55,719 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:55,719 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45974 (no session established for client) 2017-01-24 22:12:55,719 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:55,731 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:55,731 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45975 2017-01-24 22:12:55,731 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45975 2017-01-24 22:12:55,732 [myid:2] - INFO [Thread-156:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45975 (no session established for client) 2017-01-24 22:12:55,982 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:55,983 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45976 2017-01-24 22:12:55,983 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45976 2017-01-24 22:12:55,984 [myid:2] - INFO [Thread-157:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45976 (no session established for client) 2017-01-24 22:12:56,115 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:56,115 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:56,115 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:43007 2017-01-24 22:12:56,115 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:43007, server: localhost/127.0.0.1:11227 2017-01-24 22:12:56,116 [myid:0] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:56,116 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:43007 (no session established for client) 2017-01-24 22:12:56,116 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:56,234 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:56,235 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45978 2017-01-24 22:12:56,235 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45978 2017-01-24 22:12:56,236 [myid:2] - INFO [Thread-158:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45978 (no session established for client) 2017-01-24 22:12:56,486 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:56,487 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45979 2017-01-24 22:12:56,487 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45979 2017-01-24 22:12:56,488 [myid:2] - INFO [Thread-159:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45979 (no session established for client) 2017-01-24 22:12:56,738 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:56,738 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45980 2017-01-24 22:12:56,739 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45980 2017-01-24 22:12:56,739 [myid:2] - INFO [Thread-160:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45980 (no session established for client) 2017-01-24 22:12:56,989 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:56,990 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45981 2017-01-24 22:12:56,990 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45981 2017-01-24 22:12:56,991 [myid:2] - INFO [Thread-161:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45981 (no session established for client) 2017-01-24 22:12:57,241 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:57,242 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45982 2017-01-24 22:12:57,242 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45982 2017-01-24 22:12:57,242 [myid:2] - INFO [Thread-162:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45982 (no session established for client) 2017-01-24 22:12:57,493 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:57,493 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45983 2017-01-24 22:12:57,494 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45983 2017-01-24 22:12:57,494 [myid:2] - INFO [Thread-163:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45983 (no session established for client) 2017-01-24 22:12:57,744 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:57,745 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45984 2017-01-24 22:12:57,745 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45984 2017-01-24 22:12:57,746 [myid:2] - INFO [Thread-164:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45984 (no session established for client) 2017-01-24 22:12:57,885 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:57,885 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:12:57,885 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52516 2017-01-24 22:12:57,885 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52516, server: localhost/127.0.0.1:11230 2017-01-24 22:12:57,886 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:57,886 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52516 (no session established for client) 2017-01-24 22:12:57,886 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:57,996 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:57,997 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45986 2017-01-24 22:12:57,997 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45986 2017-01-24 22:12:57,997 [myid:2] - INFO [Thread-165:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45986 (no session established for client) 2017-01-24 22:12:58,248 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:58,248 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45987 2017-01-24 22:12:58,249 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45987 2017-01-24 22:12:58,249 [myid:2] - INFO [Thread-166:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45987 (no session established for client) 2017-01-24 22:12:58,499 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:58,500 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45988 2017-01-24 22:12:58,500 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45988 2017-01-24 22:12:58,501 [myid:2] - INFO [Thread-167:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45988 (no session established for client) 2017-01-24 22:12:58,751 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:58,752 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45989 2017-01-24 22:12:58,752 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45989 2017-01-24 22:12:58,752 [myid:2] - INFO [Thread-168:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45989 (no session established for client) 2017-01-24 22:12:58,983 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:58,983 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:12:58,983 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45990 2017-01-24 22:12:58,983 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45990, server: localhost/127.0.0.1:11233 2017-01-24 22:12:58,984 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:12:58,984 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45990 (no session established for client) 2017-01-24 22:12:58,984 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:12:59,003 [myid:] - INFO [Thread-12:FourLetterWordMain@43] - connecting to 127.0.0.1 11233 2017-01-24 22:12:59,003 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45991 2017-01-24 22:12:59,003 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:45991 2017-01-24 22:12:59,004 [myid:2] - INFO [Thread-169:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45991 (no session established for client) 2017-01-24 22:12:59,004 [myid:] - INFO [Thread-12:JUnit4ZKTestRunner$LoggedInvokeMethod@62] - TEST METHOD FAILED testRollingUpgrade java.lang.AssertionError: waiting for server1being up at org.junit.Assert.fail(Assert.java:91) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.restartServer(QuorumAuthUpgradeTest.java:232) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.testRollingUpgrade(QuorumAuthUpgradeTest.java:204) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52) at org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28) 2017-01-24 22:12:59,005 [myid:] - INFO [main:QuorumBase@314] - Shutting down quorum peer QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227 2017-01-24 22:12:59,005 [myid:] - INFO [main:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:896) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:315) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$TestQPMain.shutdown(QuorumPeerTestBase.java:59) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.shutdown(QuorumPeerTestBase.java:152) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdown(QuorumAuthTestBase.java:138) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdownAll(QuorumAuthTestBase.java:131) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.tearDown(QuorumAuthUpgradeTest.java:68) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033) 2017-01-24 22:12:59,006 [myid:] - INFO [main:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:12:59,006 [myid:] - INFO [main:ZooKeeperServer@419] - shutting down 2017-01-24 22:12:59,006 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11227:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method 2017-01-24 22:12:59,007 [myid:0] - ERROR [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@715] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:696) 2017-01-24 22:12:59,007 [myid:0] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:59,007 [myid:0] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@980] - Connection broken for id 2, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:59,008 [myid:0] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:59,008 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:59,007 [myid:2] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 2, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:59,008 [myid:2] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:59,008 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:59,009 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:59,008 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:12:59,009 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:12:59,008 [myid:0] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:59,010 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:59,009 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:59,010 [myid:2] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:12:59,011 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:59,011 [myid:2] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:12:59,011 [myid:] - INFO [main:QuorumBase@318] - Shutting down leader election QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227 2017-01-24 22:12:59,011 [myid:] - INFO [main:QuorumBase@323] - Waiting for QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227 to exit thread 2017-01-24 22:12:59,371 [myid:0] - INFO [WorkerSender[myid=0]:FastLeaderElection$Messenger$WorkerSender@370] - WorkerSender is down 2017-01-24 22:12:59,372 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection$Messenger$WorkerReceiver@340] - WorkerReceiver is down 2017-01-24 22:12:59,885 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:12:59,886 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:12:59,886 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1102] - Session 0x159d440f0ed0000 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2017-01-24 22:13:00,007 [myid:0] - INFO [localhost/127.0.0.1:11229:QuorumCnxManager$Listener@728] - Leaving listener 2017-01-24 22:13:01,546 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:01,546 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:13:01,547 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52524 2017-01-24 22:13:01,547 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52524, server: localhost/127.0.0.1:11230 2017-01-24 22:13:01,547 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:13:01,547 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52524 (no session established for client) 2017-01-24 22:13:01,547 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:13:01,960 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:01,960 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:13:01,961 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45994 2017-01-24 22:13:01,961 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45994, server: localhost/127.0.0.1:11233 2017-01-24 22:13:01,961 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:13:01,961 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45994 (no session established for client) 2017-01-24 22:13:01,961 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:13:03,003 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:03,003 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:13:03,004 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1102] - Session 0x159d440f0ed0000 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2017-01-24 22:13:04,254 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:04,254 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:13:04,254 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52527 2017-01-24 22:13:04,254 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52527, server: localhost/127.0.0.1:11230 2017-01-24 22:13:04,255 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:13:04,255 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52527 (no session established for client) 2017-01-24 22:13:04,255 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:13:04,947 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:04,947 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:13:04,947 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:45997 2017-01-24 22:13:04,947 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:45997, server: localhost/127.0.0.1:11233 2017-01-24 22:13:04,948 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:13:04,948 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45997 (no session established for client) 2017-01-24 22:13:04,948 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:13:05,888 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:05,889 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:13:05,889 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1102] - Session 0x159d440f0ed0000 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2017-01-24 22:13:07,549 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:07,549 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:13:07,550 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52530 2017-01-24 22:13:07,550 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52530, server: localhost/127.0.0.1:11230 2017-01-24 22:13:07,550 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:13:07,550 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52530 (no session established for client) 2017-01-24 22:13:07,550 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:13:07,986 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:07,986 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:13:07,987 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:46000 2017-01-24 22:13:07,987 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:46000, server: localhost/127.0.0.1:11233 2017-01-24 22:13:07,987 [myid:2] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:13:07,987 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:46000 (no session established for client) 2017-01-24 22:13:07,987 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:13:08,583 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:08,583 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:13:08,583 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1102] - Session 0x159d440f0ed0000 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2017-01-24 22:13:08,866 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@867] - Unexpected exception java.lang.InterruptedException: Timeout while waiting for epoch to be acked by quorum at org.apache.zookeeper.server.quorum.Leader.waitForEpochAck(Leader.java:906) at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:392) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:864) 2017-01-24 22:13:08,866 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Leader@491] - Shutting down 2017-01-24 22:13:08,866 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:Leader@497] - Shutdown called java.lang.Exception: shutdown Leader! reason: Forcing shutdown at org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:497) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:870) 2017-01-24 22:13:08,867 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:ZooKeeperServer@419] - shutting down 2017-01-24 22:13:08,867 [myid:2] - INFO [Thread-49:Leader$LearnerCnxAcceptor@318] - exception while shutting down acceptor: java.net.SocketException: Socket closed 2017-01-24 22:13:08,867 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@90] - Exception when following the leader java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:152) at org.apache.zookeeper.server.quorum.Learner.syncWithLeader(Learner.java:321) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:83) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:851) 2017-01-24 22:13:08,867 [myid:2] - ERROR [LearnerHandler-/127.0.0.1:42608:LearnerHandler@596] - Unexpected exception causing shutdown java.lang.InterruptedException at java.lang.Object.wait(Native Method) at org.apache.zookeeper.server.quorum.Leader.waitForEpochAck(Leader.java:902) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:322) 2017-01-24 22:13:08,868 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:13:08,868 [myid:2] - ERROR [LearnerHandler-/127.0.0.1:42602:LearnerHandler@596] - Unexpected exception causing shutdown java.lang.InterruptedException at java.lang.Object.wait(Native Method) at org.apache.zookeeper.server.quorum.Leader.waitForEpochAck(Leader.java:902) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:322) 2017-01-24 22:13:08,868 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@90] - Exception when following the leader java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:152) at org.apache.zookeeper.server.quorum.Learner.syncWithLeader(Learner.java:321) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:83) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:851) 2017-01-24 22:13:08,868 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@781] - LOOKING 2017-01-24 22:13:08,869 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:13:08,869 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:13:08,869 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:ZooKeeperServer@419] - shutting down 2017-01-24 22:13:08,870 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumPeer@781] - LOOKING 2017-01-24 22:13:08,868 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42602:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:42602 ******** 2017-01-24 22:13:08,870 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4772652757083079773.junit.dir/data/version-2/snapshot.200000002 2017-01-24 22:13:08,871 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FastLeaderElection@744] - New election. My id = 1, proposed zxid=0x300000001 2017-01-24 22:13:08,868 [myid:2] - WARN [LearnerHandler-/127.0.0.1:42608:LearnerHandler@598] - ******* GOODBYE /127.0.0.1:42608 ******** 2017-01-24 22:13:08,868 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:13:08,872 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11229 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:13:08,872 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:ZooKeeperServer@419] - shutting down 2017-01-24 22:13:08,872 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11227:QuorumPeer@879] - QuorumPeer main thread exited 2017-01-24 22:13:08,873 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x300000001 (n.zxid), 0x4 (n.round), LOOKING (n.state), 1 (n.sid), 0x3 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:08,870 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FileSnap@83] - Reading snapshot /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6343054086158567637.junit.dir/data/version-2/snapshot.0 2017-01-24 22:13:08,874 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x300000001 (n.zxid), 0x4 (n.round), LOOKING (n.state), 1 (n.sid), 0x3 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:08,875 [myid:] - INFO [main:QuorumBase@314] - Shutting down quorum peer QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 2017-01-24 22:13:08,875 [myid:2] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11233:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method 2017-01-24 22:13:08,875 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:FastLeaderElection@744] - New election. My id = 2, proposed zxid=0x0 2017-01-24 22:13:08,876 [myid:2] - WARN [WorkerSender[myid=2]:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11229 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:13:08,877 [myid:2] - INFO [WorkerSender[myid=2]:FastLeaderElection$Messenger$WorkerSender@370] - WorkerSender is down 2017-01-24 22:13:08,876 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:13:08,877 [myid:2] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:13:08,877 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@829] - Unexpected exception java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:758) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:13:08,878 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233:QuorumPeer@879] - QuorumPeer main thread exited 2017-01-24 22:13:08,877 [myid:] - INFO [main:QuorumBase@318] - Shutting down leader election QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 2017-01-24 22:13:08,878 [myid:] - INFO [main:QuorumBase@323] - Waiting for QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11233 to exit thread 2017-01-24 22:13:08,877 [myid:2] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 2, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:13:08,879 [myid:2] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:13:08,876 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@980] - Connection broken for id 2, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:13:08,879 [myid:1] - WARN [RecvWorker:2:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:13:08,876 [myid:2] - ERROR [localhost/127.0.0.1:11235:QuorumCnxManager$Listener@715] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:696) 2017-01-24 22:13:08,880 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:13:08,880 [myid:1] - WARN [SendWorker:2:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:13:08,880 [myid:] - INFO [main:ZKTestCase$1@65] - FAILED testRollingUpgrade java.lang.AssertionError: waiting for server1being up at org.junit.Assert.fail(Assert.java:91) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.restartServer(QuorumAuthUpgradeTest.java:232) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.testRollingUpgrade(QuorumAuthUpgradeTest.java:204) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52) at org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28) 2017-01-24 22:13:08,880 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testRollingUpgrade 2017-01-24 22:13:08,885 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testNullAuthLearnerServer 2017-01-24 22:13:08,885 [myid:] - INFO [Thread-170:JUnit4ZKTestRunner$LoggedInvokeMethod@50] - RUNNING TEST METHOD testNullAuthLearnerServer 2017-01-24 22:13:08,885 [myid:] - INFO [Thread-170:PortAssignment@32] - assigning port 11236 2017-01-24 22:13:08,886 [myid:] - INFO [Thread-170:PortAssignment@32] - assigning port 11237 2017-01-24 22:13:08,886 [myid:] - INFO [Thread-170:PortAssignment@32] - assigning port 11238 2017-01-24 22:13:08,886 [myid:] - INFO [Thread-170:PortAssignment@32] - assigning port 11239 2017-01-24 22:13:08,886 [myid:] - INFO [Thread-170:PortAssignment@32] - assigning port 11240 2017-01-24 22:13:08,886 [myid:] - INFO [Thread-170:PortAssignment@32] - assigning port 11241 2017-01-24 22:13:08,886 [myid:] - INFO [Thread-170:QuorumPeerTestBase$MainThread@81] - id = 0 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6384659498769109567.junit.dir clientPort = 11236 2017-01-24 22:13:08,887 [myid:] - INFO [Thread-171:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6384659498769109567.junit.dir/zoo.cfg 2017-01-24 22:13:08,887 [myid:] - INFO [Thread-170:QuorumPeerTestBase$MainThread@81] - id = 1 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test5470635973333042784.junit.dir clientPort = 11239 2017-01-24 22:13:08,887 [myid:] - WARN [Thread-171:QuorumPeerConfig@327] - No server failure will be tolerated. You need at least 3 servers. 2017-01-24 22:13:08,888 [myid:] - INFO [Thread-171:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:13:08,888 [myid:0] - INFO [Thread-171:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:13:08,888 [myid:0] - INFO [Thread-171:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:13:08,888 [myid:0] - INFO [Thread-171:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:13:08,888 [myid:0] - WARN [Thread-171:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:13:08,888 [myid:0] - INFO [Thread-171:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:13:08,889 [myid:] - INFO [Thread-170:FourLetterWordMain@43] - connecting to 127.0.0.1 11236 2017-01-24 22:13:08,897 [myid:] - INFO [Thread-170:ClientBase@246] - server 127.0.0.1:11236 not up java.net.ConnectException: Connection refused 2017-01-24 22:13:08,889 [myid:] - INFO [Thread-172:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test5470635973333042784.junit.dir/zoo.cfg 2017-01-24 22:13:08,897 [myid:] - WARN [Thread-172:QuorumPeerConfig@327] - No server failure will be tolerated. You need at least 3 servers. 2017-01-24 22:13:08,897 [myid:] - INFO [Thread-172:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:13:08,897 [myid:1] - INFO [Thread-172:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:13:08,898 [myid:1] - INFO [Thread-172:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:13:08,898 [myid:1] - INFO [Thread-172:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:13:08,898 [myid:1] - WARN [Thread-172:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:13:08,898 [myid:1] - INFO [Thread-172:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:13:08,899 [myid:1] - INFO [Thread-172:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11239 2017-01-24 22:13:08,899 [myid:1] - INFO [Thread-172:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:13:08,899 [myid:1] - INFO [Thread-172:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:13:08,900 [myid:1] - INFO [Thread-172:QuorumPeer@1277] - QuorumPeer communication is not secured! 2017-01-24 22:13:08,900 [myid:1] - INFO [Thread-172:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:13:08,900 [myid:1] - INFO [Thread-172:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:08,889 [myid:0] - INFO [Thread-171:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11236 2017-01-24 22:13:08,901 [myid:0] - INFO [Thread-171:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:13:08,901 [myid:0] - INFO [Thread-171:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:13:08,901 [myid:0] - INFO [Thread-171:QuorumPeer@1277] - QuorumPeer communication is not secured! 2017-01-24 22:13:08,901 [myid:0] - INFO [Thread-171:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:13:08,902 [myid:0] - INFO [Thread-171:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:08,903 [myid:1] - INFO [Thread-172:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:08,903 [myid:0] - INFO [Thread-171:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:08,905 [myid:1] - INFO [Thread-173:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11241 2017-01-24 22:13:08,909 [myid:0] - INFO [Thread-174:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11238 2017-01-24 22:13:08,910 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:QuorumPeer@781] - LOOKING 2017-01-24 22:13:08,910 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11239:QuorumPeer@781] - LOOKING 2017-01-24 22:13:08,910 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:FastLeaderElection@744] - New election. My id = 0, proposed zxid=0x0 2017-01-24 22:13:08,910 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11239:FastLeaderElection@744] - New election. My id = 1, proposed zxid=0x0 2017-01-24 22:13:08,911 [myid:0] - INFO [WorkerSender[myid=0]:QuorumCnxManager@331] - Have smaller server identifier, so dropping the connection: (1, 0) 2017-01-24 22:13:08,911 [myid:0] - INFO [localhost/127.0.0.1:11238:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:59130 2017-01-24 22:13:08,911 [myid:1] - INFO [localhost/127.0.0.1:11241:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:51284 2017-01-24 22:13:08,911 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:08,912 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:08,912 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@901] - Exception when using channel: for id 0 my id = 1 error = java.net.SocketException: Socket closed 2017-01-24 22:13:08,913 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:13:08,913 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:13:08,914 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@901] - Exception when using channel: for id 1 my id = 0 error = java.net.SocketException: Broken pipe 2017-01-24 22:13:08,915 [myid:0] - INFO [localhost/127.0.0.1:11238:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:59132 2017-01-24 22:13:08,915 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:13:08,915 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:13:08,917 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:08,917 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:08,918 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:08,921 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:09,074 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FastLeaderElection@778] - Notification time out: 400 2017-01-24 22:13:09,075 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11229 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:13:09,075 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x300000001 (n.zxid), 0x4 (n.round), LOOKING (n.state), 1 (n.sid), 0x3 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:09,075 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@559] - Cannot open channel to 2 at election address localhost/127.0.0.1:11235 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:13:09,119 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:QuorumPeer@849] - FOLLOWING 2017-01-24 22:13:09,119 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6384659498769109567.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6384659498769109567.junit.dir/data/version-2 2017-01-24 22:13:09,119 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 209 2017-01-24 22:13:09,120 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:Learner@233] - Unexpected exception, tries=0, connecting to localhost/127.0.0.1:11240 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.Learner.connectToLeader(Learner.java:225) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:72) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:851) 2017-01-24 22:13:09,122 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11239:QuorumPeer@861] - LEADING 2017-01-24 22:13:09,122 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11239:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test5470635973333042784.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test5470635973333042784.junit.dir/data/version-2 2017-01-24 22:13:09,122 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11239:Leader@356] - LEADING - LEADER ELECTION TOOK - 212 2017-01-24 22:13:09,123 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11239:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test5470635973333042784.junit.dir/data/version-2/snapshot.0 2017-01-24 22:13:09,147 [myid:] - INFO [Thread-170:FourLetterWordMain@43] - connecting to 127.0.0.1 11236 2017-01-24 22:13:09,147 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:40329 2017-01-24 22:13:09,147 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:40329 2017-01-24 22:13:09,148 [myid:0] - INFO [Thread-176:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:40329 (no session established for client) 2017-01-24 22:13:09,399 [myid:] - INFO [Thread-170:FourLetterWordMain@43] - connecting to 127.0.0.1 11236 2017-01-24 22:13:09,399 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:40330 2017-01-24 22:13:09,399 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:40330 2017-01-24 22:13:09,400 [myid:0] - INFO [Thread-177:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:40330 (no session established for client) 2017-01-24 22:13:09,476 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11229 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:13:09,477 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumCnxManager@559] - Cannot open channel to 2 at election address localhost/127.0.0.1:11235 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:13:09,477 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FastLeaderElection@778] - Notification time out: 800 2017-01-24 22:13:09,650 [myid:] - INFO [Thread-170:FourLetterWordMain@43] - connecting to 127.0.0.1 11236 2017-01-24 22:13:09,651 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:40333 2017-01-24 22:13:09,651 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:40333 2017-01-24 22:13:09,651 [myid:0] - INFO [Thread-178:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:40333 (no session established for client) 2017-01-24 22:13:09,880 [myid:2] - INFO [localhost/127.0.0.1:11235:QuorumCnxManager$Listener@728] - Leaving listener 2017-01-24 22:13:09,902 [myid:] - INFO [Thread-170:FourLetterWordMain@43] - connecting to 127.0.0.1 11236 2017-01-24 22:13:09,902 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:40334 2017-01-24 22:13:09,903 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:40334 2017-01-24 22:13:09,903 [myid:0] - INFO [Thread-179:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:40334 (no session established for client) 2017-01-24 22:13:10,121 [myid:1] - INFO [LearnerHandler-/127.0.0.1:51982:LearnerHandler@287] - Follower sid: 0 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@36d2499a 2017-01-24 22:13:10,123 [myid:1] - INFO [LearnerHandler-/127.0.0.1:51982:LearnerHandler@342] - Synchronizing with Follower sid: 0 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0 2017-01-24 22:13:10,124 [myid:1] - INFO [LearnerHandler-/127.0.0.1:51982:LearnerHandler@441] - Sending snapshot last zxid of peer is 0x0 zxid of leader is 0x100000000sent zxid of db as 0x0 2017-01-24 22:13:10,124 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:Learner@329] - Getting a snapshot from leader 2017-01-24 22:13:10,125 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test6384659498769109567.junit.dir/data/version-2/snapshot.0 2017-01-24 22:13:10,126 [myid:1] - INFO [LearnerHandler-/127.0.0.1:51982:LearnerHandler@477] - Received NEWLEADER-ACK message from 0 2017-01-24 22:13:10,126 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11239:Leader@934] - Have quorum of supporters, sids: [ 0,1 ]; starting up and setting last processed zxid: 0x100000000 2017-01-24 22:13:10,153 [myid:] - INFO [Thread-170:FourLetterWordMain@43] - connecting to 127.0.0.1 11236 2017-01-24 22:13:10,154 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:40336 2017-01-24 22:13:10,154 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:40336 2017-01-24 22:13:10,155 [myid:0] - INFO [Thread-181:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:13:10,155 [myid:0] - INFO [Thread-181:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:40336 (no session established for client) 2017-01-24 22:13:10,155 [myid:] - INFO [Thread-170:FourLetterWordMain@43] - connecting to 127.0.0.1 11239 2017-01-24 22:13:10,156 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11239:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:35057 2017-01-24 22:13:10,156 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11239:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:35057 2017-01-24 22:13:10,156 [myid:1] - INFO [Thread-182:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:13:10,157 [myid:1] - INFO [Thread-182:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:35057 (no session established for client) 2017-01-24 22:13:10,157 [myid:] - INFO [Thread-170:ZooKeeper@438] - Initiating client connection, connectString=127.0.0.1:11236,127.0.0.1:11239 sessionTimeout=30000 watcher=org.apache.zookeeper.test.ClientBase$CountdownWatcher@6538b0f9 2017-01-24 22:13:10,158 [myid:] - WARN [Thread-170-SendThread(localhost:11236):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:10,158 [myid:] - INFO [Thread-170-SendThread(localhost:11236):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11236 2017-01-24 22:13:10,159 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:40338 2017-01-24 22:13:10,159 [myid:] - INFO [Thread-170-SendThread(localhost:11236):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:40338, server: localhost/127.0.0.1:11236 2017-01-24 22:13:10,159 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:ZooKeeperServer@839] - Client attempting to establish new session at /127.0.0.1:40338 2017-01-24 22:13:10,160 [myid:1] - INFO [SyncThread:1:FileTxnLog@199] - Creating new log file: log.100000001 2017-01-24 22:13:10,160 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:Follower@119] - Got zxid 0x100000001 expected 0x1 2017-01-24 22:13:10,160 [myid:0] - INFO [SyncThread:0:FileTxnLog@199] - Creating new log file: log.100000001 2017-01-24 22:13:10,163 [myid:0] - INFO [CommitProcessor:0:ZooKeeperServer@595] - Established session 0x59d441a96d0000 with negotiated timeout 30000 for client /127.0.0.1:40338 2017-01-24 22:13:10,163 [myid:] - INFO [Thread-170-SendThread(localhost:11236):ClientCnxn$SendThread@1235] - Session establishment complete on server localhost/127.0.0.1:11236, sessionid = 0x59d441a96d0000, negotiated timeout = 30000 2017-01-24 22:13:10,167 [myid:1] - INFO [ProcessThread(sid:1 cport:-1)::PrepRequestProcessor@494] - Processed session termination for sessionid: 0x59d441a96d0000 2017-01-24 22:13:10,168 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:40338 which had sessionid 0x59d441a96d0000 2017-01-24 22:13:10,169 [myid:] - INFO [Thread-170:ZooKeeper@684] - Session: 0x59d441a96d0000 closed 2017-01-24 22:13:10,169 [myid:] - INFO [Thread-170-EventThread:ClientCnxn$EventThread@512] - EventThread shut down 2017-01-24 22:13:10,169 [myid:] - INFO [Thread-170:JUnit4ZKTestRunner$LoggedInvokeMethod@55] - Memory used 27198 2017-01-24 22:13:10,169 [myid:] - INFO [Thread-170:JUnit4ZKTestRunner$LoggedInvokeMethod@60] - Number of threads 53 2017-01-24 22:13:10,169 [myid:] - INFO [Thread-170:JUnit4ZKTestRunner$LoggedInvokeMethod@65] - FINISHED TEST METHOD testNullAuthLearnerServer 2017-01-24 22:13:10,169 [myid:] - INFO [main:QuorumBase@314] - Shutting down quorum peer QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236 2017-01-24 22:13:10,169 [myid:] - INFO [main:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:896) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:315) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$TestQPMain.shutdown(QuorumPeerTestBase.java:59) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.shutdown(QuorumPeerTestBase.java:152) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdown(QuorumAuthTestBase.java:138) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdownAll(QuorumAuthTestBase.java:131) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.tearDown(QuorumAuthUpgradeTest.java:68) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033) 2017-01-24 22:13:10,170 [myid:] - INFO [main:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:13:10,170 [myid:] - INFO [main:ZooKeeperServer@419] - shutting down 2017-01-24 22:13:10,170 [myid:] - INFO [main:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:13:10,170 [myid:] - INFO [main:CommitProcessor@181] - Shutting down 2017-01-24 22:13:10,170 [myid:] - INFO [main:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:13:10,170 [myid:0] - INFO [FollowerRequestProcessor:0:FollowerRequestProcessor@95] - FollowerRequestProcessor exited loop! 2017-01-24 22:13:10,170 [myid:0] - INFO [CommitProcessor:0:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:13:10,171 [myid:] - INFO [main:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:13:10,171 [myid:0] - INFO [SyncThread:0:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:13:10,172 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method 2017-01-24 22:13:10,172 [myid:0] - ERROR [localhost/127.0.0.1:11238:QuorumCnxManager$Listener@715] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:696) 2017-01-24 22:13:10,173 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:13:10,173 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:13:10,173 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:13:10,173 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:13:10,174 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:13:10,174 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:13:10,174 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:13:10,175 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:13:10,175 [myid:] - INFO [main:QuorumBase@318] - Shutting down leader election QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236 2017-01-24 22:13:10,175 [myid:] - INFO [main:QuorumBase@323] - Waiting for QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236 to exit thread 2017-01-24 22:13:10,230 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:10,231 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:13:10,231 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52552 2017-01-24 22:13:10,231 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52552, server: localhost/127.0.0.1:11230 2017-01-24 22:13:10,231 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:13:10,231 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52552 (no session established for client) 2017-01-24 22:13:10,232 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:13:10,277 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11229 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:13:10,278 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumCnxManager@559] - Cannot open channel to 2 at election address localhost/127.0.0.1:11235 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:13:10,278 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FastLeaderElection@778] - Notification time out: 1600 2017-01-24 22:13:10,350 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:10,350 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:13:10,351 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1102] - Session 0x159d440f0ed0000 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2017-01-24 22:13:10,685 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:10,685 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:13:10,685 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1102] - Session 0x159d440f0ed0000 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2017-01-24 22:13:11,173 [myid:0] - INFO [localhost/127.0.0.1:11238:QuorumCnxManager$Listener@728] - Leaving listener 2017-01-24 22:13:11,873 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@340] - WorkerReceiver is down 2017-01-24 22:13:11,875 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:11,875 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:13:11,875 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52557 2017-01-24 22:13:11,876 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52557, server: localhost/127.0.0.1:11230 2017-01-24 22:13:11,876 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:13:11,876 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52557 (no session established for client) 2017-01-24 22:13:11,876 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:13:11,879 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11229 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:13:11,879 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumCnxManager@559] - Cannot open channel to 2 at election address localhost/127.0.0.1:11235 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:13:11,879 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FastLeaderElection@778] - Notification time out: 3200 2017-01-24 22:13:11,918 [myid:0] - INFO [WorkerSender[myid=0]:FastLeaderElection$Messenger$WorkerSender@370] - WorkerSender is down 2017-01-24 22:13:11,919 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection$Messenger$WorkerReceiver@340] - WorkerReceiver is down 2017-01-24 22:13:12,132 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:13:12,132 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:13:12,132 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:ZooKeeperServer@419] - shutting down 2017-01-24 22:13:12,133 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:13:12,133 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:CommitProcessor@181] - Shutting down 2017-01-24 22:13:12,133 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:13:12,133 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:13:12,133 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11236:QuorumPeer@879] - QuorumPeer main thread exited 2017-01-24 22:13:12,135 [myid:] - INFO [main:ZKTestCase$1@60] - SUCCEEDED testNullAuthLearnerServer 2017-01-24 22:13:12,135 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testNullAuthLearnerServer 2017-01-24 22:13:12,136 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testAuthLearnerAgainstNullAuthServer 2017-01-24 22:13:12,136 [myid:] - INFO [Thread-183:JUnit4ZKTestRunner$LoggedInvokeMethod@50] - RUNNING TEST METHOD testAuthLearnerAgainstNullAuthServer 2017-01-24 22:13:12,136 [myid:] - INFO [Thread-183:PortAssignment@32] - assigning port 11242 2017-01-24 22:13:12,136 [myid:] - INFO [Thread-183:PortAssignment@32] - assigning port 11243 2017-01-24 22:13:12,136 [myid:] - INFO [Thread-183:PortAssignment@32] - assigning port 11244 2017-01-24 22:13:12,137 [myid:] - INFO [Thread-183:PortAssignment@32] - assigning port 11245 2017-01-24 22:13:12,137 [myid:] - INFO [Thread-183:PortAssignment@32] - assigning port 11246 2017-01-24 22:13:12,137 [myid:] - INFO [Thread-183:PortAssignment@32] - assigning port 11247 2017-01-24 22:13:12,137 [myid:] - INFO [Thread-183:QuorumPeerTestBase$MainThread@81] - id = 0 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test7361117950796815905.junit.dir clientPort = 11242 2017-01-24 22:13:12,138 [myid:] - INFO [Thread-184:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test7361117950796815905.junit.dir/zoo.cfg 2017-01-24 22:13:12,138 [myid:] - INFO [Thread-183:QuorumPeerTestBase$MainThread@81] - id = 1 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test296771033193329447.junit.dir clientPort = 11245 2017-01-24 22:13:12,138 [myid:] - WARN [Thread-184:QuorumPeerConfig@327] - No server failure will be tolerated. You need at least 3 servers. 2017-01-24 22:13:12,138 [myid:] - INFO [Thread-184:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:13:12,138 [myid:] - INFO [Thread-183:FourLetterWordMain@43] - connecting to 127.0.0.1 11242 2017-01-24 22:13:12,138 [myid:0] - INFO [Thread-184:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:13:12,139 [myid:0] - INFO [Thread-184:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:13:12,139 [myid:0] - INFO [Thread-184:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:13:12,139 [myid:] - INFO [Thread-183:ClientBase@246] - server 127.0.0.1:11242 not up java.net.ConnectException: Connection refused 2017-01-24 22:13:12,139 [myid:0] - WARN [Thread-184:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:13:12,139 [myid:0] - INFO [Thread-184:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:13:12,139 [myid:] - INFO [Thread-185:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test296771033193329447.junit.dir/zoo.cfg 2017-01-24 22:13:12,140 [myid:0] - INFO [Thread-184:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11242 2017-01-24 22:13:12,140 [myid:] - WARN [Thread-185:QuorumPeerConfig@327] - No server failure will be tolerated. You need at least 3 servers. 2017-01-24 22:13:12,140 [myid:] - INFO [Thread-185:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:13:12,140 [myid:0] - INFO [Thread-184:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:13:12,140 [myid:1] - INFO [Thread-185:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:13:12,140 [myid:1] - INFO [Thread-185:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:13:12,141 [myid:1] - INFO [Thread-185:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:13:12,140 [myid:0] - INFO [Thread-184:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:13:12,141 [myid:1] - WARN [Thread-185:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:13:12,141 [myid:1] - INFO [Thread-185:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:13:12,141 [myid:0] - INFO [Thread-184:QuorumPeer@1279] - quorum.auth.enableSasl set to true 2017-01-24 22:13:12,141 [myid:0] - INFO [Thread-184:QuorumPeer@1264] - quorum.auth.serverRequireSasl set to false 2017-01-24 22:13:12,141 [myid:0] - INFO [Thread-184:QuorumPeer@1270] - quorum.auth.learnerRequireSasl set to false 2017-01-24 22:13:12,141 [myid:1] - INFO [Thread-185:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11245 2017-01-24 22:13:12,142 [myid:0] - INFO [Thread-184:QuorumPeer@1286] - quorum.auth.kerberos.servicePrincipal set to zkquorum/localhost 2017-01-24 22:13:12,142 [myid:0] - INFO [Thread-184:QuorumPeer@1298] - quorum.auth.server.saslLoginContext set to QuorumServer 2017-01-24 22:13:12,142 [myid:0] - INFO [Thread-184:QuorumPeer@1292] - quorum.auth.learner.saslLoginContext set to QuorumLearner 2017-01-24 22:13:12,142 [myid:1] - INFO [Thread-185:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:13:12,142 [myid:1] - INFO [Thread-185:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:13:12,142 [myid:0] - INFO [Thread-184:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:13:12,142 [myid:1] - INFO [Thread-185:QuorumPeer@1277] - QuorumPeer communication is not secured! 2017-01-24 22:13:12,142 [myid:1] - INFO [Thread-185:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:13:12,143 [myid:1] - INFO [Thread-185:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:12,144 [myid:1] - INFO [Thread-185:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:12,146 [myid:1] - INFO [Thread-186:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11247 2017-01-24 22:13:12,146 [myid:0] - INFO [Thread-184:Login@294] - QuorumServer successfully logged in. 2017-01-24 22:13:12,146 [myid:0] - INFO [Thread-184:Login@294] - QuorumLearner successfully logged in. 2017-01-24 22:13:12,147 [myid:0] - INFO [Thread-184:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:12,148 [myid:0] - INFO [Thread-184:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:12,151 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11245:QuorumPeer@781] - LOOKING 2017-01-24 22:13:12,152 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11245:FastLeaderElection@744] - New election. My id = 1, proposed zxid=0x0 2017-01-24 22:13:12,152 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11244 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:514) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:13:12,152 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:12,153 [myid:0] - INFO [Thread-187:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11244 2017-01-24 22:13:12,158 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:QuorumPeer@781] - LOOKING 2017-01-24 22:13:12,158 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:FastLeaderElection@744] - New election. My id = 0, proposed zxid=0x0 2017-01-24 22:13:12,158 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:12,159 [myid:1] - INFO [localhost/127.0.0.1:11247:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:38550 2017-01-24 22:13:12,159 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:13:12,160 [myid:0] - INFO [localhost/127.0.0.1:11244:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:53591 2017-01-24 22:13:12,160 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:QuorumCnxManager@331] - Have smaller server identifier, so dropping the connection: (1, 0) 2017-01-24 22:13:12,167 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:12,167 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:12,167 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:12,168 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:12,368 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:QuorumPeer@849] - FOLLOWING 2017-01-24 22:13:12,368 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11245:QuorumPeer@861] - LEADING 2017-01-24 22:13:12,368 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test7361117950796815905.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test7361117950796815905.junit.dir/data/version-2 2017-01-24 22:13:12,368 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11245:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test296771033193329447.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test296771033193329447.junit.dir/data/version-2 2017-01-24 22:13:12,368 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 210 2017-01-24 22:13:12,368 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11245:Leader@356] - LEADING - LEADER ELECTION TOOK - 216 2017-01-24 22:13:12,369 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:SaslQuorumAuthLearner@79] - Skipping SASL authentication as quorum.auth.learnerRequireSasl=false 2017-01-24 22:13:12,369 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11245:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test296771033193329447.junit.dir/data/version-2/snapshot.0 2017-01-24 22:13:12,371 [myid:1] - INFO [LearnerHandler-/127.0.0.1:56735:LearnerHandler@287] - Follower sid: 0 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@68eefca4 2017-01-24 22:13:12,373 [myid:1] - INFO [LearnerHandler-/127.0.0.1:56735:LearnerHandler@342] - Synchronizing with Follower sid: 0 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0 2017-01-24 22:13:12,373 [myid:1] - INFO [LearnerHandler-/127.0.0.1:56735:LearnerHandler@441] - Sending snapshot last zxid of peer is 0x0 zxid of leader is 0x100000000sent zxid of db as 0x0 2017-01-24 22:13:12,373 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:Learner@329] - Getting a snapshot from leader 2017-01-24 22:13:12,374 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test7361117950796815905.junit.dir/data/version-2/snapshot.0 2017-01-24 22:13:12,375 [myid:1] - INFO [LearnerHandler-/127.0.0.1:56735:LearnerHandler@477] - Received NEWLEADER-ACK message from 0 2017-01-24 22:13:12,375 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11245:Leader@934] - Have quorum of supporters, sids: [ 0,1 ]; starting up and setting last processed zxid: 0x100000000 2017-01-24 22:13:12,389 [myid:] - INFO [Thread-183:FourLetterWordMain@43] - connecting to 127.0.0.1 11242 2017-01-24 22:13:12,389 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11242:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:43163 2017-01-24 22:13:12,390 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11242:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:43163 2017-01-24 22:13:12,390 [myid:0] - INFO [Thread-190:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:13:12,391 [myid:0] - INFO [Thread-190:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:43163 (no session established for client) 2017-01-24 22:13:12,391 [myid:] - INFO [Thread-183:FourLetterWordMain@43] - connecting to 127.0.0.1 11245 2017-01-24 22:13:12,391 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11245:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:34938 2017-01-24 22:13:12,392 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11245:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:34938 2017-01-24 22:13:12,392 [myid:1] - INFO [Thread-191:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:13:12,393 [myid:1] - INFO [Thread-191:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:34938 (no session established for client) 2017-01-24 22:13:12,393 [myid:] - INFO [Thread-183:ZooKeeper@438] - Initiating client connection, connectString=127.0.0.1:11242,127.0.0.1:11245 sessionTimeout=30000 watcher=org.apache.zookeeper.test.ClientBase$CountdownWatcher@494b584c 2017-01-24 22:13:12,394 [myid:] - WARN [Thread-183-SendThread(localhost:11245):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:12,394 [myid:] - INFO [Thread-183-SendThread(localhost:11245):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11245 2017-01-24 22:13:12,394 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11245:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:34939 2017-01-24 22:13:12,394 [myid:] - INFO [Thread-183-SendThread(localhost:11245):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:34939, server: localhost/127.0.0.1:11245 2017-01-24 22:13:12,395 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11245:ZooKeeperServer@839] - Client attempting to establish new session at /127.0.0.1:34939 2017-01-24 22:13:12,395 [myid:1] - INFO [SyncThread:1:FileTxnLog@199] - Creating new log file: log.100000001 2017-01-24 22:13:12,396 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:Follower@119] - Got zxid 0x100000001 expected 0x1 2017-01-24 22:13:12,396 [myid:0] - INFO [SyncThread:0:FileTxnLog@199] - Creating new log file: log.100000001 2017-01-24 22:13:12,398 [myid:1] - INFO [CommitProcessor:1:ZooKeeperServer@595] - Established session 0x159d441b2380000 with negotiated timeout 30000 for client /127.0.0.1:34939 2017-01-24 22:13:12,398 [myid:] - INFO [Thread-183-SendThread(localhost:11245):ClientCnxn$SendThread@1235] - Session establishment complete on server localhost/127.0.0.1:11245, sessionid = 0x159d441b2380000, negotiated timeout = 30000 2017-01-24 22:13:12,405 [myid:1] - INFO [ProcessThread(sid:1 cport:-1)::PrepRequestProcessor@494] - Processed session termination for sessionid: 0x159d441b2380000 2017-01-24 22:13:12,408 [myid:] - INFO [Thread-183:ZooKeeper@684] - Session: 0x159d441b2380000 closed 2017-01-24 22:13:12,408 [myid:] - INFO [Thread-183-EventThread:ClientCnxn$EventThread@512] - EventThread shut down 2017-01-24 22:13:12,408 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11245:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:34939 which had sessionid 0x159d441b2380000 2017-01-24 22:13:12,408 [myid:] - INFO [Thread-183:JUnit4ZKTestRunner$LoggedInvokeMethod@55] - Memory used 30080 2017-01-24 22:13:12,409 [myid:] - INFO [Thread-183:JUnit4ZKTestRunner$LoggedInvokeMethod@60] - Number of threads 67 2017-01-24 22:13:12,409 [myid:] - INFO [Thread-183:JUnit4ZKTestRunner$LoggedInvokeMethod@65] - FINISHED TEST METHOD testAuthLearnerAgainstNullAuthServer 2017-01-24 22:13:12,409 [myid:] - INFO [main:QuorumBase@314] - Shutting down quorum peer QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242 2017-01-24 22:13:12,409 [myid:] - INFO [main:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:896) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:315) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$TestQPMain.shutdown(QuorumPeerTestBase.java:59) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.shutdown(QuorumPeerTestBase.java:152) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdown(QuorumAuthTestBase.java:138) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdownAll(QuorumAuthTestBase.java:131) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.tearDown(QuorumAuthUpgradeTest.java:68) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033) 2017-01-24 22:13:12,410 [myid:] - INFO [main:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:13:12,410 [myid:] - INFO [main:ZooKeeperServer@419] - shutting down 2017-01-24 22:13:12,410 [myid:] - INFO [main:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:13:12,410 [myid:] - INFO [main:CommitProcessor@181] - Shutting down 2017-01-24 22:13:12,410 [myid:] - INFO [main:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:13:12,410 [myid:0] - INFO [FollowerRequestProcessor:0:FollowerRequestProcessor@95] - FollowerRequestProcessor exited loop! 2017-01-24 22:13:12,410 [myid:0] - INFO [CommitProcessor:0:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:13:12,411 [myid:] - INFO [main:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:13:12,411 [myid:0] - INFO [SyncThread:0:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:13:12,411 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11242:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method 2017-01-24 22:13:12,412 [myid:0] - ERROR [localhost/127.0.0.1:11244:QuorumCnxManager$Listener@715] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:696) 2017-01-24 22:13:12,412 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:13:12,412 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:13:12,413 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:13:12,414 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:13:12,412 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:13:12,414 [myid:] - INFO [main:QuorumBase@318] - Shutting down leader election QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242 2017-01-24 22:13:12,413 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:13:12,415 [myid:] - INFO [main:QuorumBase@323] - Waiting for QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242 to exit thread 2017-01-24 22:13:12,414 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:13:12,415 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:13:12,656 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:12,656 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:13:12,656 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1102] - Session 0x159d440f0ed0000 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2017-01-24 22:13:13,412 [myid:0] - INFO [localhost/127.0.0.1:11244:QuorumCnxManager$Listener@728] - Leaving listener 2017-01-24 22:13:13,603 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:13,603 [myid:] - INFO [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11227 2017-01-24 22:13:13,604 [myid:] - WARN [Thread-12-SendThread(localhost:11227):ClientCnxn$SendThread@1102] - Session 0x159d440f0ed0000 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2017-01-24 22:13:14,382 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:13:14,382 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:13:14,382 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:ZooKeeperServer@419] - shutting down 2017-01-24 22:13:14,382 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:13:14,382 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:CommitProcessor@181] - Shutting down 2017-01-24 22:13:14,382 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:13:14,383 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:13:14,383 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11242:QuorumPeer@879] - QuorumPeer main thread exited 2017-01-24 22:13:14,384 [myid:] - INFO [main:ZKTestCase$1@60] - SUCCEEDED testAuthLearnerAgainstNullAuthServer 2017-01-24 22:13:14,384 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testAuthLearnerAgainstNullAuthServer 2017-01-24 22:13:14,385 [myid:] - INFO [main:ZKTestCase$1@50] - STARTING testAuthLearnerServer 2017-01-24 22:13:14,385 [myid:] - INFO [Thread-192:JUnit4ZKTestRunner$LoggedInvokeMethod@50] - RUNNING TEST METHOD testAuthLearnerServer 2017-01-24 22:13:14,385 [myid:] - INFO [Thread-192:PortAssignment@32] - assigning port 11248 2017-01-24 22:13:14,386 [myid:] - INFO [Thread-192:PortAssignment@32] - assigning port 11249 2017-01-24 22:13:14,386 [myid:] - INFO [Thread-192:PortAssignment@32] - assigning port 11250 2017-01-24 22:13:14,386 [myid:] - INFO [Thread-192:PortAssignment@32] - assigning port 11251 2017-01-24 22:13:14,386 [myid:] - INFO [Thread-192:PortAssignment@32] - assigning port 11252 2017-01-24 22:13:14,386 [myid:] - INFO [Thread-192:PortAssignment@32] - assigning port 11253 2017-01-24 22:13:14,386 [myid:] - INFO [Thread-192:QuorumPeerTestBase$MainThread@81] - id = 0 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4413362313903870178.junit.dir clientPort = 11248 2017-01-24 22:13:14,387 [myid:] - INFO [Thread-193:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4413362313903870178.junit.dir/zoo.cfg 2017-01-24 22:13:14,387 [myid:] - INFO [Thread-192:QuorumPeerTestBase$MainThread@81] - id = 1 tmpDir = /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8693106920335238483.junit.dir clientPort = 11251 2017-01-24 22:13:14,387 [myid:] - WARN [Thread-193:QuorumPeerConfig@327] - No server failure will be tolerated. You need at least 3 servers. 2017-01-24 22:13:14,388 [myid:] - INFO [Thread-193:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:13:14,388 [myid:] - INFO [Thread-192:FourLetterWordMain@43] - connecting to 127.0.0.1 11248 2017-01-24 22:13:14,388 [myid:0] - INFO [Thread-193:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:13:14,388 [myid:0] - INFO [Thread-193:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:13:14,388 [myid:0] - INFO [Thread-193:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:13:14,388 [myid:] - INFO [Thread-192:ClientBase@246] - server 127.0.0.1:11248 not up java.net.ConnectException: Connection refused 2017-01-24 22:13:14,388 [myid:0] - WARN [Thread-193:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:13:14,389 [myid:0] - INFO [Thread-193:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:13:14,388 [myid:] - INFO [Thread-194:QuorumPeerConfig@111] - Reading configuration from: /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8693106920335238483.junit.dir/zoo.cfg 2017-01-24 22:13:14,389 [myid:0] - INFO [Thread-193:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11248 2017-01-24 22:13:14,389 [myid:] - WARN [Thread-194:QuorumPeerConfig@327] - No server failure will be tolerated. You need at least 3 servers. 2017-01-24 22:13:14,389 [myid:] - INFO [Thread-194:QuorumPeerConfig@374] - Defaulting to majority quorums 2017-01-24 22:13:14,390 [myid:1] - INFO [Thread-194:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-01-24 22:13:14,390 [myid:1] - INFO [Thread-194:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-01-24 22:13:14,390 [myid:1] - INFO [Thread-194:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-01-24 22:13:14,390 [myid:0] - INFO [Thread-193:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:13:14,390 [myid:0] - INFO [Thread-193:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:13:14,390 [myid:1] - WARN [Thread-194:QuorumPeerMain@129] - Unable to register log4j JMX control javax.management.InstanceAlreadyExistsException: log4j:hiearchy=default at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:127) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.run(QuorumPeerTestBase.java:140) at java.lang.Thread.run(Thread.java:745) 2017-01-24 22:13:14,391 [myid:1] - INFO [Thread-194:QuorumPeerMain@132] - Starting quorum peer 2017-01-24 22:13:14,391 [myid:0] - INFO [Thread-193:QuorumPeer@1279] - quorum.auth.enableSasl set to true 2017-01-24 22:13:14,391 [myid:0] - INFO [Thread-193:QuorumPeer@1264] - quorum.auth.serverRequireSasl set to true 2017-01-24 22:13:14,391 [myid:0] - INFO [Thread-193:QuorumPeer@1270] - quorum.auth.learnerRequireSasl set to true 2017-01-24 22:13:14,391 [myid:1] - INFO [Thread-194:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11251 2017-01-24 22:13:14,391 [myid:0] - INFO [Thread-193:QuorumPeer@1286] - quorum.auth.kerberos.servicePrincipal set to zkquorum/localhost 2017-01-24 22:13:14,391 [myid:0] - INFO [Thread-193:QuorumPeer@1298] - quorum.auth.server.saslLoginContext set to QuorumServer 2017-01-24 22:13:14,392 [myid:0] - INFO [Thread-193:QuorumPeer@1292] - quorum.auth.learner.saslLoginContext set to QuorumLearner 2017-01-24 22:13:14,392 [myid:1] - INFO [Thread-194:QuorumPeer@1048] - minSessionTimeout set to -1 2017-01-24 22:13:14,392 [myid:1] - INFO [Thread-194:QuorumPeer@1059] - maxSessionTimeout set to -1 2017-01-24 22:13:14,392 [myid:0] - INFO [Thread-193:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:13:14,392 [myid:1] - INFO [Thread-194:QuorumPeer@1279] - quorum.auth.enableSasl set to true 2017-01-24 22:13:14,392 [myid:1] - INFO [Thread-194:QuorumPeer@1264] - quorum.auth.serverRequireSasl set to true 2017-01-24 22:13:14,392 [myid:1] - INFO [Thread-194:QuorumPeer@1270] - quorum.auth.learnerRequireSasl set to true 2017-01-24 22:13:14,392 [myid:1] - INFO [Thread-194:QuorumPeer@1286] - quorum.auth.kerberos.servicePrincipal set to zkquorum/localhost 2017-01-24 22:13:14,392 [myid:1] - INFO [Thread-194:QuorumPeer@1298] - quorum.auth.server.saslLoginContext set to QuorumServer 2017-01-24 22:13:14,393 [myid:1] - INFO [Thread-194:QuorumPeer@1292] - quorum.auth.learner.saslLoginContext set to QuorumLearner 2017-01-24 22:13:14,393 [myid:1] - INFO [Thread-194:QuorumPeer@1306] - quorum.cnxn.threads.size set to 20 2017-01-24 22:13:14,393 [myid:0] - INFO [Thread-193:Login@294] - QuorumServer successfully logged in. 2017-01-24 22:13:14,397 [myid:1] - INFO [Thread-194:Login@294] - QuorumServer successfully logged in. 2017-01-24 22:13:14,397 [myid:0] - INFO [Thread-193:Login@294] - QuorumLearner successfully logged in. 2017-01-24 22:13:14,397 [myid:1] - INFO [Thread-194:Login@294] - QuorumLearner successfully logged in. 2017-01-24 22:13:14,397 [myid:0] - INFO [Thread-193:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:14,398 [myid:1] - INFO [Thread-194:QuorumPeer@540] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:14,398 [myid:0] - INFO [Thread-193:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:14,399 [myid:1] - INFO [Thread-194:QuorumPeer@555] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-01-24 22:13:14,405 [myid:0] - INFO [Thread-195:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11250 2017-01-24 22:13:14,405 [myid:1] - INFO [Thread-196:QuorumCnxManager$Listener@691] - My election bind port: 0.0.0.0/0.0.0.0:11253 2017-01-24 22:13:14,406 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11251:QuorumPeer@781] - LOOKING 2017-01-24 22:13:14,406 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:QuorumPeer@781] - LOOKING 2017-01-24 22:13:14,407 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11251:FastLeaderElection@744] - New election. My id = 1, proposed zxid=0x0 2017-01-24 22:13:14,407 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:FastLeaderElection@744] - New election. My id = 0, proposed zxid=0x0 2017-01-24 22:13:14,407 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:14,407 [myid:0] - INFO [localhost/127.0.0.1:11250:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:60553 2017-01-24 22:13:14,408 [myid:1] - INFO [localhost/127.0.0.1:11253:QuorumCnxManager$Listener@698] - Received connection request /127.0.0.1:59854 2017-01-24 22:13:14,408 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:14,411 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-1:SecurityUtils@68] - QuorumLearner will use DIGEST-MD5 as SASL mechanism. 2017-01-24 22:13:14,412 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:SecurityUtils@68] - QuorumLearner will use DIGEST-MD5 as SASL mechanism. 2017-01-24 22:13:14,415 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-2:SaslQuorumServerCallbackHandler@143] - Successfully authenticated learner: authenticationID=test; authorizationID=test. 2017-01-24 22:13:14,415 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-2:SaslQuorumServerCallbackHandler@143] - Successfully authenticated learner: authenticationID=test; authorizationID=test. 2017-01-24 22:13:14,416 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-2:SaslQuorumAuthServer@114] - Successfully completed the authentication using SASL. learner addr: /127.0.0.1:59854 2017-01-24 22:13:14,416 [myid:1] - INFO [QuorumConnectionThread-[myid=1]-1:SaslQuorumAuthLearner@151] - Successfully completed the authentication using SASL. server addr: localhost/127.0.0.1:11250, status: SUCCESS 2017-01-24 22:13:14,416 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-2:SaslQuorumAuthServer@114] - Successfully completed the authentication using SASL. learner addr: /127.0.0.1:60553 2017-01-24 22:13:14,416 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:SaslQuorumAuthLearner@151] - Successfully completed the authentication using SASL. server addr: localhost/127.0.0.1:11253, status: SUCCESS 2017-01-24 22:13:14,417 [myid:0] - INFO [QuorumConnectionThread-[myid=0]-1:QuorumCnxManager@331] - Have smaller server identifier, so dropping the connection: (1, 0) 2017-01-24 22:13:14,427 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:14,428 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:14,428 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:14,428 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@542] - Notification: 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state) 2017-01-24 22:13:14,628 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:QuorumPeer@849] - FOLLOWING 2017-01-24 22:13:14,629 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4413362313903870178.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4413362313903870178.junit.dir/data/version-2 2017-01-24 22:13:14,629 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11251:QuorumPeer@861] - LEADING 2017-01-24 22:13:14,629 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:Follower@64] - FOLLOWING - LEADER ELECTION TOOK - 222 2017-01-24 22:13:14,629 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11251:ZooKeeperServer@162] - Created server with tickTime 4000 minSessionTimeout 8000 maxSessionTimeout 80000 datadir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8693106920335238483.junit.dir/data/version-2 snapdir /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8693106920335238483.junit.dir/data/version-2 2017-01-24 22:13:14,629 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11251:Leader@356] - LEADING - LEADER ELECTION TOOK - 222 2017-01-24 22:13:14,630 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:SecurityUtils@68] - QuorumLearner will use DIGEST-MD5 as SASL mechanism. 2017-01-24 22:13:14,630 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11251:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test8693106920335238483.junit.dir/data/version-2/snapshot.0 2017-01-24 22:13:14,632 [myid:1] - INFO [Thread-197:SaslQuorumServerCallbackHandler@143] - Successfully authenticated learner: authenticationID=test; authorizationID=test. 2017-01-24 22:13:14,633 [myid:1] - INFO [Thread-197:SaslQuorumAuthServer@114] - Successfully completed the authentication using SASL. learner addr: /127.0.0.1:36656 2017-01-24 22:13:14,634 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:SaslQuorumAuthLearner@151] - Successfully completed the authentication using SASL. server addr: localhost/127.0.0.1:11252, status: SUCCESS 2017-01-24 22:13:14,634 [myid:1] - INFO [LearnerHandler-/127.0.0.1:36656:LearnerHandler@287] - Follower sid: 0 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@5b8ec669 2017-01-24 22:13:14,636 [myid:1] - INFO [LearnerHandler-/127.0.0.1:36656:LearnerHandler@342] - Synchronizing with Follower sid: 0 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0 2017-01-24 22:13:14,636 [myid:1] - INFO [LearnerHandler-/127.0.0.1:36656:LearnerHandler@441] - Sending snapshot last zxid of peer is 0x0 zxid of leader is 0x100000000sent zxid of db as 0x0 2017-01-24 22:13:14,636 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:Learner@329] - Getting a snapshot from leader 2017-01-24 22:13:14,638 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:FileTxnSnapLog@281] - Snapshotting: 0x0 to /data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test4413362313903870178.junit.dir/data/version-2/snapshot.0 2017-01-24 22:13:14,638 [myid:] - INFO [Thread-192:FourLetterWordMain@43] - connecting to 127.0.0.1 11248 2017-01-24 22:13:14,639 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11248:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52527 2017-01-24 22:13:14,639 [myid:1] - INFO [LearnerHandler-/127.0.0.1:36656:LearnerHandler@477] - Received NEWLEADER-ACK message from 0 2017-01-24 22:13:14,639 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11248:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:52527 2017-01-24 22:13:14,643 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11251:Leader@934] - Have quorum of supporters, sids: [ 0,1 ]; starting up and setting last processed zxid: 0x100000000 2017-01-24 22:13:14,647 [myid:0] - INFO [Thread-199:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52527 (no session established for client) 2017-01-24 22:13:14,898 [myid:] - INFO [Thread-192:FourLetterWordMain@43] - connecting to 127.0.0.1 11248 2017-01-24 22:13:14,898 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11248:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52528 2017-01-24 22:13:14,899 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11248:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:52528 2017-01-24 22:13:14,899 [myid:0] - INFO [Thread-200:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:13:14,900 [myid:0] - INFO [Thread-200:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52528 (no session established for client) 2017-01-24 22:13:14,900 [myid:] - INFO [Thread-192:FourLetterWordMain@43] - connecting to 127.0.0.1 11251 2017-01-24 22:13:14,900 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11251:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:59828 2017-01-24 22:13:14,901 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11251:NIOServerCnxn@821] - Processing stat command from /127.0.0.1:59828 2017-01-24 22:13:14,901 [myid:1] - INFO [Thread-201:NIOServerCnxn$StatCommand@655] - Stat command output 2017-01-24 22:13:14,902 [myid:1] - INFO [Thread-201:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:59828 (no session established for client) 2017-01-24 22:13:14,902 [myid:] - INFO [Thread-192:ZooKeeper@438] - Initiating client connection, connectString=127.0.0.1:11248,127.0.0.1:11251 sessionTimeout=30000 watcher=org.apache.zookeeper.test.ClientBase$CountdownWatcher@76e543fa 2017-01-24 22:13:14,903 [myid:] - WARN [Thread-192-SendThread(localhost:11251):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:14,903 [myid:] - INFO [Thread-192-SendThread(localhost:11251):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11251 2017-01-24 22:13:14,903 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11251:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:59829 2017-01-24 22:13:14,903 [myid:] - INFO [Thread-192-SendThread(localhost:11251):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:59829, server: localhost/127.0.0.1:11251 2017-01-24 22:13:14,904 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11251:ZooKeeperServer@839] - Client attempting to establish new session at /127.0.0.1:59829 2017-01-24 22:13:14,904 [myid:1] - INFO [SyncThread:1:FileTxnLog@199] - Creating new log file: log.100000001 2017-01-24 22:13:14,904 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:Follower@119] - Got zxid 0x100000001 expected 0x1 2017-01-24 22:13:14,905 [myid:0] - INFO [SyncThread:0:FileTxnLog@199] - Creating new log file: log.100000001 2017-01-24 22:13:14,907 [myid:1] - INFO [CommitProcessor:1:ZooKeeperServer@595] - Established session 0x159d441bb130000 with negotiated timeout 30000 for client /127.0.0.1:59829 2017-01-24 22:13:14,907 [myid:] - INFO [Thread-192-SendThread(localhost:11251):ClientCnxn$SendThread@1235] - Session establishment complete on server localhost/127.0.0.1:11251, sessionid = 0x159d441bb130000, negotiated timeout = 30000 2017-01-24 22:13:14,910 [myid:1] - INFO [ProcessThread(sid:1 cport:-1)::PrepRequestProcessor@494] - Processed session termination for sessionid: 0x159d441bb130000 2017-01-24 22:13:14,912 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11251:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:59829 which had sessionid 0x159d441bb130000 2017-01-24 22:13:14,912 [myid:] - INFO [Thread-192:ZooKeeper@684] - Session: 0x159d441bb130000 closed 2017-01-24 22:13:14,912 [myid:] - INFO [Thread-192:JUnit4ZKTestRunner$LoggedInvokeMethod@55] - Memory used 33263 2017-01-24 22:13:14,912 [myid:] - INFO [Thread-192:JUnit4ZKTestRunner$LoggedInvokeMethod@60] - Number of threads 85 2017-01-24 22:13:14,912 [myid:] - INFO [Thread-192:JUnit4ZKTestRunner$LoggedInvokeMethod@65] - FINISHED TEST METHOD testAuthLearnerServer 2017-01-24 22:13:14,913 [myid:] - INFO [main:QuorumBase@314] - Shutting down quorum peer QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248 2017-01-24 22:13:14,913 [myid:] - INFO [Thread-192-EventThread:ClientCnxn$EventThread@512] - EventThread shut down 2017-01-24 22:13:14,913 [myid:] - INFO [main:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:896) at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:315) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$TestQPMain.shutdown(QuorumPeerTestBase.java:59) at org.apache.zookeeper.server.quorum.QuorumPeerTestBase$MainThread.shutdown(QuorumPeerTestBase.java:152) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdown(QuorumAuthTestBase.java:138) at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.shutdownAll(QuorumAuthTestBase.java:131) at org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.tearDown(QuorumAuthUpgradeTest.java:68) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033) 2017-01-24 22:13:14,913 [myid:] - INFO [main:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:13:14,913 [myid:] - INFO [main:ZooKeeperServer@419] - shutting down 2017-01-24 22:13:14,914 [myid:] - INFO [main:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:13:14,914 [myid:] - INFO [main:CommitProcessor@181] - Shutting down 2017-01-24 22:13:14,914 [myid:] - INFO [main:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:13:14,914 [myid:0] - INFO [FollowerRequestProcessor:0:FollowerRequestProcessor@95] - FollowerRequestProcessor exited loop! 2017-01-24 22:13:14,914 [myid:0] - INFO [CommitProcessor:0:CommitProcessor@150] - CommitProcessor exited loop! 2017-01-24 22:13:14,914 [myid:] - INFO [main:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:13:14,915 [myid:0] - INFO [SyncThread:0:SyncRequestProcessor@155] - SyncRequestProcessor exited! 2017-01-24 22:13:14,915 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11248:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method 2017-01-24 22:13:14,916 [myid:0] - ERROR [localhost/127.0.0.1:11250:QuorumCnxManager$Listener@715] - Exception while listening java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:696) 2017-01-24 22:13:14,917 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@980] - Connection broken for id 0, my id = 1, error = java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:13:14,917 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@980] - Connection broken for id 1, my id = 0, error = java.net.SocketException: Socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:965) 2017-01-24 22:13:14,917 [myid:0] - WARN [RecvWorker:1:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:13:14,917 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:13:14,917 [myid:1] - WARN [RecvWorker:0:QuorumCnxManager$RecvWorker@983] - Interrupting SendWorker 2017-01-24 22:13:14,918 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@896] - Interrupted while waiting for message on queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1049) at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:73) at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:884) 2017-01-24 22:13:14,918 [myid:0] - WARN [SendWorker:1:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:13:14,919 [myid:1] - WARN [SendWorker:0:QuorumCnxManager$SendWorker@906] - Send worker leaving thread 2017-01-24 22:13:14,920 [myid:] - INFO [main:QuorumBase@318] - Shutting down leader election QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248 2017-01-24 22:13:14,920 [myid:] - INFO [main:QuorumBase@323] - Waiting for QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248 to exit thread 2017-01-24 22:13:15,080 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumCnxManager@559] - Cannot open channel to 0 at election address localhost/127.0.0.1:11229 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:13:15,080 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:QuorumCnxManager@559] - Cannot open channel to 2 at election address localhost/127.0.0.1:11235 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:579) at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:769) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:827) 2017-01-24 22:13:15,081 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11230:FastLeaderElection@778] - Notification time out: 6400 2017-01-24 22:13:15,167 [myid:0] - INFO [WorkerSender[myid=0]:FastLeaderElection$Messenger$WorkerSender@370] - WorkerSender is down 2017-01-24 22:13:15,167 [myid:0] - INFO [WorkerReceiver[myid=0]:FastLeaderElection$Messenger$WorkerReceiver@340] - WorkerReceiver is down 2017-01-24 22:13:15,664 [myid:] - WARN [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:15,664 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11230 2017-01-24 22:13:15,664 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:52580 2017-01-24 22:13:15,664 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /127.0.0.1:52580, server: localhost/127.0.0.1:11230 2017-01-24 22:13:15,665 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running 2017-01-24 22:13:15,665 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11230:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:52580 (no session established for client) 2017-01-24 22:13:15,665 [myid:] - INFO [Thread-12-SendThread(localhost:11230):ClientCnxn$SendThread@1098] - Unable to read additional data from server sessionid 0x159d440f0ed0000, likely server has closed socket, closing socket connection and attempting reconnect 2017-01-24 22:13:15,917 [myid:0] - INFO [localhost/127.0.0.1:11250:QuorumCnxManager$Listener@728] - Leaving listener 2017-01-24 22:13:16,602 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@957] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/data/jenkins/workspace/CDH5-ZooKeeper-3.4.5-JDK7/build/test/tmp/test2999887027410032136.junit.dir/jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-01-24 22:13:16,602 [myid:] - INFO [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:11233 2017-01-24 22:13:16,602 [myid:] - WARN [Thread-12-SendThread(localhost:11233):ClientCnxn$SendThread@1102] - Session 0x159d440f0ed0000 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2017-01-24 22:13:16,651 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:Follower@167] - shutdown called java.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:855) 2017-01-24 22:13:16,651 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:FollowerZooKeeperServer@139] - Shutting down 2017-01-24 22:13:16,651 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:ZooKeeperServer@419] - shutting down 2017-01-24 22:13:16,651 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:FollowerRequestProcessor@105] - Shutting down 2017-01-24 22:13:16,651 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:CommitProcessor@181] - Shutting down 2017-01-24 22:13:16,652 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:FinalRequestProcessor@415] - shutdown of request processor complete 2017-01-24 22:13:16,652 [myid:0] - INFO [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:SyncRequestProcessor@175] - Shutting down 2017-01-24 22:13:16,652 [myid:0] - WARN [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:11248:QuorumPeer@879] - QuorumPeer main thread exited 2017-01-24 22:13:16,653 [myid:] - INFO [main:ZKTestCase$1@60] - SUCCEEDED testAuthLearnerServer 2017-01-24 22:13:16,654 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testAuthLearnerServer {noformat}
    via by Michael Han,
  • Error installing VSA | VMware Communities
    via by Unknown author,
  • GitHub comment 24#251897907
    via GitHub by wittyResry
    ,
    • java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539)

    Users with the same issue

    gamebusterz
    1 times, last one,
    mauritius
    27 times, last one,
    Unknown visitor1 times, last one,
    Unknown visitor2 times, last one,
    Unknown visitor1 times, last one,
    14 more bugmates