com.datastax.driver.core.TransportException

tip

This was a bug at HBase 1.0.1, it has been patched, so an update to 1.0.2 or further should solve the problem. URL for the Jira thread: https://goo.gl/c5oa6f

You have a different solution? A short tip here would help you and many other users who saw this issue last week.

  • If a data-store node is down when the connection pool code tries to connect to it, naturally an exception is thrown by the networking code. The connection pool code however logs this exception as an ERROR with a stack-trace. This is questionable behaviour: * Stack-traces should be logged for exceptions that indicate program logic errors, and only logic errors, as they are useful to only programmers, and not useful to computer operators. * C* is designed to cope with data-store nodes that go down. A Data-store node being down is not an error condition, but is a warning condition; C* may still (with suitable replication and consistency-level settings) satisfy queries despite the node being down. Here is an example log message: {noformat} ERROR 2015-05-27 15:29:01,585 Error creating pool to /172.16.12.204:9042 com.datastax.driver.core.TransportException: [/172.16.12.204:9042] Cannot connect at com.datastax.driver.core.Connection.<init>(Connection.java:109) ~[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:32) ~[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.Connection$Factory.open(Connection.java:586) ~[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.SingleConnectionPool.<init>(SingleConnectionPool.java:76) ~[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.HostConnectionPool.newInstance(HostConnectionPool.java:35) ~[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.SessionManager.replacePool(SessionManager.java:271) ~[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.SessionManager.access$400(SessionManager.java:40) ~[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:308) [cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:300) [cassandra-driver-core-2.1.5.jar:na] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_75] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75] Caused by: java.net.ConnectException: Connection refused: /172.16.12.204:9042 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_75] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) ~[na:1.7.0_75] at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) ~[netty-3.9.0.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) ~[netty-3.9.0.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) ~[netty-3.9.0.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) ~[netty-3.9.0.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ~[netty-3.9.0.Final.jar:na] ... 3 common frames omitted {noformat}
    via by Benedict Adamson,
  • GitHub comment 62#144899449
    via GitHub by ziploc1010
    ,
  • This seems to be a cosmetic issue where the debug changes into an error. This happens during a reconnect when one of the cassandra nodes are just in the startup phase. This seems to happen around the the time cassandra jumps it status to normal. I was able to reproduce this by just restarting cassandra, this was tested with 4 cassandra nodes. [App.log] 2015-07-22 13:57:02,479 [ew I/O boss #56] DEBUG [com.datastax.driver.core.Connection] Defuncting connection to /169.254.100.1:9042 com.datastax.driver.core.TransportException: [/169.254.100.1:9042] Unexpected exception triggered (java.net.ConnectException: Connection refused: /169.254.100.1:9042) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:691) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:60) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525) at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:110) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.ConnectException: Connection refused: /169.254.100.1:9042 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ... 3 more 2015-07-22 13:57:02,479 [ tasks worker-3] ERROR [com.datastax.driver.core.Session] Error creating pool to /169.254.100.1:9042 com.datastax.driver.core.TransportException: [/169.254.100.1:9042] Cannot connect at com.datastax.driver.core.Connection.<init>(Connection.java:106) at com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:32) at com.datastax.driver.core.Connection$Factory.open(Connection.java:521) at com.datastax.driver.core.DynamicConnectionPool.<init>(DynamicConnectionPool.java:73) at com.datastax.driver.core.HostConnectionPool.newInstance(HostConnectionPool.java:33) at com.datastax.driver.core.SessionManager$2.call(SessionManager.java:199) at com.datastax.driver.core.SessionManager$2.call(SessionManager.java:192) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.ConnectException: Connection refused: /169.254.100.1:9042 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ... 3 more 2015-07-22 13:57:02,479 [river worker-18] DEBUG [com.datastax.driver.core.Cluster] Host /169.254.100.1:9042 is DOWN [...] 2015-07-22 13:57:17,554 [ Reconnection-0] DEBUG [com.datastax.driver.core.Cluster] Successful reconnection to /169.254.100.1:9042, setting host UP 2015-07-22 13:57:17,554 [ Reconnection-0] DEBUG [com.datastax.driver.core.Cluster] Host /169.254.100.1:9042 is UP 2015-07-22 13:57:17,554 [ Reconnection-0] DEBUG [com.datastax.driver.core.Cluster] Cancelling reconnection attempt since node is UP 2015-07-22 13:57:17,565 [ tasks worker-4] DEBUG [com.datastax.driver.core.Connection] Connection[/169.254.100.1:9042-17, inFlight=0, closed=false] Transport initialized and ready 2015-07-22 13:57:17,574 [ tasks worker-4] DEBUG [com.datastax.driver.core.Connection] Connection[/169.254.100.1:9042-18, inFlight=0, closed=false] Transport initialized and ready 2015-07-22 13:57:17,574 [ tasks worker-4] DEBUG [com.datastax.driver.core.Session] Added connection pool for /169.254.100.1:9042 [Cassandra system.log] INFO [main] 2015-07-22 13:57:02,389 ColumnFamilyStore.java (line 795) Enqueuing flush of Memtable-local@520527297(85/850 serialized/live bytes, 4 ops) INFO [FlushWriter:1] 2015-07-22 13:57:02,390 Memtable.java (line 362) Writing Memtable-local@520527297(85/850 serialized/live bytes, 4 ops) INFO [HANDSHAKE-/169.254.100.2] 2015-07-22 13:57:02,438 OutboundTcpConnection.java (line 389) Handshaking version with /169.254.100.2 INFO [HANDSHAKE-/169.254.100.3] 2015-07-22 13:57:02,439 OutboundTcpConnection.java (line 389) Handshaking version with /169.254.100.3 INFO [HANDSHAKE-/169.254.100.4] 2015-07-22 13:57:02,455 OutboundTcpConnection.java (line 389) Handshaking version with /169.254.100.4 INFO [GossipStage:1] 2015-07-22 13:57:02,462 Gossiper.java (line 981) Node /169.254.100.4 has restarted, now UP INFO [GossipStage:1] 2015-07-22 13:57:02,467 StorageService.java (line 1530) Node /169.254.100.4 state jump to normal INFO [RequestResponseStage:1] 2015-07-22 13:57:02,480 Gossiper.java (line 948) InetAddress /169.254.100.4 is now UP INFO [GossipStage:1] 2015-07-22 13:57:02,518 Gossiper.java (line 981) Node /169.254.100.2 has restarted, now UP INFO [RequestResponseStage:2] 2015-07-22 13:57:02,519 Gossiper.java (line 948) InetAddress /169.254.100.2 is now UP INFO [GossipStage:1] 2015-07-22 13:57:02,520 StorageService.java (line 1530) Node /169.254.100.2 state jump to normal INFO [GossipStage:1] 2015-07-22 13:57:02,554 Gossiper.java (line 981) Node /169.254.100.3 has restarted, now UP INFO [RequestResponseStage:3] 2015-07-22 13:57:02,555 Gossiper.java (line 948) InetAddress /169.254.100.3 is now UP INFO [GossipStage:1] 2015-07-22 13:57:02,556 StorageService.java (line 1530) Node /169.254.100.3 state jump to normal INFO [GossipStage:1] 2015-07-22 13:57:02,591 StorageService.java (line 1530) Node /169.254.100.4 state jump to normal INFO [FlushWriter:1] 2015-07-22 13:57:02,592 Memtable.java (line 402) Completed flushing /var/cassandra/data/system/local/system-local-jb-135-Data.db (116 bytes) for commitlog position ReplayPosition(segmentId=1437566219069, position=560581) INFO [main] 2015-07-22 13:57:02,613 ColumnFamilyStore.java (line 795) Enqueuing flush of Memtable-local@1314647135(10081/100810 serialized/live bytes, 259 ops) INFO [FlushWriter:1] 2015-07-22 13:57:02,614 Memtable.java (line 362) Writing Memtable-local@1314647135(10081/100810 serialized/live bytes, 259 ops) INFO [FlushWriter:1] 2015-07-22 13:57:02,880 Memtable.java (line 402) Completed flushing /var/cassandra/data/system/local/system-local-jb-136-Data.db (5272 bytes) for commitlog position ReplayPosition(segmentId=1437566219069, position=613704) INFO [CompactionExecutor:4] 2015-07-22 13:57:02,881 CompactionTask.java (line 120) Compacting [SSTableReader(path='/var/cassandra/data/system/local/system-local-jb-135-Data.db'), SSTableReader(path='/var/cassandra/data/system/local/system-local-jb-136-Data.db'), SSTableR eader(path='/var/cassandra/data/system/local/system-local-jb-134-Data.db'), SSTableReader(path='/var/cassandra/data/system/local/system-local-jb-133-Data.db')] INFO [main] 2015-07-22 13:57:02,897 StorageService.java (line 1530) Node /169.254.100.1 state jump to normal INFO [main] 2015-07-22 13:57:02,909 CassandraDaemon.java (line 614) Waiting for gossip to settle before accepting client requests... INFO [CompactionExecutor:4] 2015-07-22 13:57:03,567 CompactionTask.java (line 299) Compacted 4 sstables to [/var/cassandra/data/system/local/system-local-jb-137,]. 11,355 bytes to 5,724 (~50% of original) in 684ms = 0.007981MB/s. 4 total partitions merged to 1. Partit ion merge counts were {4:1, } INFO [main] 2015-07-22 13:57:10,910 CassandraDaemon.java (line 646) No gossip backlog; proceeding INFO [main] 2015-07-22 13:57:10,991 Server.java (line 158) Starting listening for CQL clients on /169.254.100.1:9042... INFO [main] 2015-07-22 13:57:11,043 ThriftServer.java (line 99) Using TFramedTransport with a max frame size of 15728640 bytes. INFO [main] 2015-07-22 13:57:11,045 ThriftServer.java (line 118) Binding thrift service to /169.254.100.1:6160 INFO [main] 2015-07-22 13:57:11,053 TServerCustomFactory.java (line 47) Using synchronous/threadpool thrift server on node1 : 6160 INFO [Thread-2] 2015-07-22 13:57:11,054 ThriftServer.java (line 135) Listening for thrift clients...
    via by Filip Druvholm,
  • This seems to be a cosmetic issue where the debug changes into an error. This happens during a reconnect when one of the cassandra nodes are just in the startup phase. This seems to happen around the the time cassandra jumps it status to normal. I was able to reproduce this by just restarting cassandra, this was tested with 4 cassandra nodes. [App.log] 2015-07-22 13:57:02,479 [ew I/O boss #56] DEBUG [com.datastax.driver.core.Connection] Defuncting connection to /169.254.100.1:9042 com.datastax.driver.core.TransportException: [/169.254.100.1:9042] Unexpected exception triggered (java.net.ConnectException: Connection refused: /169.254.100.1:9042) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:691) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:60) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525) at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:110) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.ConnectException: Connection refused: /169.254.100.1:9042 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ... 3 more 2015-07-22 13:57:02,479 [ tasks worker-3] ERROR [com.datastax.driver.core.Session] Error creating pool to /169.254.100.1:9042 com.datastax.driver.core.TransportException: [/169.254.100.1:9042] Cannot connect at com.datastax.driver.core.Connection.<init>(Connection.java:106) at com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:32) at com.datastax.driver.core.Connection$Factory.open(Connection.java:521) at com.datastax.driver.core.DynamicConnectionPool.<init>(DynamicConnectionPool.java:73) at com.datastax.driver.core.HostConnectionPool.newInstance(HostConnectionPool.java:33) at com.datastax.driver.core.SessionManager$2.call(SessionManager.java:199) at com.datastax.driver.core.SessionManager$2.call(SessionManager.java:192) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.ConnectException: Connection refused: /169.254.100.1:9042 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ... 3 more 2015-07-22 13:57:02,479 [river worker-18] DEBUG [com.datastax.driver.core.Cluster] Host /169.254.100.1:9042 is DOWN [...] 2015-07-22 13:57:17,554 [ Reconnection-0] DEBUG [com.datastax.driver.core.Cluster] Successful reconnection to /169.254.100.1:9042, setting host UP 2015-07-22 13:57:17,554 [ Reconnection-0] DEBUG [com.datastax.driver.core.Cluster] Host /169.254.100.1:9042 is UP 2015-07-22 13:57:17,554 [ Reconnection-0] DEBUG [com.datastax.driver.core.Cluster] Cancelling reconnection attempt since node is UP 2015-07-22 13:57:17,565 [ tasks worker-4] DEBUG [com.datastax.driver.core.Connection] Connection[/169.254.100.1:9042-17, inFlight=0, closed=false] Transport initialized and ready 2015-07-22 13:57:17,574 [ tasks worker-4] DEBUG [com.datastax.driver.core.Connection] Connection[/169.254.100.1:9042-18, inFlight=0, closed=false] Transport initialized and ready 2015-07-22 13:57:17,574 [ tasks worker-4] DEBUG [com.datastax.driver.core.Session] Added connection pool for /169.254.100.1:9042 [Cassandra system.log] INFO [main] 2015-07-22 13:57:02,389 ColumnFamilyStore.java (line 795) Enqueuing flush of Memtable-local@520527297(85/850 serialized/live bytes, 4 ops) INFO [FlushWriter:1] 2015-07-22 13:57:02,390 Memtable.java (line 362) Writing Memtable-local@520527297(85/850 serialized/live bytes, 4 ops) INFO [HANDSHAKE-/169.254.100.2] 2015-07-22 13:57:02,438 OutboundTcpConnection.java (line 389) Handshaking version with /169.254.100.2 INFO [HANDSHAKE-/169.254.100.3] 2015-07-22 13:57:02,439 OutboundTcpConnection.java (line 389) Handshaking version with /169.254.100.3 INFO [HANDSHAKE-/169.254.100.4] 2015-07-22 13:57:02,455 OutboundTcpConnection.java (line 389) Handshaking version with /169.254.100.4 INFO [GossipStage:1] 2015-07-22 13:57:02,462 Gossiper.java (line 981) Node /169.254.100.4 has restarted, now UP INFO [GossipStage:1] 2015-07-22 13:57:02,467 StorageService.java (line 1530) Node /169.254.100.4 state jump to normal INFO [RequestResponseStage:1] 2015-07-22 13:57:02,480 Gossiper.java (line 948) InetAddress /169.254.100.4 is now UP INFO [GossipStage:1] 2015-07-22 13:57:02,518 Gossiper.java (line 981) Node /169.254.100.2 has restarted, now UP INFO [RequestResponseStage:2] 2015-07-22 13:57:02,519 Gossiper.java (line 948) InetAddress /169.254.100.2 is now UP INFO [GossipStage:1] 2015-07-22 13:57:02,520 StorageService.java (line 1530) Node /169.254.100.2 state jump to normal INFO [GossipStage:1] 2015-07-22 13:57:02,554 Gossiper.java (line 981) Node /169.254.100.3 has restarted, now UP INFO [RequestResponseStage:3] 2015-07-22 13:57:02,555 Gossiper.java (line 948) InetAddress /169.254.100.3 is now UP INFO [GossipStage:1] 2015-07-22 13:57:02,556 StorageService.java (line 1530) Node /169.254.100.3 state jump to normal INFO [GossipStage:1] 2015-07-22 13:57:02,591 StorageService.java (line 1530) Node /169.254.100.4 state jump to normal INFO [FlushWriter:1] 2015-07-22 13:57:02,592 Memtable.java (line 402) Completed flushing /var/cassandra/data/system/local/system-local-jb-135-Data.db (116 bytes) for commitlog position ReplayPosition(segmentId=1437566219069, position=560581) INFO [main] 2015-07-22 13:57:02,613 ColumnFamilyStore.java (line 795) Enqueuing flush of Memtable-local@1314647135(10081/100810 serialized/live bytes, 259 ops) INFO [FlushWriter:1] 2015-07-22 13:57:02,614 Memtable.java (line 362) Writing Memtable-local@1314647135(10081/100810 serialized/live bytes, 259 ops) INFO [FlushWriter:1] 2015-07-22 13:57:02,880 Memtable.java (line 402) Completed flushing /var/cassandra/data/system/local/system-local-jb-136-Data.db (5272 bytes) for commitlog position ReplayPosition(segmentId=1437566219069, position=613704) INFO [CompactionExecutor:4] 2015-07-22 13:57:02,881 CompactionTask.java (line 120) Compacting [SSTableReader(path='/var/cassandra/data/system/local/system-local-jb-135-Data.db'), SSTableReader(path='/var/cassandra/data/system/local/system-local-jb-136-Data.db'), SSTableR eader(path='/var/cassandra/data/system/local/system-local-jb-134-Data.db'), SSTableReader(path='/var/cassandra/data/system/local/system-local-jb-133-Data.db')] INFO [main] 2015-07-22 13:57:02,897 StorageService.java (line 1530) Node /169.254.100.1 state jump to normal INFO [main] 2015-07-22 13:57:02,909 CassandraDaemon.java (line 614) Waiting for gossip to settle before accepting client requests... INFO [CompactionExecutor:4] 2015-07-22 13:57:03,567 CompactionTask.java (line 299) Compacted 4 sstables to [/var/cassandra/data/system/local/system-local-jb-137,]. 11,355 bytes to 5,724 (~50% of original) in 684ms = 0.007981MB/s. 4 total partitions merged to 1. Partit ion merge counts were {4:1, } INFO [main] 2015-07-22 13:57:10,910 CassandraDaemon.java (line 646) No gossip backlog; proceeding INFO [main] 2015-07-22 13:57:10,991 Server.java (line 158) Starting listening for CQL clients on /169.254.100.1:9042... INFO [main] 2015-07-22 13:57:11,043 ThriftServer.java (line 99) Using TFramedTransport with a max frame size of 15728640 bytes. INFO [main] 2015-07-22 13:57:11,045 ThriftServer.java (line 118) Binding thrift service to /169.254.100.1:6160 INFO [main] 2015-07-22 13:57:11,053 TServerCustomFactory.java (line 47) Using synchronous/threadpool thrift server on node1 : 6160 INFO [Thread-2] 2015-07-22 13:57:11,054 ThriftServer.java (line 135) Listening for thrift clients...
    via by Filip Druvholm,
    • com.datastax.driver.core.TransportException: [/172.16.12.204:9042] Cannot connect at com.datastax.driver.core.Connection.<init>(Connection.java:109)[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:32)[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.Connection$Factory.open(Connection.java:586)[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.SingleConnectionPool.<init>(SingleConnectionPool.java:76)[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.HostConnectionPool.newInstance(HostConnectionPool.java:35)[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.SessionManager.replacePool(SessionManager.java:271)[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.SessionManager.access$400(SessionManager.java:40)[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:308)[cassandra-driver-core-2.1.5.jar:na] at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:300)[cassandra-driver-core-2.1.5.jar:na] at java.util.concurrent.FutureTask.run(FutureTask.java:262)[na:1.7.0_75] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[na:1.7.0_75] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[na:1.7.0_75] at java.lang.Thread.run(Thread.java:745)[na:1.7.0_75] Caused by: java.net.ConnectException: Connection refused: /172.16.12.204:9042 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)[na:1.7.0_75] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)[na:1.7.0_75] at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150)[netty-3.9.0.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)[netty-3.9.0.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)[netty-3.9.0.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)[netty-3.9.0.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)[netty-3.9.0.Final.jar:na] ... 3 more

    Users with the same issue

    rp
    1 times, last one,
    bpbhat77
    55 times, last one,
    Unknown visitor1 times, last one,
    fervidnerd
    126 times, last one,
    Unknown visitor2 times, last one,
    114 more bugmates