org.apache.thrift.transport.TTransportException

If you like a tip written by other Samebug users, mark is as helpful! Marks help our algorithm provide you better solutions and also help other users.
tip

It's possible you're trying to write to a connection that's already closed. Another cause for this is that you closed the socket with unread data in the socket receive buffer.


rafael
tip

This might be caused by unmatching versions of SSL. Java starts normally with SSLv2 and your server might not be able to negotiate. You have to force Java to use SSLv3. See this comment: https://goo.gl/hx0YB3


rafael

You have a different solution? A short tip here would help you and many other users who saw this issue last week.

  • Periodically, the tests won't be able to create a socket to talk to one another. Since we don't have retry logic, this causes threads to fail and tests to fail. Below is from the logs of running the unit tests. Heartbeat thread dies {code} 2014-10-15 20:52:45,477 INFO (TachyonWorker.java:start) - The worker server ends @ NetAddress(mHost:build4-centos65-gpcloud-w-docker, mPort:37631, mSecondaryPort:59844) 2014-10-15 20:52:45,665 ERROR (HeartbeatThread.java:run) - Uncaught exception in heartbeat executor, Heartbeat Thread shutting down java.lang.RuntimeException: java.io.IOException: org.apache.thrift.transport.TTransportException at com.google.common.base.Throwables.propagate(Throwables.java:160) at tachyon.worker.WorkerClientHeartbeatExecutor.heartbeat(WorkerClientHeartbeatExecutor.java:27) at tachyon.HeartbeatThread.run(HeartbeatThread.java:37) Caused by: java.io.IOException: org.apache.thrift.transport.TTransportException at tachyon.worker.WorkerClient.userHeartbeat(WorkerClient.java:404) at tachyon.worker.WorkerClientHeartbeatExecutor.heartbeat(WorkerClientHeartbeatExecutor.java:25) ... 1 more Caused by: org.apache.thrift.transport.TTransportException at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129) at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:362) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:284) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:191) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at tachyon.thrift.WorkerService$Client.recv_userHeartbeat(WorkerService.java:393) at tachyon.thrift.WorkerService$Client.userHeartbeat(WorkerService.java:380) at tachyon.worker.WorkerClient.userHeartbeat(WorkerClient.java:402 {code} {code} 2014-10-15 20:54:08,701 INFO (TachyonMaster.java:start) - Tachyon Master version 0.6.0-SNAPSHOT ended @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:08,701 ERROR (MasterClient.java:worker_heartbeat) - java.net.SocketException: Connection reset org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129) at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:362) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:284) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:191) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at tachyon.thrift.MasterService$Client.recv_worker_heartbeat(MasterService.java:356) at tachyon.thrift.MasterService$Client.worker_heartbeat(MasterService.java:341) at tachyon.master.MasterClient.worker_heartbeat(MasterClient.java:736) at tachyon.worker.WorkerStorage.heartbeat(WorkerStorage.java:599) at tachyon.worker.TachyonWorker.run(TachyonWorker.java:245) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:196) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read1(BufferedInputStream.java:275) at java.io.BufferedInputStream.read(BufferedInputStream.java:334) at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) ... 14 more 2014-10-15 20:54:08,703 INFO (MasterClient.java:connect) - Tachyon client (version 0.6.0-SNAPSHOT) is trying to connect master @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:08,704 ERROR (MasterClient.java:connect) - Failed to connect (1) to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 : java.net.ConnectException: Connection refused 2014-10-15 20:54:09,705 INFO (MasterClient.java:connect) - Tachyon client (version 0.6.0-SNAPSHOT) is trying to connect master @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:09,706 ERROR (MasterClient.java:connect) - Failed to connect (2) to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 : java.net.ConnectException: Connection refused 2014-10-15 20:54:10,706 INFO (MasterClient.java:connect) - Tachyon client (version 0.6.0-SNAPSHOT) is trying to connect master @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:10,707 ERROR (MasterClient.java:connect) - Failed to connect (3) to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 : java.net.ConnectException: Connection refused 2014-10-15 20:54:11,707 INFO (MasterClient.java:connect) - Tachyon client (version 0.6.0-SNAPSHOT) is trying to connect master @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:11,708 ERROR (MasterClient.java:connect) - Failed to connect (4) to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 : java.net.ConnectException: Connection refused 2014-10-15 20:54:12,708 INFO (MasterClient.java:connect) - Tachyon client (version 0.6.0-SNAPSHOT) is trying to connect master @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:12,709 ERROR (MasterClient.java:connect) - Failed to connect (5) to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 : java.net.ConnectException: Connection refused 2014-10-15 20:54:13,710 ERROR (TachyonWorker.java:run) - Failed to connect to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 after 5 attempts java.io.IOException: Failed to connect to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 after 5 attempts at tachyon.master.MasterClient.connect(MasterClient.java:188) at tachyon.master.MasterClient.worker_heartbeat(MasterClient.java:733) at tachyon.worker.WorkerStorage.heartbeat(WorkerStorage.java:599) at tachyon.worker.TachyonWorker.run(TachyonWorker.java:245) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift.transport.TSocket.open(TSocket.java:185) at org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81) at tachyon.master.MasterClient.connect(MasterClient.java:156) ... 4 more Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.thrift.transport.TSocket.open(TSocket.java:180) ... 6 more {code}
    via by David Capwell,
  • Periodically, the tests won't be able to create a socket to talk to one another. Since we don't have retry logic, this causes threads to fail and tests to fail. Below is from the logs of running the unit tests. Heartbeat thread dies {code} 2014-10-15 20:52:45,477 INFO (TachyonWorker.java:start) - The worker server ends @ NetAddress(mHost:build4-centos65-gpcloud-w-docker, mPort:37631, mSecondaryPort:59844) 2014-10-15 20:52:45,665 ERROR (HeartbeatThread.java:run) - Uncaught exception in heartbeat executor, Heartbeat Thread shutting down java.lang.RuntimeException: java.io.IOException: org.apache.thrift.transport.TTransportException at com.google.common.base.Throwables.propagate(Throwables.java:160) at tachyon.worker.WorkerClientHeartbeatExecutor.heartbeat(WorkerClientHeartbeatExecutor.java:27) at tachyon.HeartbeatThread.run(HeartbeatThread.java:37) Caused by: java.io.IOException: org.apache.thrift.transport.TTransportException at tachyon.worker.WorkerClient.userHeartbeat(WorkerClient.java:404) at tachyon.worker.WorkerClientHeartbeatExecutor.heartbeat(WorkerClientHeartbeatExecutor.java:25) ... 1 more Caused by: org.apache.thrift.transport.TTransportException at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129) at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:362) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:284) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:191) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at tachyon.thrift.WorkerService$Client.recv_userHeartbeat(WorkerService.java:393) at tachyon.thrift.WorkerService$Client.userHeartbeat(WorkerService.java:380) at tachyon.worker.WorkerClient.userHeartbeat(WorkerClient.java:402 {code} {code} 2014-10-15 20:54:08,701 INFO (TachyonMaster.java:start) - Tachyon Master version 0.6.0-SNAPSHOT ended @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:08,701 ERROR (MasterClient.java:worker_heartbeat) - java.net.SocketException: Connection reset org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129) at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:362) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:284) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:191) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at tachyon.thrift.MasterService$Client.recv_worker_heartbeat(MasterService.java:356) at tachyon.thrift.MasterService$Client.worker_heartbeat(MasterService.java:341) at tachyon.master.MasterClient.worker_heartbeat(MasterClient.java:736) at tachyon.worker.WorkerStorage.heartbeat(WorkerStorage.java:599) at tachyon.worker.TachyonWorker.run(TachyonWorker.java:245) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:196) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read1(BufferedInputStream.java:275) at java.io.BufferedInputStream.read(BufferedInputStream.java:334) at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) ... 14 more 2014-10-15 20:54:08,703 INFO (MasterClient.java:connect) - Tachyon client (version 0.6.0-SNAPSHOT) is trying to connect master @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:08,704 ERROR (MasterClient.java:connect) - Failed to connect (1) to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 : java.net.ConnectException: Connection refused 2014-10-15 20:54:09,705 INFO (MasterClient.java:connect) - Tachyon client (version 0.6.0-SNAPSHOT) is trying to connect master @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:09,706 ERROR (MasterClient.java:connect) - Failed to connect (2) to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 : java.net.ConnectException: Connection refused 2014-10-15 20:54:10,706 INFO (MasterClient.java:connect) - Tachyon client (version 0.6.0-SNAPSHOT) is trying to connect master @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:10,707 ERROR (MasterClient.java:connect) - Failed to connect (3) to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 : java.net.ConnectException: Connection refused 2014-10-15 20:54:11,707 INFO (MasterClient.java:connect) - Tachyon client (version 0.6.0-SNAPSHOT) is trying to connect master @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:11,708 ERROR (MasterClient.java:connect) - Failed to connect (4) to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 : java.net.ConnectException: Connection refused 2014-10-15 20:54:12,708 INFO (MasterClient.java:connect) - Tachyon client (version 0.6.0-SNAPSHOT) is trying to connect master @ build4-centos65-gpcloud-w-docker/172.17.0.5:52447 2014-10-15 20:54:12,709 ERROR (MasterClient.java:connect) - Failed to connect (5) to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 : java.net.ConnectException: Connection refused 2014-10-15 20:54:13,710 ERROR (TachyonWorker.java:run) - Failed to connect to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 after 5 attempts java.io.IOException: Failed to connect to master build4-centos65-gpcloud-w-docker/172.17.0.5:52447 after 5 attempts at tachyon.master.MasterClient.connect(MasterClient.java:188) at tachyon.master.MasterClient.worker_heartbeat(MasterClient.java:733) at tachyon.worker.WorkerStorage.heartbeat(WorkerStorage.java:599) at tachyon.worker.TachyonWorker.run(TachyonWorker.java:245) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift.transport.TSocket.open(TSocket.java:185) at org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81) at tachyon.master.MasterClient.connect(MasterClient.java:156) ... 4 more Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.thrift.transport.TSocket.open(TSocket.java:180) ... 6 more {code}
    via by David Capwell,
  • Hive metastore not working in HDInsight 3.0
    via by Unknown author,
  • Hive metastore not working in HDInsight 3.0
    via by Unknown author,
  • MongoDB concurrency problem
    via by JunKai Wu,
  • Sending custom message using channel
    via by rahul singh,
    • org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129) at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:362) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:284) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:191) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at tachyon.thrift.MasterService$Client.recv_worker_heartbeat(MasterService.java:356) at tachyon.thrift.MasterService$Client.worker_heartbeat(MasterService.java:341) at tachyon.master.MasterClient.worker_heartbeat(MasterClient.java:736) at tachyon.worker.WorkerStorage.heartbeat(WorkerStorage.java:599) at tachyon.worker.TachyonWorker.run(TachyonWorker.java:245) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:196) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read1(BufferedInputStream.java:275) at java.io.BufferedInputStream.read(BufferedInputStream.java:334) at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) ... 14 more

    Users with the same issue

    Unknown visitor1 times, last one,
    Unknown visitor1 times, last one,
    Nikolay Rybak
    1 times, last one,
    mauritius
    2 times, last one,
    Unknown visitor1 times, last one,
    195 more bugmates