java.lang.RuntimeException

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • When deploying to a nimbus a topology which is larger in size >17MB, we get an exception. In storm 0.9.3 this could be mitigated by using the following config on the storm.yaml to increse the buffer size to handle the topology size. i.e. 50MB would be nimbus.thrift.max_buffer_size: 50000000 This configuration does not resolve the issue in the master branch of storm and we cannot deploy topologies which are large in size. Here is the log on the client side when attempting to deploy to the nimbus node: java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:251) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:272) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:155) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at com.trustwave.siem.storm.topology.deployer.TopologyDeployer.deploy(TopologyDeployer.java:149) [siem-ng-storm-deployer-cloud.jar:] at com.trustwave.siem.storm.topology.deployer.TopologyDeployer.main(TopologyDeployer.java:87) [siem-ng-storm-deployer-cloud.jar:] Caused by: org.apache.thrift7.transport.TTransportException at org.apache.thrift7.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TTransport.readAll(TTransport.java:86) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TFramedTransport.readFrame(TFramedTransport.java:129) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TFramedTransport.read(TFramedTransport.java:101) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TTransport.readAll(TTransport.java:86) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:77) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.generated.Nimbus$Client.recv_submitTopology(Nimbus.java:238) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:222) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:237) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] ... 4 more Here is the log on the server side (nimbus.log): 2016-01-13 10:48:07.206 o.a.s.d.nimbus [INFO] Cleaning inbox ... deleted: stormjar-c8666220-fa19-426b-a7e4-c62dfb57f1f0.jar 2016-01-13 10:55:09.823 o.a.s.d.nimbus [INFO] Uploading file from client to /var/storm-data/nimbus/inbox/stormjar-80ecdf05-6a25-4281-8c78-10062ac5e396.jar 2016-01-13 10:55:11.910 o.a.s.d.nimbus [INFO] Finished uploading file from client: /var/storm-data/nimbus/inbox/stormjar-80ecdf05-6a25-4281-8c78-10062ac5e396.jar 2016-01-13 10:55:12.084 o.a.t.s.AbstractNonblockingServer$FrameBuffer [WARN] Exception while invoking! org.apache.thrift7.transport.TTransportException: Frame size (17435758) larger than max length (16384000)! at org.apache.thrift7.transport.TFramedTransport.readFrame(TFramedTransport.java:137) at org.apache.thrift7.transport.TFramedTransport.read(TFramedTransport.java:101) at org.apache.thrift7.transport.TTransport.readAll(TTransport.java:86) at org.apache.thrift7.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) at org.apache.thrift7.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) at org.apache.thrift7.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) at org.apache.thrift7.TBaseProcessor.process(TBaseProcessor.java:27) at org.apache.storm.security.auth.SimpleTransportPlugin$SimpleWrapProcessor.process(SimpleTransportPlugin.java:158) at org.apache.thrift7.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518) at org.apache.thrift7.server.Invocation.run(Invocation.java:18) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
    via by Rudra Sharma,
  • When deploying to a nimbus a topology which is larger in size >17MB, we get an exception. In storm 0.9.3 this could be mitigated by using the following config on the storm.yaml to increse the buffer size to handle the topology size. i.e. 50MB would be nimbus.thrift.max_buffer_size: 50000000 This configuration does not resolve the issue in the master branch of storm and we cannot deploy topologies which are large in size. Here is the log on the client side when attempting to deploy to the nimbus node: java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:251) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:272) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:155) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at com.trustwave.siem.storm.topology.deployer.TopologyDeployer.deploy(TopologyDeployer.java:149) [siem-ng-storm-deployer-cloud.jar:] at com.trustwave.siem.storm.topology.deployer.TopologyDeployer.main(TopologyDeployer.java:87) [siem-ng-storm-deployer-cloud.jar:] Caused by: org.apache.thrift7.transport.TTransportException at org.apache.thrift7.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TTransport.readAll(TTransport.java:86) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TFramedTransport.readFrame(TFramedTransport.java:129) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TFramedTransport.read(TFramedTransport.java:101) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TTransport.readAll(TTransport.java:86) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:77) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.generated.Nimbus$Client.recv_submitTopology(Nimbus.java:238) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:222) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:237) ~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] ... 4 more Here is the log on the server side (nimbus.log): 2016-01-13 10:48:07.206 o.a.s.d.nimbus [INFO] Cleaning inbox ... deleted: stormjar-c8666220-fa19-426b-a7e4-c62dfb57f1f0.jar 2016-01-13 10:55:09.823 o.a.s.d.nimbus [INFO] Uploading file from client to /var/storm-data/nimbus/inbox/stormjar-80ecdf05-6a25-4281-8c78-10062ac5e396.jar 2016-01-13 10:55:11.910 o.a.s.d.nimbus [INFO] Finished uploading file from client: /var/storm-data/nimbus/inbox/stormjar-80ecdf05-6a25-4281-8c78-10062ac5e396.jar 2016-01-13 10:55:12.084 o.a.t.s.AbstractNonblockingServer$FrameBuffer [WARN] Exception while invoking! org.apache.thrift7.transport.TTransportException: Frame size (17435758) larger than max length (16384000)! at org.apache.thrift7.transport.TFramedTransport.readFrame(TFramedTransport.java:137) at org.apache.thrift7.transport.TFramedTransport.read(TFramedTransport.java:101) at org.apache.thrift7.transport.TTransport.readAll(TTransport.java:86) at org.apache.thrift7.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) at org.apache.thrift7.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) at org.apache.thrift7.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) at org.apache.thrift7.TBaseProcessor.process(TBaseProcessor.java:27) at org.apache.storm.security.auth.SimpleTransportPlugin$SimpleWrapProcessor.process(SimpleTransportPlugin.java:158) at org.apache.thrift7.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518) at org.apache.thrift7.server.Invocation.run(Invocation.java:18) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
    via by Rudra Sharma,
    • java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:251)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:272)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:155)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at com.trustwave.siem.storm.topology.deployer.TopologyDeployer.deploy(TopologyDeployer.java:149)[siem-ng-storm-deployer-cloud.jar:] at com.trustwave.siem.storm.topology.deployer.TopologyDeployer.main(TopologyDeployer.java:87)[siem-ng-storm-deployer-cloud.jar:] Caused by: org.apache.thrift7.transport.TTransportException at org.apache.thrift7.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TTransport.readAll(TTransport.java:86)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TFramedTransport.readFrame(TFramedTransport.java:129)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TFramedTransport.read(TFramedTransport.java:101)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.transport.TTransport.readAll(TTransport.java:86)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:77)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.generated.Nimbus$Client.recv_submitTopology(Nimbus.java:238)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:222)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:237)[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT] ... 4 more

    Users with the same issue

    Unknown visitor
    Unknown visitor1 times, last one,
    Unknown visitor
    Unknown visitor1 times, last one,
    Unknown visitor
    Unknown visitor2 times, last one,
    Unknown visitor
    Unknown visitor1 times, last one,