java.lang.RuntimeException: java.lang.Exception: Thrift error for org.apache.tephra.distributed.TransactionServiceClient$2@273e2fa4: Unable to discover tx service.

Google Groups | ron cai | 3 months ago
  1. 0

    cdap-master can't be restarted

    Google Groups | 3 months ago | ron cai
    java.lang.RuntimeException: java.lang.Exception: Thrift error for org.apache.tephra.distributed.TransactionServiceClient$2@273e2fa4: Unable to discover tx service.
  2. 0

    Master seems to be failing to discover transaction service on secure clusters because the HDFS_DELEGATION_TOKEN for the cdap user cannot be found in the cache. Exception: {code} 2016-01-26T19:39:30,366Z INFO o.a.t.i.a.ApplicationMasterService [slave4.sjc1.continuuity.net] [ApplicationMasterService] ApplicationMaste rService:handleCompleted(ApplicationMasterService.java:440) - Container container_1446847897094_3208_01_001504 completed with COMPLETE:toke n (HDFS_DELEGATION_TOKEN token 6728 for cdap) can't be found in cache org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): token (HDFS_DELEGATION_TOKEN token 6728 for cdap) can't be found in cache at org.apache.hadoop.ipc.Client.call(Client.java:1468) at org.apache.hadoop.ipc.Client.call(Client.java:1399) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:752) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy11.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1982) at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1128) at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1124) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1124) at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251) at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61) at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359) at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) {code} {code} ata/datasets/app.meta user=<null>: java.lang.RuntimeException: java.lang.Exception: Thrift error for co.cask.tephra.distributed.TransactionServiceClient$2@1f176d5: Unable to discover tx service. at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[com.google.guava.guava-13.0.1.jar:na] at co.cask.tephra.distributed.TransactionServiceClient.startShort(TransactionServiceClient.java:269) ~[co.cask.tephra.tephra-core-0 .6.3.jar:na] at co.cask.cdap.data2.transaction.DistributedTransactionSystemClientService.startShort(DistributedTransactionSystemClientService.ja va:99) ~[co.cask.cdap.cdap-data-fabric-3.3.0.jar:na] at co.cask.tephra.TransactionContext.start(TransactionContext.java:89) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.tephra.DefaultTransactionExecutor.executeOnce(DefaultTransactionExecutor.java:133) ~[co.cask.tephra.tephra-core-0.6.3.ja r:na] at co.cask.tephra.DefaultTransactionExecutor.executeWithRetry(DefaultTransactionExecutor.java:115) ~[co.cask.tephra.tephra-core-0.6 .3.jar:na] at co.cask.tephra.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:72) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.tephra.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:90) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.cdap.data2.dataset2.tx.TransactionalDatasetRegistry.execute(TransactionalDatasetRegistry.java:71) ~[co.cask.cdap.cdap-da ta-fabric-3.3.0.jar:na] at co.cask.cdap.data2.dataset2.tx.TransactionalDatasetRegistry.executeUnchecked(TransactionalDatasetRegistry.java:87) ~[co.cask.cda p.cdap-data-fabric-3.3.0.jar:na] at co.cask.cdap.data2.datafabric.dataset.instance.DatasetInstanceManager.get(DatasetInstanceManager.java:62) ~[co.cask.cdap.cdap-da ta-fabric-3.3.0.jar:na] at co.cask.cdap.data2.datafabric.dataset.service.DatasetInstanceService.get(DatasetInstanceService.java:123) ~[co.cask.cdap.cdap-data-fabric-3.3.0.jar:na] at co.cask.cdap.data2.datafabric.dataset.service.DatasetInstanceHandler.get(DatasetInstanceHandler.java:83) ~[co.cask.cdap.cdap-data-fabric-3.3.0.jar:na] at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_75] at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_75] at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80) ~[co.cask.http.netty-http-0.14.0.jar:na] at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38) [co.cask.http.netty-http-0.14.0.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [io.netty.netty-3.6.6.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [io.netty.netty-3.6.6.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [io.netty.netty-3.6.6.Final.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124) [io.netty.netty-3.6.6.Final.jar:na] at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:63) [co.cask.cdap.cdap-common-3.3.0.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [io.netty.netty-3.6.6.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [io.netty.netty-3.6.6.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [io.netty.netty-3.6.6.Final.jar:na] at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) [io.netty.netty-3.6.6.Final.jar:na] at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) [io.netty.netty-3.6.6.Final.jar:na] at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) [io.netty.netty-3.6.6.Final.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75] Caused by: java.lang.Exception: Thrift error for co.cask.tephra.distributed.TransactionServiceClient$2@1f176d5: Unable to discover tx service. at co.cask.tephra.distributed.TransactionServiceClient.execute(TransactionServiceClient.java:228) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.tephra.distributed.TransactionServiceClient.execute(TransactionServiceClient.java:186) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.tephra.distributed.TransactionServiceClient.startShort(TransactionServiceClient.java:260) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] ... 30 common frames omitted Caused by: org.apache.thrift.TException: Unable to discover tx service. at co.cask.tephra.distributed.AbstractClientProvider.newClient(AbstractClientProvider.java:104) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.tephra.distributed.AbstractClientProvider.newClient(AbstractClientProvider.java:83) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.tephra.distributed.PooledClientProvider$TxClientPool.create(PooledClientProvider.java:46) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.tephra.distributed.PooledClientProvider$TxClientPool.create(PooledClientProvider.java:39) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.tephra.distributed.ElasticPool.getOrCreate(ElasticPool.java:136) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.tephra.distributed.ElasticPool.obtain(ElasticPool.java:123) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.tephra.distributed.PooledClientProvider.getCloseableClient(PooledClientProvider.java:99) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] at co.cask.tephra.distributed.TransactionServiceClient.execute(TransactionServiceClient.java:215) ~[co.cask.tephra.tephra-core-0.6.3.jar:na] ... 32 common frames omitted {code} This seems to be a recurrence of [CDAP-562|https://issues.cask.co/browse/CDAP-562] and related to [TWILL-106|https://issues.apache.org/jira/browse/TWILL-106]

    Cask Community Issue Tracker | 10 months ago | Bhooshan Mogal
    java.lang.RuntimeException: java.lang.Exception: Thrift error for co.cask.tephra.distributed.TransactionServiceClient$2@1f176d5: Unable to discover tx service.
  3. 0

    spark cannot connect to tachyon master

    Google Groups | 2 years ago | Puja Gupta
    org.apache.thrift.TException: Service name not found in message name: user_getUserId. Did you forget to use a TMultiplexProtocol in your client?
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    fail to run simple test on Spark

    Google Groups | 1 year ago | Charlie Chen
    org.apache.thrift.TException: Service name not found in message name: user_getUserId. Did you forget to use a TMultiplexProtocol in your client?
  6. 0

    [ACCUMULO-3653] Compaction with custom compaction strategy class doesn't validate specified class - ASF JIRA

    apache.org | 1 year ago
    org.apache.thrift.TException: org.apache.accumulo.tserver.session.MultiScanSession cannot be cast to org.apache.accumulo.tserver.session.ScanSession

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. org.apache.thrift.TException

      Unable to discover tx service.

      at org.apache.tephra.distributed.AbstractClientProvider.newClient()
    2. org.apache.tephra
      AbstractTransactionExecutor.executeUnchecked
      1. org.apache.tephra.distributed.AbstractClientProvider.newClient(AbstractClientProvider.java:106)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      2. org.apache.tephra.distributed.AbstractClientProvider.newClient(AbstractClientProvider.java:85)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      3. org.apache.tephra.distributed.PooledClientProvider$TxClientPool.create(PooledClientProvider.java:48)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      4. org.apache.tephra.distributed.PooledClientProvider$TxClientPool.create(PooledClientProvider.java:41)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      5. org.apache.tephra.distributed.ElasticPool.getOrCreate(ElasticPool.java:138)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      6. org.apache.tephra.distributed.ElasticPool.obtain(ElasticPool.java:125)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      7. org.apache.tephra.distributed.PooledClientProvider.getCloseableClient(PooledClientProvider.java:101)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      8. org.apache.tephra.distributed.TransactionServiceClient.execute(TransactionServiceClient.java:214)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      9. org.apache.tephra.distributed.TransactionServiceClient.execute(TransactionServiceClient.java:185)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      10. org.apache.tephra.distributed.TransactionServiceClient.startShort(TransactionServiceClient.java:259)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      11. org.apache.tephra.TransactionContext.start(TransactionContext.java:91)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      12. org.apache.tephra.DefaultTransactionExecutor.executeOnce(DefaultTransactionExecutor.java:135)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      13. org.apache.tephra.DefaultTransactionExecutor.executeWithRetry(DefaultTransactionExecutor.java:117)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      14. org.apache.tephra.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:74)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      15. org.apache.tephra.AbstractTransactionExecutor.executeUnchecked(AbstractTransactionExecutor.java:43)[org.apache.tephra.tephra-core-0.8.0-incubating.jar:0.8.0-incubating]
      15 frames
    3. co.cask.cdap
      NamespaceHttpHandler.getAllNamespaces
      1. co.cask.cdap.store.DefaultNamespaceStore.list(DefaultNamespaceStore.java:161)[co.cask.cdap.cdap-data-fabric-3.5.0.jar:na]
      2. co.cask.cdap.internal.app.namespace.DefaultNamespaceAdmin.list(DefaultNamespaceAdmin.java:360)[co.cask.cdap.cdap-app-fabric-3.5.0.jar:na]
      3. co.cask.cdap.gateway.handlers.NamespaceHttpHandler.getAllNamespaces(NamespaceHttpHandler.java:61)[co.cask.cdap.cdap-app-fabric-3.5.0.jar:na]
      3 frames
    4. Java RT
      Method.invoke
      1. sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)[na:na]
      2. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[na:1.8.0_60]
      3. java.lang.reflect.Method.invoke(Method.java:497)[na:1.8.0_60]
      3 frames
    5. co.cask.http
      HttpDispatcher.messageReceived
      1. co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80)[co.cask.http.netty-http-0.14.0.jar:na]
      2. co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38)[co.cask.http.netty-http-0.14.0.jar:na]
      2 frames
    6. Netty
      OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run
      1. org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)[io.netty.netty-3.6.6.Final.jar:na]
      2. org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)[io.netty.netty-3.6.6.Final.jar:na]
      3. org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)[io.netty.netty-3.6.6.Final.jar:na]
      4. org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43)[io.netty.netty-3.6.6.Final.jar:na]
      5. org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67)[io.netty.netty-3.6.6.Final.jar:na]
      6. org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314)[io.netty.netty-3.6.6.Final.jar:na]
      6 frames
    7. Java RT
      Thread.run
      1. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[na:1.8.0_60]
      2. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[na:1.8.0_60]
      3. java.lang.Thread.run(Thread.java:745)[na:1.8.0_60]
      3 frames