java.lang.RuntimeException: co.cask.tephra.TransactionFailureException: Transaction function failure for transaction.

Google Groups | Unknown author | 6 months ago
  1. 0

    Error while start cdap standalone on cloudera vm 5.5

    Google Groups | 6 months ago | Unknown author
    java.lang.RuntimeException: co.cask.tephra.TransactionFailureException: Transaction function failure for transaction.
  2. 0

    The Log service sees errors like: {code} 15/09/13 05:03:09 WARN security.UserGroupInformation: PriviledgedActionException as:cdap (auth:KERBEROS) cause:org.apache.hadoop.ipc.RemoteException(org.apac he.hadoop.security.token.SecretManager$InvalidToken): token (HDFS_DELEGATION_TOKEN token 7 for cdap) is expired 15/09/13 05:03:09 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.to ken.SecretManager$InvalidToken): token (HDFS_DELEGATION_TOKEN token 7 for cdap) is expired 15/09/13 05:03:09 WARN security.UserGroupInformation: PriviledgedActionException as:cdap (auth:KERBEROS) cause:org.apache.hadoop.ipc.RemoteException(org.apac he.hadoop.security.token.SecretManager$InvalidToken): token (HDFS_DELEGATION_TOKEN token 7 for cdap) is expired Exception in thread "HDFSTransactionStateStorage STARTING" java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token .SecretManager$InvalidToken): token (HDFS_DELEGATION_TOKEN token 7 for cdap) is expired at com.google.common.base.Throwables.propagate(Throwables.java:160) at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:47) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): token (HDFS_DELEGATION_TOKEN token 7 for cdap) is expired at org.apache.hadoop.ipc.Client.call(Client.java:1411) at org.apache.hadoop.ipc.Client.call(Client.java:1364) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206) at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:744) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1925) at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1079) at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1075) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1075) at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400) at co.cask.tephra.persist.HDFSTransactionStateStorage.startUp(HDFSTransactionStateStorage.java:108) at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) ... 1 more {code} Every attempt to restart that container sees the same issue. There were also ~10k attempts to restart before app master died.

    Cask Community Issue Tracker | 1 year ago | Albert Shau
    java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token .SecretManager$InvalidToken): token (HDFS_DELEGATION_TOKEN token 7 for cdap) is expired
  3. 0

    We saw integration test failure for HDP 2.3 which runs HBase 1.1 Here is the stack trace from the failures {code} http://104.196.32.177:10000/v3/namespaces/default/streams build 22-Mar-2016 09:13:38 2016-03-22 09:13:38,245 - INFO [main:c.c.c.t.AudiTestBase$1@122] - Received response: [200] Response Body: [] build 22-Mar-2016 09:13:38 Tests run: 31, Failures: 0, Errors: 2, Skipped: 2, Time elapsed: 3,846.799 sec <<< FAILURE! - in co.cask.cdap.test.suite.HDP23Suite build 22-Mar-2016 09:13:38 testSearchUsingSystemMetadata(co.cask.cdap.apps.metadata.PurchaseMetadataTest) Time elapsed: 43.719 sec <<< ERROR! build 22-Mar-2016 09:13:38 java.io.IOException: 500: build 22-Mar-2016 09:13:38 at co.cask.cdap.client.util.RESTClient.execute(RESTClient.java:125) build 22-Mar-2016 09:13:38 at co.cask.cdap.client.util.RESTClient.execute(RESTClient.java:68) build 22-Mar-2016 09:13:38 at co.cask.cdap.client.MetadataClient.searchMetadata(MetadataClient.java:89) build 22-Mar-2016 09:13:38 at co.cask.cdap.apps.metadata.PurchaseMetadataTest.searchMetadata(PurchaseMetadataTest.java:603) build 22-Mar-2016 09:13:38 at co.cask.cdap.apps.metadata.PurchaseMetadataTest.assertArtifactSearch(PurchaseMetadataTest.java:342) build 22-Mar-2016 09:13:38 at co.cask.cdap.apps.metadata.PurchaseMetadataTest.testSearchUsingSystemMetadata(PurchaseMetadataTest.java:323) build 22-Mar-2016 09:13:38 build 22-Mar-2016 09:13:38 testLineage(co.cask.cdap.apps.metadata.PurchaseMetadataTest) Time elapsed: 275.41 sec <<< ERROR! build 22-Mar-2016 09:13:38 java.io.IOException: 500: Illegal Type system of metadata source. build 22-Mar-2016 09:13:38 at co.cask.cdap.client.util.RESTClient.execute(RESTClient.java:125) build 22-Mar-2016 09:13:38 at co.cask.cdap.client.util.RESTClient.execute(RESTClient.java:68) build 22-Mar-2016 09:13:38 at co.cask.cdap.client.MetadataClient.searchMetadata(MetadataClient.java:89) build 22-Mar-2016 09:13:38 at co.cask.cdap.apps.metadata.PurchaseMetadataTest.searchMetadata(PurchaseMetadataTest.java:603) build 22-Mar-2016 09:13:38 at co.cask.cdap.apps.metadata.PurchaseMetadataTest.testLineage(PurchaseMetadataTest.java:293) {code} After doing some digging it looks like the new changes which went in https://issues.cask.co/browse/CDAP-4274 introduced a FuzzyRowFilter which has some incopatibility issues on hbase 1.1 When we search for any search query (existing or non existing) we get a null pointer exception. Here is the stack trace: {code} 2016-03-23 01:54:16,658 - ERROR [netty-executor-488:c.g.c.b.Throwables@160] - Unexpected error: request=GET /v3/namespaces/default/metadata/search?q uery=mango user=cdapitn: java.lang.RuntimeException: co.cask.tephra.TransactionFailureException: Transaction function failure for transaction. at com.google.common.base.Throwables.propagate(Throwables.java:160) at co.cask.tephra.AbstractTransactionExecutor.executeUnchecked(AbstractTransactionExecutor.java:43) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.execute(DefaultMetadataStore.java:496) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.fetchMetadata(DefaultMetadataStore.java:410) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadataOnType(DefaultMetadataStore.java:403) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:363) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:355) at co.cask.cdap.metadata.DefaultMetadataAdmin.searchMetadata(DefaultMetadataAdmin.java:151) at co.cask.cdap.metadata.MetadataHttpHandler.searchMetadata(MetadataHttpHandler.java:836) at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80) at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124) at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:64) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) co.cask.tephra.TransactionFailureException: Transaction function failure for transaction. at co.cask.tephra.DefaultTransactionExecutor.executeOnce(DefaultTransactionExecutor.java:138) at co.cask.tephra.DefaultTransactionExecutor.executeWithRetry(DefaultTransactionExecutor.java:115) at co.cask.tephra.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:72) at co.cask.tephra.AbstractTransactionExecutor.executeUnchecked(AbstractTransactionExecutor.java:41) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.execute(DefaultMetadataStore.java:496) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.fetchMetadata(DefaultMetadataStore.java:410) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadataOnType(DefaultMetadataStore.java:403) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:363) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:355) at co.cask.cdap.metadata.DefaultMetadataAdmin.searchMetadata(DefaultMetadataAdmin.java:151) at co.cask.cdap.metadata.MetadataHttpHandler.searchMetadata(MetadataHttpHandler.java:836) at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80) at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124) at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:64) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) java.lang.IllegalArgumentException: null at java.util.PriorityQueue.<init>(PriorityQueue.java:152) at org.apache.hadoop.hbase.filter.FuzzyRowFilter$RowTracker.<init>(FuzzyRowFilter.java:182) at org.apache.hadoop.hbase.filter.FuzzyRowFilter.<init>(FuzzyRowFilter.java:92) at co.cask.cdap.data2.dataset2.lib.table.hbase.HBaseTable.setFilterIfNeeded(HBaseTable.java:305) at co.cask.cdap.data2.dataset2.lib.table.hbase.HBaseTable.scanPersisted(HBaseTable.java:286) at co.cask.cdap.data2.dataset2.lib.table.BufferingTable.scan(BufferingTable.java:655) at co.cask.cdap.api.dataset.lib.IndexedTable.scan(IndexedTable.java:548) at co.cask.cdap.data2.metadata.dataset.MetadataDataset.getMetadata(MetadataDataset.java:454) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore$14.apply(DefaultMetadataStore.java:414) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore$14.apply(DefaultMetadataStore.java:411) at co.cask.tephra.DefaultTransactionExecutor.executeOnce(DefaultTransactionExecutor.java:136) at co.cask.tephra.DefaultTransactionExecutor.executeWithRetry(DefaultTransactionExecutor.java:115) at co.cask.tephra.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:72) at co.cask.tephra.AbstractTransactionExecutor.executeUnchecked(AbstractTransactionExecutor.java:41) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.execute(DefaultMetadataStore.java:496) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.fetchMetadata(DefaultMetadataStore.java:410) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadataOnType(DefaultMetadataStore.java:403) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:363) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:355) at co.cask.cdap.metadata.DefaultMetadataAdmin.searchMetadata(DefaultMetadataAdmin.java:151) at co.cask.cdap.metadata.MetadataHttpHandler.searchMetadata(MetadataHttpHandler.java:836) at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) 2016-03-23 02:50:16,712 - ERROR [netty-executor-13:c.g.c.b.Throwables@160] - Unexpected error: request=GET /v3/namespaces/default/metadata/search?qu ery=purchase* user=cdapitn: java.lang.RuntimeException: co.cask.tephra.TransactionFailureException: Transaction function failure for transaction. at com.google.common.base.Throwables.propagate(Throwables.java:160) at co.cask.tephra.AbstractTransactionExecutor.executeUnchecked(AbstractTransactionExecutor.java:43) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.execute(DefaultMetadataStore.java:496) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.fetchMetadata(DefaultMetadataStore.java:410) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadataOnType(DefaultMetadataStore.java:403) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:363) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:355) at co.cask.cdap.metadata.DefaultMetadataAdmin.searchMetadata(DefaultMetadataAdmin.java:151) at co.cask.cdap.metadata.MetadataHttpHandler.searchMetadata(MetadataHttpHandler.java:836) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80) at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124) at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:64) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) {code} When we search with any search query with the * (prefix based search) we get this error: {code} java.lang.IllegalArgumentException: Illegal Type ns1 of metadata source. at co.cask.cdap.data2.metadata.dataset.KeyHelper.getTargetIdIdFromKey(KeyHelper.java:110) at co.cask.cdap.data2.metadata.dataset.MdsKey.getNamespacedIdFromKey(MdsKey.java:127) at co.cask.cdap.data2.metadata.dataset.MetadataDataset.convertRow(MetadataDataset.java:493) at co.cask.cdap.data2.metadata.dataset.MetadataDataset.getMetadata(MetadataDataset.java:463) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore$14.apply(DefaultMetadataStore.java:414) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore$14.apply(DefaultMetadataStore.java:411) at co.cask.tephra.DefaultTransactionExecutor.executeOnce(DefaultTransactionExecutor.java:136) at co.cask.tephra.DefaultTransactionExecutor.executeWithRetry(DefaultTransactionExecutor.java:115) at co.cask.tephra.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:72) at co.cask.tephra.AbstractTransactionExecutor.executeUnchecked(AbstractTransactionExecutor.java:41) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.execute(DefaultMetadataStore.java:496) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.fetchMetadata(DefaultMetadataStore.java:410) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadataOnType(DefaultMetadataStore.java:403) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:363) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:355) at co.cask.cdap.metadata.DefaultMetadataAdmin.searchMetadata(DefaultMetadataAdmin.java:151) at co.cask.cdap.metadata.MetadataHttpHandler.searchMetadata(MetadataHttpHandler.java:836) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80) at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124) at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:64) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) {code}

    Cask Community Issue Tracker | 9 months ago | Rohit Sinha
    java.lang.RuntimeException: co.cask.tephra.TransactionFailureException: Transaction function failure for transaction.
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Error while start cdap standalone on cloudera vm 5.5

    Google Groups | 6 months ago | Unknown author
    java.lang.RuntimeException: co.cask.tephra.TransactionFailureException: Transaction function failure for transaction.
  6. 0

    We saw integration test failure for HDP 2.3 which runs HBase 1.1 Here is the stack trace from the failures {code} http://104.196.32.177:10000/v3/namespaces/default/streams build 22-Mar-2016 09:13:38 2016-03-22 09:13:38,245 - INFO [main:c.c.c.t.AudiTestBase$1@122] - Received response: [200] Response Body: [] build 22-Mar-2016 09:13:38 Tests run: 31, Failures: 0, Errors: 2, Skipped: 2, Time elapsed: 3,846.799 sec <<< FAILURE! - in co.cask.cdap.test.suite.HDP23Suite build 22-Mar-2016 09:13:38 testSearchUsingSystemMetadata(co.cask.cdap.apps.metadata.PurchaseMetadataTest) Time elapsed: 43.719 sec <<< ERROR! build 22-Mar-2016 09:13:38 java.io.IOException: 500: build 22-Mar-2016 09:13:38 at co.cask.cdap.client.util.RESTClient.execute(RESTClient.java:125) build 22-Mar-2016 09:13:38 at co.cask.cdap.client.util.RESTClient.execute(RESTClient.java:68) build 22-Mar-2016 09:13:38 at co.cask.cdap.client.MetadataClient.searchMetadata(MetadataClient.java:89) build 22-Mar-2016 09:13:38 at co.cask.cdap.apps.metadata.PurchaseMetadataTest.searchMetadata(PurchaseMetadataTest.java:603) build 22-Mar-2016 09:13:38 at co.cask.cdap.apps.metadata.PurchaseMetadataTest.assertArtifactSearch(PurchaseMetadataTest.java:342) build 22-Mar-2016 09:13:38 at co.cask.cdap.apps.metadata.PurchaseMetadataTest.testSearchUsingSystemMetadata(PurchaseMetadataTest.java:323) build 22-Mar-2016 09:13:38 build 22-Mar-2016 09:13:38 testLineage(co.cask.cdap.apps.metadata.PurchaseMetadataTest) Time elapsed: 275.41 sec <<< ERROR! build 22-Mar-2016 09:13:38 java.io.IOException: 500: Illegal Type system of metadata source. build 22-Mar-2016 09:13:38 at co.cask.cdap.client.util.RESTClient.execute(RESTClient.java:125) build 22-Mar-2016 09:13:38 at co.cask.cdap.client.util.RESTClient.execute(RESTClient.java:68) build 22-Mar-2016 09:13:38 at co.cask.cdap.client.MetadataClient.searchMetadata(MetadataClient.java:89) build 22-Mar-2016 09:13:38 at co.cask.cdap.apps.metadata.PurchaseMetadataTest.searchMetadata(PurchaseMetadataTest.java:603) build 22-Mar-2016 09:13:38 at co.cask.cdap.apps.metadata.PurchaseMetadataTest.testLineage(PurchaseMetadataTest.java:293) {code} After doing some digging it looks like the new changes which went in https://issues.cask.co/browse/CDAP-4274 introduced a FuzzyRowFilter which has some incopatibility issues on hbase 1.1 When we search for any search query (existing or non existing) we get a null pointer exception. Here is the stack trace: {code} 2016-03-23 01:54:16,658 - ERROR [netty-executor-488:c.g.c.b.Throwables@160] - Unexpected error: request=GET /v3/namespaces/default/metadata/search?q uery=mango user=cdapitn: java.lang.RuntimeException: co.cask.tephra.TransactionFailureException: Transaction function failure for transaction. at com.google.common.base.Throwables.propagate(Throwables.java:160) at co.cask.tephra.AbstractTransactionExecutor.executeUnchecked(AbstractTransactionExecutor.java:43) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.execute(DefaultMetadataStore.java:496) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.fetchMetadata(DefaultMetadataStore.java:410) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadataOnType(DefaultMetadataStore.java:403) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:363) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:355) at co.cask.cdap.metadata.DefaultMetadataAdmin.searchMetadata(DefaultMetadataAdmin.java:151) at co.cask.cdap.metadata.MetadataHttpHandler.searchMetadata(MetadataHttpHandler.java:836) at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80) at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124) at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:64) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) co.cask.tephra.TransactionFailureException: Transaction function failure for transaction. at co.cask.tephra.DefaultTransactionExecutor.executeOnce(DefaultTransactionExecutor.java:138) at co.cask.tephra.DefaultTransactionExecutor.executeWithRetry(DefaultTransactionExecutor.java:115) at co.cask.tephra.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:72) at co.cask.tephra.AbstractTransactionExecutor.executeUnchecked(AbstractTransactionExecutor.java:41) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.execute(DefaultMetadataStore.java:496) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.fetchMetadata(DefaultMetadataStore.java:410) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadataOnType(DefaultMetadataStore.java:403) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:363) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:355) at co.cask.cdap.metadata.DefaultMetadataAdmin.searchMetadata(DefaultMetadataAdmin.java:151) at co.cask.cdap.metadata.MetadataHttpHandler.searchMetadata(MetadataHttpHandler.java:836) at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80) at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124) at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:64) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) java.lang.IllegalArgumentException: null at java.util.PriorityQueue.<init>(PriorityQueue.java:152) at org.apache.hadoop.hbase.filter.FuzzyRowFilter$RowTracker.<init>(FuzzyRowFilter.java:182) at org.apache.hadoop.hbase.filter.FuzzyRowFilter.<init>(FuzzyRowFilter.java:92) at co.cask.cdap.data2.dataset2.lib.table.hbase.HBaseTable.setFilterIfNeeded(HBaseTable.java:305) at co.cask.cdap.data2.dataset2.lib.table.hbase.HBaseTable.scanPersisted(HBaseTable.java:286) at co.cask.cdap.data2.dataset2.lib.table.BufferingTable.scan(BufferingTable.java:655) at co.cask.cdap.api.dataset.lib.IndexedTable.scan(IndexedTable.java:548) at co.cask.cdap.data2.metadata.dataset.MetadataDataset.getMetadata(MetadataDataset.java:454) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore$14.apply(DefaultMetadataStore.java:414) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore$14.apply(DefaultMetadataStore.java:411) at co.cask.tephra.DefaultTransactionExecutor.executeOnce(DefaultTransactionExecutor.java:136) at co.cask.tephra.DefaultTransactionExecutor.executeWithRetry(DefaultTransactionExecutor.java:115) at co.cask.tephra.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:72) at co.cask.tephra.AbstractTransactionExecutor.executeUnchecked(AbstractTransactionExecutor.java:41) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.execute(DefaultMetadataStore.java:496) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.fetchMetadata(DefaultMetadataStore.java:410) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadataOnType(DefaultMetadataStore.java:403) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:363) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:355) at co.cask.cdap.metadata.DefaultMetadataAdmin.searchMetadata(DefaultMetadataAdmin.java:151) at co.cask.cdap.metadata.MetadataHttpHandler.searchMetadata(MetadataHttpHandler.java:836) at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) 2016-03-23 02:50:16,712 - ERROR [netty-executor-13:c.g.c.b.Throwables@160] - Unexpected error: request=GET /v3/namespaces/default/metadata/search?qu ery=purchase* user=cdapitn: java.lang.RuntimeException: co.cask.tephra.TransactionFailureException: Transaction function failure for transaction. at com.google.common.base.Throwables.propagate(Throwables.java:160) at co.cask.tephra.AbstractTransactionExecutor.executeUnchecked(AbstractTransactionExecutor.java:43) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.execute(DefaultMetadataStore.java:496) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.fetchMetadata(DefaultMetadataStore.java:410) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadataOnType(DefaultMetadataStore.java:403) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:363) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:355) at co.cask.cdap.metadata.DefaultMetadataAdmin.searchMetadata(DefaultMetadataAdmin.java:151) at co.cask.cdap.metadata.MetadataHttpHandler.searchMetadata(MetadataHttpHandler.java:836) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80) at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124) at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:64) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) {code} When we search with any search query with the * (prefix based search) we get this error: {code} java.lang.IllegalArgumentException: Illegal Type ns1 of metadata source. at co.cask.cdap.data2.metadata.dataset.KeyHelper.getTargetIdIdFromKey(KeyHelper.java:110) at co.cask.cdap.data2.metadata.dataset.MdsKey.getNamespacedIdFromKey(MdsKey.java:127) at co.cask.cdap.data2.metadata.dataset.MetadataDataset.convertRow(MetadataDataset.java:493) at co.cask.cdap.data2.metadata.dataset.MetadataDataset.getMetadata(MetadataDataset.java:463) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore$14.apply(DefaultMetadataStore.java:414) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore$14.apply(DefaultMetadataStore.java:411) at co.cask.tephra.DefaultTransactionExecutor.executeOnce(DefaultTransactionExecutor.java:136) at co.cask.tephra.DefaultTransactionExecutor.executeWithRetry(DefaultTransactionExecutor.java:115) at co.cask.tephra.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:72) at co.cask.tephra.AbstractTransactionExecutor.executeUnchecked(AbstractTransactionExecutor.java:41) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.execute(DefaultMetadataStore.java:496) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.fetchMetadata(DefaultMetadataStore.java:410) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadataOnType(DefaultMetadataStore.java:403) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:363) at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.searchMetadata(DefaultMetadataStore.java:355) at co.cask.cdap.metadata.DefaultMetadataAdmin.searchMetadata(DefaultMetadataAdmin.java:151) at co.cask.cdap.metadata.MetadataHttpHandler.searchMetadata(MetadataHttpHandler.java:836) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80) at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124) at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:64) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) {code}

    Cask Community Issue Tracker | 9 months ago | Rohit Sinha
    java.lang.RuntimeException: co.cask.tephra.TransactionFailureException: Transaction function failure for transaction.

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.RuntimeException

      co.cask.tephra.TransactionFailureException: Transaction function failure for transaction.

      at com.google.common.base.Throwables.propagate()
    2. Guava
      AbstractIdleService$1$1.run
      1. com.google.common.base.Throwables.propagate(Throwables.java:160)
      2. com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:47)
      2 frames
    3. Java RT
      Thread.run
      1. java.lang.Thread.run(Thread.java:745)
      1 frame