scheduler.TaskSetManager

Lost task 0.0 in stage 0.0 (TID 0, XXX): java.io.IOException: Failed on local exception: java.io.IOException: Couldn't set up IO streams; Host Details : local host is: "XXXX"; destination host is: "XXX":9000;

Samebug tips0

There are no available Samebug tips for this exception. If you know how to solve this issue, help other users by writing a short tip.

Don't give up yet. Paste your full stack trace to get a solution.

Solutions on the web107

  • via Stack Overflow by Daniel Zolnai
    , 10 months ago
    Lost task 0.0 in stage 0.0 (TID 0, XXX): java.io.IOException: Failed on local exception: java.io.IOException: Couldn't set up IO streams; Host Details : local host is: "XXXX"; destination host is: "XXX":9000;
  • Lost task 0.0 in stage 0.0 (TID 0, 40.221.94.235): java.io.IOException: No FileSystem for scheme: spark
  • Lost task 0.0 in stage 0.0 (TID 0, ip-10-101-124-14): ExecutorLostFailure (executor 1 exited caused by one of the running tasks) Reason: Container marked as failed: container_1456648448960_0003_01_000002 on host: ip-10-101-124-14. Exit status: 1
  • Stack trace

    • scheduler.TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, XXX): java.io.IOException: Failed on local exception: java.io.IOException: Couldn't set up IO streams; Host Details : local host is: "XXXX"; destination host is: "XXX":9000; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772) at org.apache.hadoop.ipc.Client.call(Client.java:1472) at org.apache.hadoop.ipc.Client.call(Client.java:1399) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at com.sun.proxy.$Proxy13.getBlockLocations(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:254) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy14.getBlockLocations(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1220) at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1210) at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1200) at org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:271) at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:238) at org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream.java:231) at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1498) at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:302) at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:298) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:298) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:766) at org.apache.spark.input.WholeTextFileRecordReader.nextKeyValue(WholeTextFileRecordReader.scala:79) at org.apache.hadoop.mapreduce.lib.input.CombineFileRecordReader.nextKeyValue(CombineFileRecordReader.java:69) at org.apache.spark.rdd.NewHadoopRDD$$anon$1.hasNext(NewHadoopRDD.scala:163) at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:39) at org.apache.spark.util.Utils$.getIteratorSize(Utils.scala:1553) at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:1121) at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:1121) at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1848) at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1848) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) at org.apache.spark.scheduler.Task.run(Task.scala:88) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Couldn't set up IO streams at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:786) at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:368) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1521) at org.apache.hadoop.ipc.Client.call(Client.java:1438) ... 38 more Caused by: java.util.ServiceConfigurationError: org.apache.hadoop.security.SecurityInfo: Provider org.apache.hadoop.security.AnnotatedSecurityInfo not found at java.util.ServiceLoader.fail(ServiceLoader.java:231) at java.util.ServiceLoader.access$300(ServiceLoader.java:181) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:365) at java.util.ServiceLoader$1.next(ServiceLoader.java:445) at org.apache.hadoop.security.SecurityUtil.getTokenInfo(SecurityUtil.java:327) at org.apache.hadoop.security.SaslRpcClient.getServerToken(SaslRpcClient.java:263) at org.apache.hadoop.security.SaslRpcClient.createSaslClient(SaslRpcClient.java:219) at org.apache.hadoop.security.SaslRpcClient.selectSaslClient(SaslRpcClient.java:159) at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:396) at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:553) at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:368) at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:722) at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:718) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:717) ... 41 more

    Write tip

    You have a different solution? A short tip here would help you and many other users who saw this issue last week.

    Users with the same issue

    457 times, 4 weeks ago
    Unknown user
    Once, 10 months ago
    Unknown user
    Once, 11 months ago
    Unknown user
    Once, 1 year ago
    Unknown user
    Once, 1 year ago
    82 more bugmates