java.io.IOException: Copied: 0 Skipped: 0 Failed: 1

Apache's JIRA Issue Tracker | Viraj Bhat | 9 years ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    Copying folders containing + & ! characters between hdfs (using hftp) does not work in distcp For example: Copying folder "string1+string2" at "namenode.address.com", hftp port myport to "/myotherhome/folder" on "myothermachine" does not work myothermachine prompt>>> hadoop --config ~/mycluster/ distcp "hftp://namenode.address.com:myport/myhome/dir/string1+string2" /myotherhome/folder/ -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Error results for hadoop job1: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 08/07/16 00:27:39 INFO tools.DistCp: srcPaths=[hftp://namenode.address.com:myport/myhome/dir/string1+string2] 08/07/16 00:27:39 INFO tools.DistCp: destPath=/myotherhome/folder/ 08/07/16 00:27:41 INFO tools.DistCp: srcCount=2 08/07/16 00:27:42 INFO mapred.JobClient: Running job: job1 08/07/16 00:27:43 INFO mapred.JobClient: map 0% reduce 0% 08/07/16 00:27:58 INFO mapred.JobClient: Task Id : attempt_1_m_000000_0, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:538) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:226) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208) 08/07/16 00:28:14 INFO mapred.JobClient: Task Id : attempt_1_m_000000_1, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:538) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:226) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208) 08/07/16 00:28:28 INFO mapred.JobClient: Task Id : attempt_1_m_000000_2, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:538) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:226) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208) With failures, global counters are inaccurate; consider running with -i Copy failed: java.io.IOException: Job failed! at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1053) at org.apache.hadoop.tools.DistCp.copy(DistCp.java:615) at org.apache.hadoop.tools.DistCp.run(DistCp.java:764) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at org.apache.hadoop.tools.DistCp.main(DistCp.java:784) -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Error log for the map task which failed -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- INFO org.apache.hadoop.tools.DistCp: FAIL string1+string2/myjobtrackermachine.com-joblog.tar.gz : java.io.IOException: Server returned HTTP response code: 500 for URL: http://mymachine.com:myport/streamFile?filename=/myhome/dir/string1+string2/myjobtrackermachine.com-joblog.tar.gz&ugi=myid,mygroup at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1241) at org.apache.hadoop.dfs.HftpFileSystem.open(HftpFileSystem.java:117) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:371) at org.apache.hadoop.tools.DistCp$CopyFilesMapper.copy(DistCp.java:377) at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:504) at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:279) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:226) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Apache's JIRA Issue Tracker | 9 years ago | Viraj Bhat
    java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
  2. 0

    Copying folders containing + & ! characters between hdfs (using hftp) does not work in distcp For example: Copying folder "string1+string2" at "namenode.address.com", hftp port myport to "/myotherhome/folder" on "myothermachine" does not work myothermachine prompt>>> hadoop --config ~/mycluster/ distcp "hftp://namenode.address.com:myport/myhome/dir/string1+string2" /myotherhome/folder/ -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Error results for hadoop job1: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 08/07/16 00:27:39 INFO tools.DistCp: srcPaths=[hftp://namenode.address.com:myport/myhome/dir/string1+string2] 08/07/16 00:27:39 INFO tools.DistCp: destPath=/myotherhome/folder/ 08/07/16 00:27:41 INFO tools.DistCp: srcCount=2 08/07/16 00:27:42 INFO mapred.JobClient: Running job: job1 08/07/16 00:27:43 INFO mapred.JobClient: map 0% reduce 0% 08/07/16 00:27:58 INFO mapred.JobClient: Task Id : attempt_1_m_000000_0, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:538) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:226) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208) 08/07/16 00:28:14 INFO mapred.JobClient: Task Id : attempt_1_m_000000_1, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:538) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:226) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208) 08/07/16 00:28:28 INFO mapred.JobClient: Task Id : attempt_1_m_000000_2, Status : FAILED java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:538) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:226) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208) With failures, global counters are inaccurate; consider running with -i Copy failed: java.io.IOException: Job failed! at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1053) at org.apache.hadoop.tools.DistCp.copy(DistCp.java:615) at org.apache.hadoop.tools.DistCp.run(DistCp.java:764) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at org.apache.hadoop.tools.DistCp.main(DistCp.java:784) -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Error log for the map task which failed -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- INFO org.apache.hadoop.tools.DistCp: FAIL string1+string2/myjobtrackermachine.com-joblog.tar.gz : java.io.IOException: Server returned HTTP response code: 500 for URL: http://mymachine.com:myport/streamFile?filename=/myhome/dir/string1+string2/myjobtrackermachine.com-joblog.tar.gz&ugi=myid,mygroup at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1241) at org.apache.hadoop.dfs.HftpFileSystem.open(HftpFileSystem.java:117) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:371) at org.apache.hadoop.tools.DistCp$CopyFilesMapper.copy(DistCp.java:377) at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:504) at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:279) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:226) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Apache's JIRA Issue Tracker | 9 years ago | Viraj Bhat
    java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
  3. 0

    Distcp failure - Server returned HTTP response code: 500

    Google Groups | 6 years ago | sonia gehlot
    java.io.IOException: Copied: 0 Skipped: 5 Failed: 1
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Re: distcp just fails (was:distcp fails with ConnectException)

    hadoop-mapreduce-user | 6 years ago | hadoopman
    java.io.IOException: Copied: 0 Skipped: 0 Failed: 5
  6. 0

    Re: HDFS and distcp issue??

    hadoop-common-user | 6 years ago | Dmitriy Ryaboy
    java.io.IOException: Copied: 0 Skipped: 0 Failed: 5 at > org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:572)

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.io.IOException

      Copied: 0 Skipped: 0 Failed: 1

      at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close()
    2. Hadoop
      DistCp$CopyFilesMapper.close
      1. org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:538)
      1 frame
    3. Hadoop
      TaskTracker$Child.main
      1. org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
      2. org.apache.hadoop.mapred.MapTask.run(MapTask.java:226)
      3. org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208)
      3 frames