java.io.IOException: 4 missing blocks, the stripe is: Offset=0, length=8388608, fetchedChunksNum=0, missingChunksNum=4

Apache's JIRA Issue Tracker | gao shan | 5 months ago
  1. 0

    The following is the steps to reproduce: 1) hadoop fs -mkdir /ec 2) set erasured code policy as "6-3" 3) "write" data by : time hadoop jar /opt/hadoop/hadoop-3.0.0-SNAPSHOT/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0-SNAPSHOT.jar TestDFSIO -D test.build.data=/ec -write -nrFiles 30 -fileSize 12288 -bufferSize 1073741824 4) Manually down 3 nodes. Kill the threads of "datanode" and "nodemanager" in 3 DataNode. 5) By using erasured code to "read" data by: time hadoop jar /opt/hadoop/hadoop-3.0.0-SNAPSHOT/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0-SNAPSHOT.jar TestDFSIO -D test.build.data=/ec -read -nrFiles 30 -fileSize 12288 -bufferSize 1073741824 then the failure occurs and the exception is thrown as: INFO mapreduce.Job: Task Id : attempt_1465445965249_0008_m_000034_2, Status : FAILED Error: java.io.IOException: 4 missing blocks, the stripe is: Offset=0, length=8388608, fetchedChunksNum=0, missingChunksNum=4 at org.apache.hadoop.hdfs.DFSStripedInputStream$StripeReader.checkMissingBlocks(DFSStripedInputStream.java:614) at org.apache.hadoop.hdfs.DFSStripedInputStream$StripeReader.readParityChunks(DFSStripedInputStream.java:647) at org.apache.hadoop.hdfs.DFSStripedInputStream$StripeReader.readStripe(DFSStripedInputStream.java:762) at org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:316) at org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:450) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:941) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.fs.TestDFSIO$ReadMapper.doIO(TestDFSIO.java:531) at org.apache.hadoop.fs.TestDFSIO$ReadMapper.doIO(TestDFSIO.java:508) at org.apache.hadoop.fs.IOMapperBase.map(IOMapperBase.java:134) at org.apache.hadoop.fs.IOMapperBase.map(IOMapperBase.java:37) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)

    Apache's JIRA Issue Tracker | 5 months ago | gao shan
    java.io.IOException: 4 missing blocks, the stripe is: Offset=0, length=8388608, fetchedChunksNum=0, missingChunksNum=4
  2. 0

    empty data file exception

    Google Groups | 4 years ago | Ben
    cascading.tuple.TupleException: unable to read from input identifier: hdfs:/url/user/root/raw_data/data201.gz
  3. Speed up your debug routine!

    Automated exception search integrated into your IDE

  4. 0

    Why is My Spark Job Failing? by Sandy Ryza of Cloudera

    slideshare.net | 1 year ago
    java.io.IOException: Filesystem closed

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.io.IOException

      4 missing blocks, the stripe is: Offset=0, length=8388608, fetchedChunksNum=0, missingChunksNum=4

      at org.apache.hadoop.hdfs.DFSStripedInputStream$StripeReader.checkMissingBlocks()
    2. Apache Hadoop HDFS
      DFSInputStream.read
      1. org.apache.hadoop.hdfs.DFSStripedInputStream$StripeReader.checkMissingBlocks(DFSStripedInputStream.java:614)
      2. org.apache.hadoop.hdfs.DFSStripedInputStream$StripeReader.readParityChunks(DFSStripedInputStream.java:647)
      3. org.apache.hadoop.hdfs.DFSStripedInputStream$StripeReader.readStripe(DFSStripedInputStream.java:762)
      4. org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:316)
      5. org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:450)
      6. org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:941)
      6 frames
    3. Java RT
      DataInputStream.read
      1. java.io.DataInputStream.read(DataInputStream.java:149)
      1 frame
    4. Hadoop
      IOMapperBase.map
      1. org.apache.hadoop.fs.TestDFSIO$ReadMapper.doIO(TestDFSIO.java:531)
      2. org.apache.hadoop.fs.TestDFSIO$ReadMapper.doIO(TestDFSIO.java:508)
      3. org.apache.hadoop.fs.IOMapperBase.map(IOMapperBase.java:134)
      4. org.apache.hadoop.fs.IOMapperBase.map(IOMapperBase.java:37)
      4 frames
    5. Hadoop
      YarnChild$2.run
      1. org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
      2. org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
      3. org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      4. org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
      4 frames
    6. Java RT
      Subject.doAs
      1. java.security.AccessController.doPrivileged(Native Method)
      2. javax.security.auth.Subject.doAs(Subject.java:422)
      2 frames
    7. Hadoop
      UserGroupInformation.doAs
      1. org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
      1 frame
    8. Hadoop
      YarnChild.main
      1. org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
      1 frame