java.lang.RuntimeException: java.util.zip.ZipException: oversubscribed dynamic bit lengths tree

JIRA | Michael Stack | 1 decade ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    Iterating over ARCRecords, if ZipException we generally fail the ARC read when we should try to just skip to the next record (if strict parse flag is not set). Here are exceptions to fix from Brad: ia311430.us.archive.org:/1/items/FS-586980/FS-586980.arc.gz Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: invalid literal/length code at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: invalid literal/length code at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more Here's a summary of the exceptions found in each. the "+++++" line separates the filename from the exception, the "----" lines separate the exception classes. I want this on my desk first thing Monday morning, so call the old lady and make sure she buys some coffee to get you thru the weekend. ; P Thanks, Brad -------------------------- ARC(FS-591213.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.io.IOException: Corrupt GZIP trailer at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.io.IOException: Corrupt GZIP trailer at java.util.zip.GZIPInputStream.readTrailer(GZIPInputStream.java:175) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:89) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ARC(FS-088773.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: invalid block type at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: invalid block type at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ---------------------------- ARC(FS-589004.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: oversubscribed dynamic bit lengths tree at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: oversubscribed dynamic bit lengths tree at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ARC(IA-001950.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: too many length or distance symbols at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: too many length or distance symbols at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ARC(GR-475057.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: invalid distance code at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: invalid distance code at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ARC(GR-480371.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: invalid literal/length code at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: invalid literal/length code at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ARC(GR-024211.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: incomplete dynamic bit lengths tree at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: incomplete dynamic bit lengths tree at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ----------------------------

    JIRA | 1 decade ago | Michael Stack
    java.lang.RuntimeException: java.util.zip.ZipException: oversubscribed dynamic bit lengths tree
  2. 0

    Iterating over ARCRecords, if ZipException we generally fail the ARC read when we should try to just skip to the next record (if strict parse flag is not set). Here are exceptions to fix from Brad: ia311430.us.archive.org:/1/items/FS-586980/FS-586980.arc.gz Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: invalid literal/length code at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: invalid literal/length code at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more Here's a summary of the exceptions found in each. the "+++++" line separates the filename from the exception, the "----" lines separate the exception classes. I want this on my desk first thing Monday morning, so call the old lady and make sure she buys some coffee to get you thru the weekend. ; P Thanks, Brad -------------------------- ARC(FS-591213.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.io.IOException: Corrupt GZIP trailer at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.io.IOException: Corrupt GZIP trailer at java.util.zip.GZIPInputStream.readTrailer(GZIPInputStream.java:175) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:89) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ARC(FS-088773.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: invalid block type at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: invalid block type at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ---------------------------- ARC(FS-589004.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: oversubscribed dynamic bit lengths tree at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: oversubscribed dynamic bit lengths tree at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ARC(IA-001950.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: too many length or distance symbols at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: too many length or distance symbols at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ARC(GR-475057.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: invalid distance code at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: invalid distance code at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ARC(GR-480371.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: invalid literal/length code at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: invalid literal/length code at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ---------------------------- ARC(GR-024211.arc.gz):retrieved ++++++++++++++++++++++++++++ Exception in thread "main" java.lang.RuntimeException: java.util.zip.ZipException: incomplete dynamic bit lengths tree at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:408) at org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124) at org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198) Caused by: java.util.zip.ZipException: incomplete dynamic bit lengths tree at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140) at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87) at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154) at org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134) at org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424) at org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325) at org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406) ... 2 more ----------------------------

    JIRA | 1 decade ago | Michael Stack
    java.lang.RuntimeException: java.util.zip.ZipException: oversubscribed dynamic bit lengths tree

    Root Cause Analysis

    1. java.util.zip.ZipException

      oversubscribed dynamic bit lengths tree

      at java.util.zip.InflaterInputStream.read()
    2. Java RT
      InflaterInputStream.skip
      1. java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140)
      2. java.util.zip.GZIPInputStream.read(GZIPInputStream.java:87)
      3. java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:184)
      3 frames
    3. webarchive-commons
      ARCReader$ARCRecordIterator.hasNext
      1. org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:154)
      2. org.archive.io.GzippedInputStream.gotoEOR(GzippedInputStream.java:134)
      3. org.archive.io.arc.ARCReaderFactory$CompressedARCReader.gotoEOR(ARCReaderFactory.java:424)
      4. org.archive.io.arc.ARCReader.cleanupCurrentRecord(ARCReader.java:325)
      5. org.archive.io.arc.ARCReader$ARCRecordIterator.hasNext(ARCReader.java:406)
      5 frames
    4. org.archive.wayback
      IndexClient.main
      1. org.archive.wayback.resourceindex.indexer.ArcIndexer.indexArc(ArcIndexer.java:124)
      2. org.archive.wayback.resourceindex.indexer.IndexClient.main(IndexClient.java:198)
      2 frames