java.io.IOException: Failed to create directory: /3/crawldata/e12/20120509175424

JIRA | Adam Miller | 5 years ago
  1. 0

    Possible race condition when creating writable directory. In the example below, multiple threads attempt to write the the same not-yet-existant directory. This results in a NullPointerException, and the data is not written. In this example, the dns information seems to have been cached and used, but not written. The warc numbers seem to increment as well, even though the write failed, leading to non-contiguous numbers. 2012-05-09 17:55:04.657 SEVERE thread-17 org.archive.crawler.framework.ToeThread.recoverableProblem() Problem java.lang.NullPointerException occured when trying to process 'dns:www.newt.org' at step ABOUT_TO_BEGIN_PROCESSOR in Which have stack trace: java.lang.NullPointerException        at org.archive.modules.writer.WARCWriterProcessor.write(WARCWriterProcessor.java:284)        at org.archive.modules.writer.WARCWriterProcessor.innerProcessResult(WARCWriterProcessor.java:209)        at org.archive.modules.Processor.process(Processor.java:142)        at org.archive.modules.ProcessorChain.process(ProcessorChain.java:131)        at org.archive.crawler.framework.ToeThread.run(ToeThread.java:151) java.io.IOException: Failed to create directory: /3/crawldata/e12/20120509175424        at org.archive.util.FileUtils.ensureWriteableDirectory(FileUtils.java:677)        at org.archive.modules.writer.WriterPoolProcessor.calcOutputDirs(WriterPoolProcessor.java:446)        at org.archive.io.WriterPoolMember.createFile(WriterPoolMember.java:204)        at org.archive.io.WriterPoolMember.checkSize(WriterPoolMember.java:181)        at org.archive.modules.writer.WARCWriterProcessor.write(WARCWriterProcessor.java:231)        at org.archive.modules.writer.WARCWriterProcessor.innerProcessResult(WARCWriterProcessor.java:209)        at org.archive.modules.Processor.process(Processor.java:142)        at org.archive.modules.ProcessorChain.process(ProcessorChain.java:131)        at org.archive.crawler.framework.ToeThread.run(ToeThread.java:151)

    JIRA | 5 years ago | Adam Miller
    java.io.IOException: Failed to create directory: /3/crawldata/e12/20120509175424
  2. 0

    Possible race condition when creating writable directory. In the example below, multiple threads attempt to write the the same not-yet-existant directory. This results in a NullPointerException, and the data is not written. In this example, the dns information seems to have been cached and used, but not written. The warc numbers seem to increment as well, even though the write failed, leading to non-contiguous numbers. 2012-05-09 17:55:04.657 SEVERE thread-17 org.archive.crawler.framework.ToeThread.recoverableProblem() Problem java.lang.NullPointerException occured when trying to process 'dns:www.newt.org' at step ABOUT_TO_BEGIN_PROCESSOR in Which have stack trace: java.lang.NullPointerException        at org.archive.modules.writer.WARCWriterProcessor.write(WARCWriterProcessor.java:284)        at org.archive.modules.writer.WARCWriterProcessor.innerProcessResult(WARCWriterProcessor.java:209)        at org.archive.modules.Processor.process(Processor.java:142)        at org.archive.modules.ProcessorChain.process(ProcessorChain.java:131)        at org.archive.crawler.framework.ToeThread.run(ToeThread.java:151) java.io.IOException: Failed to create directory: /3/crawldata/e12/20120509175424        at org.archive.util.FileUtils.ensureWriteableDirectory(FileUtils.java:677)        at org.archive.modules.writer.WriterPoolProcessor.calcOutputDirs(WriterPoolProcessor.java:446)        at org.archive.io.WriterPoolMember.createFile(WriterPoolMember.java:204)        at org.archive.io.WriterPoolMember.checkSize(WriterPoolMember.java:181)        at org.archive.modules.writer.WARCWriterProcessor.write(WARCWriterProcessor.java:231)        at org.archive.modules.writer.WARCWriterProcessor.innerProcessResult(WARCWriterProcessor.java:209)        at org.archive.modules.Processor.process(Processor.java:142)        at org.archive.modules.ProcessorChain.process(ProcessorChain.java:131)        at org.archive.crawler.framework.ToeThread.run(ToeThread.java:151)

    JIRA | 5 years ago | Adam Miller
    java.io.IOException: Failed to create directory: /3/crawldata/e12/20120509175424
  3. 0

    GitHub comment 167#239924423

    GitHub | 4 months ago | nlevitt
    java.io.IOException: Failed to create directory: /1/ait-h3-jobs/gh-167/20160815201316/warcs
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    [build:80.8452] java.io.IOException: Failed to create directory: psf\Home

    YouTrack | 2 years ago
    java.lang.RuntimeException: java.io.IOException: Failed to create directory: psf\Home
  6. 0

    Run Test cucumber-android 1.1.8 - java.io.IOException

    GitHub | 2 years ago | chiavegatto
    java.io.IOException: Failed to create directory /target/cucumber

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.io.IOException

      Failed to create directory: /3/crawldata/e12/20120509175424

      at org.archive.util.FileUtils.ensureWriteableDirectory()
    2. webarchive-commons
      FileUtils.ensureWriteableDirectory
      1. org.archive.util.FileUtils.ensureWriteableDirectory(FileUtils.java:677)
      1 frame
    3. org.archive.modules
      WriterPoolProcessor.calcOutputDirs
      1. org.archive.modules.writer.WriterPoolProcessor.calcOutputDirs(WriterPoolProcessor.java:446)
      1 frame
    4. webarchive-commons
      WriterPoolMember.checkSize
      1. org.archive.io.WriterPoolMember.createFile(WriterPoolMember.java:204)
      2. org.archive.io.WriterPoolMember.checkSize(WriterPoolMember.java:181)
      2 frames
    5. org.archive.modules
      ProcessorChain.process
      1. org.archive.modules.writer.WARCWriterProcessor.write(WARCWriterProcessor.java:231)
      2. org.archive.modules.writer.WARCWriterProcessor.innerProcessResult(WARCWriterProcessor.java:209)
      3. org.archive.modules.Processor.process(Processor.java:142)
      4. org.archive.modules.ProcessorChain.process(ProcessorChain.java:131)
      4 frames
    6. org.archive.crawler
      ToeThread.run
      1. org.archive.crawler.framework.ToeThread.run(ToeThread.java:151)
      1 frame