com.atlassian.jira.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException

Atlassian JIRA | Chris Fuller [On Leave until 2016/09/14] | 2 years ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    [JRA-41409] Lucene index can be corrupted by a graceful shutdown - Atlassian JIRA

    atlassian.com | 2 years ago
    com.atlassian.jira.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException
  2. 0

    {{QueuingIndex}} uses thread interruption to implement cancellation. The problem is that there is a race condition if the indexer is closed while an index write operation is in progress, as the interrupt causes the underlying index file channels to abort their write operations, leading to corrupt indexes. See chapter 7 of _Java Concurrency in Practice_ for a detailed description of task cancellation strategies and more information about why using {{interrupt()}} when running other people's code is not a good idea. This is the underlying culprit in JRA-38544. As near as I can tell, it has existed since the indexing code was rewritten in JIRA 4.0. It is normally a very rarely triggered race condition, but especially busy large systems are more likely to encounter it. Example stack trace: {noformat} 2014-12-19 18:07:45,525 some-thread-name WARN some-user [atlassian.jira.index.AccumulatingResultBuilder] Indexing failed for Issue - '720151' 2014-12-19 18:07:45,530 some-thread-name WARN some-user [atlassian.jira.index.AccumulatingResultBuilder] java.nio.channels.ClosedByInterruptException com.atlassian.jira.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException at com.atlassian.jira.index.WriterWrapper.commit(WriterWrapper.java:137) at com.atlassian.jira.index.DefaultIndexEngine$WriterReference.commit(DefaultIndexEngine.java:230) at com.atlassian.jira.index.DefaultIndexEngine$FlushPolicy$2.commit(DefaultIndexEngine.java:63) at com.atlassian.jira.index.DefaultIndexEngine$FlushPolicy.perform(DefaultIndexEngine.java:88) at com.atlassian.jira.index.DefaultIndexEngine.write(DefaultIndexEngine.java:158) at com.atlassian.jira.index.DefaultIndex.perform(DefaultIndex.java:36) at com.atlassian.jira.index.QueueingIndex$Task.index(QueueingIndex.java:145) at com.atlassian.jira.index.QueueingIndex$Task.run(QueueingIndex.java:130) at java.lang.Thread.run(Thread.java:724) Caused by: java.nio.channels.ClosedByInterruptException at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:880) at org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:264) at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216) at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:85) at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:114) at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:702) at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:677) at org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:249) at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3571) at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3508) at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3371) at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3444) at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3426) at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3410) at com.atlassian.jira.index.WriterWrapper.commit(WriterWrapper.java:133) ... 8 more {noformat}

    Atlassian JIRA | 2 years ago | Chris Fuller
    com.atlassian.jira.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException
  3. 0

    {{QueuingIndex}} uses thread interruption to implement cancellation. The problem is that there is a race condition if the indexer is closed while an index write operation is in progress, as the interrupt causes the underlying index file channels to abort their write operations, leading to corrupt indexes. See chapter 7 of _Java Concurrency in Practice_ for a detailed description of task cancellation strategies and more information about why using {{interrupt()}} when running other people's code is not a good idea. This is the underlying culprit in JRA-38544. As near as I can tell, it has existed since the indexing code was rewritten in JIRA 4.0. It is normally a very rarely triggered race condition, but especially busy large systems are more likely to encounter it. Example stack trace: {noformat} 2014-12-19 18:07:45,525 some-thread-name WARN some-user [atlassian.jira.index.AccumulatingResultBuilder] Indexing failed for Issue - '720151' 2014-12-19 18:07:45,530 some-thread-name WARN some-user [atlassian.jira.index.AccumulatingResultBuilder] java.nio.channels.ClosedByInterruptException com.atlassian.jira.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException at com.atlassian.jira.index.WriterWrapper.commit(WriterWrapper.java:137) at com.atlassian.jira.index.DefaultIndexEngine$WriterReference.commit(DefaultIndexEngine.java:230) at com.atlassian.jira.index.DefaultIndexEngine$FlushPolicy$2.commit(DefaultIndexEngine.java:63) at com.atlassian.jira.index.DefaultIndexEngine$FlushPolicy.perform(DefaultIndexEngine.java:88) at com.atlassian.jira.index.DefaultIndexEngine.write(DefaultIndexEngine.java:158) at com.atlassian.jira.index.DefaultIndex.perform(DefaultIndex.java:36) at com.atlassian.jira.index.QueueingIndex$Task.index(QueueingIndex.java:145) at com.atlassian.jira.index.QueueingIndex$Task.run(QueueingIndex.java:130) at java.lang.Thread.run(Thread.java:724) Caused by: java.nio.channels.ClosedByInterruptException at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:880) at org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:264) at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216) at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:85) at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:114) at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:702) at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:677) at org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:249) at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3571) at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3508) at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3371) at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3444) at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3426) at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3410) at com.atlassian.jira.index.WriterWrapper.commit(WriterWrapper.java:133) ... 8 more {noformat}

    Atlassian JIRA | 2 years ago | Chris Fuller [On Leave until 2016/09/14]
    com.atlassian.jira.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Recommended way to handle ClosedByInterruptException?

    Google Groups | 4 years ago | rap...@gmail.com
    java.nio.channels.ClosedByInterruptException

    Root Cause Analysis

    1. java.nio.channels.ClosedByInterruptException

      No message provided

      at java.nio.channels.spi.AbstractInterruptibleChannel.end()
    2. Java RT
      FileChannelImpl.map
      1. java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
      2. sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:880)
      2 frames
    3. Lucene
      IndexWriter.commit
      1. org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:264)
      2. org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
      3. org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:85)
      4. org.apache.lucene.index.SegmentReader.get(SegmentReader.java:114)
      5. org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:702)
      6. org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:677)
      7. org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:249)
      8. org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3571)
      9. org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3508)
      10. org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3371)
      11. org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3444)
      12. org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3426)
      13. org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3410)
      13 frames
    4. com.atlassian.jira
      QueueingIndex$Task.run
      1. com.atlassian.jira.index.WriterWrapper.commit(WriterWrapper.java:133)
      2. com.atlassian.jira.index.DefaultIndexEngine$WriterReference.commit(DefaultIndexEngine.java:230)
      3. com.atlassian.jira.index.DefaultIndexEngine$FlushPolicy$2.commit(DefaultIndexEngine.java:63)
      4. com.atlassian.jira.index.DefaultIndexEngine$FlushPolicy.perform(DefaultIndexEngine.java:88)
      5. com.atlassian.jira.index.DefaultIndexEngine.write(DefaultIndexEngine.java:158)
      6. com.atlassian.jira.index.DefaultIndex.perform(DefaultIndex.java:36)
      7. com.atlassian.jira.index.QueueingIndex$Task.index(QueueingIndex.java:145)
      8. com.atlassian.jira.index.QueueingIndex$Task.run(QueueingIndex.java:130)
      8 frames
    5. Java RT
      Thread.run
      1. java.lang.Thread.run(Thread.java:724)
      1 frame