java.nio.file.FileSystemException: /nexus_mount4/nexus/fast-linking/nexus-3.1.0-SNAPSHOT/data/elasticsearch/nexus/nodes/0/indices/dcce44c0cc6b2349fa68bacb48cf73e166f24fae/2/index/_10x4.nvd: No space left on device

Sonatype JIRA | Alberto Avritzer | 7 months ago
  1. 0

    MIgration test of 640Gbytes from NX2 (nexus-2.13.0-SNAPSHOT-bundle.zip) to NX3 (nexus-3.1.0-SNAPSHOT-bundle-faster-linking-4df539e.zip) on 1TB SSD in AWS. Hardlink and Elastic search configured for migration. Test failed after 11h:27m:20s, with exception on NX3 UI ava.nio.file.FileSystemException: /nexus_mount4/nexus/fast-linking/nexus-3.1.0-SNAPSHOT/data/elasticsearch/nexus/nodes/0/indices/dcce44c0cc6b2349fa68bacb48cf73e166f24fae/2/index/_10x4.nvd: No space left on device. However log and df shows 111Gbyte still available on device. Clip of Nexus Log: 051E3F-2D2C57D8-939F7190-F7AA98FB][/nexus_mount4/nexus/fast-linking/nexus-3.1.0-SNAPSHOT/data/elasticsearch/nexus/nodes/0] free: 111.4gb[11.3%], replicas will not be assigned to this node 2016-05-02 05:30:44,614-0400 INFO [elasticsearch[F1EFF128-E6051E3F-2D2C57D8-939F7190-F7AA98FB][management][T#1]] org.elasticsearch.cluster.routing.allocation.decider - [F1EFF128-E6051E3F-2D2C57D8-939F7190-F7AA98FB] low disk watermark [85%] exceeded on [TIGBSdFhR9y3tjVsr3W9qg][F1EFF128-E6051E3F-2D2C57D8-939F7190-F7AA98FB][/nexus_mount4/nexus/fast-linking/nexus-3.1.0-SNAPSHOT/data/elasticsearch/nexus/nodes/0] free: 111.2gb[11.3%], replicas will not be assigned to this node 2016-05-02 05:31:03,060-0400 WARN [elasticsearch[F1EFF128-E6051E3F-2D2C57D8-939F7190-F7AA98FB][refresh][T#3]] org.elasticsearch.index.engine - [F1EFF128-E6051E3F-2D2C57D8-939F7190-F7AA98FB] [dcce44c0cc6b2349fa68bacb48cf73e166f24fae][2] failed engine [refresh failed] java.nio.file.FileSystemException: /nexus_mount4/nexus/fast-linking/nexus-3.1.0-SNAPSHOT/data/elasticsearch/nexus/nodes/0/indices/dcce44c0cc6b2349fa68bacb48cf73e166f24fae/2/index/_10x4.nvd: No space left on device at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) [na:1.8.0_74] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) [na:1.8.0_74] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) [na:1.8.0_74] at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) [na:1.8.0_74] at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) [na:1.8.0_74] at java.nio.file.Files.newOutputStream(Files.java:216) [na:1.8.0_74] at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.store.FileSwitchDirectory.createOutput(FileSwitchDirectory.java:155) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.store.RateLimitedFSDirectory.createOutput(RateLimitedFSDirectory.java:40) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.store.FilterDirectory.createOutput(FilterDirectory.java:73) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.codecs.lucene53.Lucene53NormsConsumer.<init>(Lucene53NormsConsumer.java:43) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.codecs.lucene53.Lucene53NormsFormat.normsConsumer(Lucene53NormsFormat.java:77) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.index.DefaultIndexingChain.writeNorms(DefaultIndexingChain.java:193) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:92) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:422) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503) [org.apache.servicemix.bundles.elasticsearch:2.2.0.1] @

    Sonatype JIRA | 7 months ago | Alberto Avritzer
    java.nio.file.FileSystemException: /nexus_mount4/nexus/fast-linking/nexus-3.1.0-SNAPSHOT/data/elasticsearch/nexus/nodes/0/indices/dcce44c0cc6b2349fa68bacb48cf73e166f24fae/2/index/_10x4.nvd: No space left on device
  2. 0

    Cassandra File too Large Error on Start Up

    Stack Overflow | 8 months ago | RichardBruce
    org.apache.cassandra.io.FSWriteError: java.nio.file.FileSystemException: bin/../data/hints/389cb0d3-87b9-4221-8352-065e8ce50fdb-1460462523225-1.crc32: File too large
  3. 0

    How can I handle a large amount of ElasticSearch Index operations?

    Stack Overflow | 8 months ago | Colin Witkamp
    java.nio.file.FileSystemException: /Users/user/Downloads/elasticsearch-2.3.1/data/elasticsearch/nodes/0/indices/firebase/1/index/_a.fdt: Too many open files in system
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    ElasticSearch入门操作 - 为程序员服务

    outofmemory.cn | 11 months ago
    org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [flume-2015-04-14][2] failed recovery
  6. 0

    I continuously getting the same error on my cluster

    GitHub | 3 months ago | samusenkoiv
    java.nio.file.FileSystemException: /usr/share/elasticsearch/data/rallyware-elasticsearch/nodes/0/indices/request-new-db-2016.03/1/translog: Operation not permitted

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.nio.file.FileSystemException

      /nexus_mount4/nexus/fast-linking/nexus-3.1.0-SNAPSHOT/data/elasticsearch/nexus/nodes/0/indices/dcce44c0cc6b2349fa68bacb48cf73e166f24fae/2/index/_10x4.nvd: No space left on device

      at sun.nio.fs.UnixException.translateToIOException()
    2. Java RT
      Files.newOutputStream
      1. sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)[na:1.8.0_74]
      2. sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)[na:1.8.0_74]
      3. sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)[na:1.8.0_74]
      4. sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)[na:1.8.0_74]
      5. java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)[na:1.8.0_74]
      6. java.nio.file.Files.newOutputStream(Files.java:216)[na:1.8.0_74]
      6 frames
    3. Lucene
      TrackingDirectoryWrapper.createOutput
      1. org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      2. org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      3. org.apache.lucene.store.FileSwitchDirectory.createOutput(FileSwitchDirectory.java:155)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      4. org.apache.lucene.store.RateLimitedFSDirectory.createOutput(RateLimitedFSDirectory.java:40)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      5. org.apache.lucene.store.FilterDirectory.createOutput(FilterDirectory.java:73)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      6. org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      7. org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      7 frames
    4. org.apache.lucene
      Lucene53NormsFormat.normsConsumer
      1. org.apache.lucene.codecs.lucene53.Lucene53NormsConsumer.<init>(Lucene53NormsConsumer.java:43)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      2. org.apache.lucene.codecs.lucene53.Lucene53NormsFormat.normsConsumer(Lucene53NormsFormat.java:77)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      2 frames
    5. Lucene
      DocumentsWriter.doFlush
      1. org.apache.lucene.index.DefaultIndexingChain.writeNorms(DefaultIndexingChain.java:193)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      2. org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:92)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      3. org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:422)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      4. org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:503)[org.apache.servicemix.bundles.elasticsearch:2.2.0.1]
      4 frames