java.lang.RuntimeException: java.io.IOException: Destination '/apps/opt/atlassian/application-data/jira/caches/indexes/issues' already exists

Atlassian JIRA | Andriy Yakovlev [Atlassian] | 1 year ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    h3. Summary JIRA Datacenter nod - failed to restore the snapshot from another node h3. Environment * JIRA Datacenter 6.4.12 * 2 Nodes h3. Steps to Reproduce # Start index replication at Node2 from Node1 # Watch the status in logs h3. Expected Results Index is replicated and Node2 is operational Message in logs: {noformat} 2016-01-12 01:35:17,456 NodeReindexServiceThread:thread-1 INFO [jira.index.ha.DefaultIndexRecoveryManager] Recovering search indexes - 100% complete... 2016-01-12 01:35:17,483 NodeReindexServiceThread:thread-1 INFO [jira.index.ha.DefaultIndexCopyService] Index restore complete {noformat} h3. Actual Results Index is not replicated and Node2 is not operational The below exception is thrown in the log file: {noformat} 2016-01-12 01:34:24,945 NodeReindexServiceThread:thread-1 ERROR [jira.index.ha.DefaultNodeReindexService] Error re-indexing node changes java.lang.RuntimeException: java.io.IOException: Destination '/apps/opt/atlassian/application-data/jira/caches/indexes/issues' already exists at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager$ReplaceIndexRunner.run(DefaultIndexRecoveryManager.java:374) at com.atlassian.jira.issue.index.DefaultIndexManager.withReindexLock(DefaultIndexManager.java:382) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.atlassian.util.profiling.object.ObjectProfiler.profiledInvoke(ObjectProfiler.java:83) at com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:28) at com.sun.proxy.$Proxy17.withReindexLock(Unknown Source) at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager.recoverIndexFromBackup(DefaultIndexRecoveryManager.java:124) at com.atlassian.jira.index.ha.DefaultIndexCopyService$MessageConsumer.restoreIndex(DefaultIndexCopyService.java:175) at com.atlassian.jira.index.ha.DefaultIndexCopyService.restoreIndex(DefaultIndexCopyService.java:79) at com.atlassian.jira.index.ha.DefaultNodeReindexService.updateAffectedIndexes(DefaultNodeReindexService.java:344) at com.atlassian.jira.index.ha.DefaultNodeReindexService.reIndex(DefaultNodeReindexService.java:291) at com.atlassian.jira.index.ha.DefaultNodeReindexService.access$000(DefaultNodeReindexService.java:58) at com.atlassian.jira.index.ha.DefaultNodeReindexService$1.run(DefaultNodeReindexService.java:82) (...) Caused by: java.io.IOException: Destination '/apps/opt/atlassian/application-data/jira/caches/indexes/issues' already exists at org.apache.commons.io.FileUtils.moveDirectory(FileUtils.java:1739) at org.apache.commons.io.FileUtils.moveDirectoryToDirectory(FileUtils.java:1781) at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager$ReplaceIndexRunner.replaceIndexes(DefaultIndexRecoveryManager.java:452) at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager$ReplaceIndexRunner.run(DefaultIndexRecoveryManager.java:370) ... 22 more {noformat} h3. Notes Index restore also initiated automatically when one of nodes finishes full re-index and asks other nodes to apply new index. h3.Workaround Restart Node so it will pickup latest valid Index snapshot from other nodes

    Atlassian JIRA | 1 year ago | Andriy Yakovlev [Atlassian]
    java.lang.RuntimeException: java.io.IOException: Destination '/apps/opt/atlassian/application-data/jira/caches/indexes/issues' already exists
  2. 0

    h3. Summary JIRA Datacenter nod - failed to restore the snapshot from another node h3. Environment * JIRA Datacenter 6.4.12 * 2 Nodes h3. Steps to Reproduce # Start index replication at Node2 from Node1 # Watch the status in logs h3. Expected Results Index is replicated and Node2 is operational Message in logs: {noformat} 2016-01-12 01:35:17,456 NodeReindexServiceThread:thread-1 INFO [jira.index.ha.DefaultIndexRecoveryManager] Recovering search indexes - 100% complete... 2016-01-12 01:35:17,483 NodeReindexServiceThread:thread-1 INFO [jira.index.ha.DefaultIndexCopyService] Index restore complete {noformat} h3. Actual Results Index is not replicated and Node2 is not operational The below exception is thrown in the log file: {noformat} 2016-01-12 01:34:24,945 NodeReindexServiceThread:thread-1 ERROR [jira.index.ha.DefaultNodeReindexService] Error re-indexing node changes java.lang.RuntimeException: java.io.IOException: Destination '/apps/opt/atlassian/application-data/jira/caches/indexes/issues' already exists at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager$ReplaceIndexRunner.run(DefaultIndexRecoveryManager.java:374) at com.atlassian.jira.issue.index.DefaultIndexManager.withReindexLock(DefaultIndexManager.java:382) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.atlassian.util.profiling.object.ObjectProfiler.profiledInvoke(ObjectProfiler.java:83) at com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:28) at com.sun.proxy.$Proxy17.withReindexLock(Unknown Source) at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager.recoverIndexFromBackup(DefaultIndexRecoveryManager.java:124) at com.atlassian.jira.index.ha.DefaultIndexCopyService$MessageConsumer.restoreIndex(DefaultIndexCopyService.java:175) at com.atlassian.jira.index.ha.DefaultIndexCopyService.restoreIndex(DefaultIndexCopyService.java:79) at com.atlassian.jira.index.ha.DefaultNodeReindexService.updateAffectedIndexes(DefaultNodeReindexService.java:344) at com.atlassian.jira.index.ha.DefaultNodeReindexService.reIndex(DefaultNodeReindexService.java:291) at com.atlassian.jira.index.ha.DefaultNodeReindexService.access$000(DefaultNodeReindexService.java:58) at com.atlassian.jira.index.ha.DefaultNodeReindexService$1.run(DefaultNodeReindexService.java:82) (...) Caused by: java.io.IOException: Destination '/apps/opt/atlassian/application-data/jira/caches/indexes/issues' already exists at org.apache.commons.io.FileUtils.moveDirectory(FileUtils.java:1739) at org.apache.commons.io.FileUtils.moveDirectoryToDirectory(FileUtils.java:1781) at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager$ReplaceIndexRunner.replaceIndexes(DefaultIndexRecoveryManager.java:452) at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager$ReplaceIndexRunner.run(DefaultIndexRecoveryManager.java:370) ... 22 more {noformat} h3. Notes Index restore also initiated automatically when one of nodes finishes full re-index and asks other nodes to apply new index. h3.Workaround Restart Node so it will pickup latest valid Index snapshot from other nodes

    Atlassian JIRA | 1 year ago | Andriy Yakovlev [Atlassian]
    java.lang.RuntimeException: java.io.IOException: Destination '/apps/opt/atlassian/application-data/jira/caches/indexes/issues' already exists

    Root Cause Analysis

    1. java.lang.RuntimeException

      java.io.IOException: Destination '/apps/opt/atlassian/application-data/jira/caches/indexes/issues' already exists

      at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager$ReplaceIndexRunner.run()
    2. com.atlassian.jira
      DefaultIndexManager.withReindexLock
      1. com.atlassian.jira.index.ha.DefaultIndexRecoveryManager$ReplaceIndexRunner.run(DefaultIndexRecoveryManager.java:374)
      2. com.atlassian.jira.issue.index.DefaultIndexManager.withReindexLock(DefaultIndexManager.java:382)
      2 frames
    3. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:497)
      4 frames
    4. com.atlassian.util
      ObjectProfiler.profiledInvoke
      1. com.atlassian.util.profiling.object.ObjectProfiler.profiledInvoke(ObjectProfiler.java:83)
      1 frame
    5. com.atlassian.jira
      SwitchingInvocationHandler.invoke
      1. com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:28)
      1 frame
    6. com.sun.proxy
      $Proxy17.withReindexLock
      1. com.sun.proxy.$Proxy17.withReindexLock(Unknown Source)
      1 frame
    7. com.atlassian.jira
      DefaultNodeReindexService$1.run
      1. com.atlassian.jira.index.ha.DefaultIndexRecoveryManager.recoverIndexFromBackup(DefaultIndexRecoveryManager.java:124)
      2. com.atlassian.jira.index.ha.DefaultIndexCopyService$MessageConsumer.restoreIndex(DefaultIndexCopyService.java:175)
      3. com.atlassian.jira.index.ha.DefaultIndexCopyService.restoreIndex(DefaultIndexCopyService.java:79)
      4. com.atlassian.jira.index.ha.DefaultNodeReindexService.updateAffectedIndexes(DefaultNodeReindexService.java:344)
      5. com.atlassian.jira.index.ha.DefaultNodeReindexService.reIndex(DefaultNodeReindexService.java:291)
      6. com.atlassian.jira.index.ha.DefaultNodeReindexService.access$000(DefaultNodeReindexService.java:58)
      7. com.atlassian.jira.index.ha.DefaultNodeReindexService$1.run(DefaultNodeReindexService.java:82)
      7 frames