java.io.IOException: Offset(0) plus length(80) is larger than file length(11)

JIRA | David Capwell | 3 years ago
tip
Do you know that we can give you better hits? Get more relevant results from Samebug’s stack trace search.
  1. 0

    This was run on my mac. 14/08/13 09:17:24 WARN : Failed to read local file /Volumes/ramdisk/tachyonworker/3221225472 because: java.io.IOException: Offset(0) plus length(80) is larger than file length(11) at tachyon.client.TachyonFS.readLocalByteBuffer(TachyonFS.java:1152) at tachyon.client.TachyonFile.readLocalByteBuffer(TachyonFile.java:323) at tachyon.client.BlockInStream.get(BlockInStream.java:60) at tachyon.client.TachyonFile.getInStream(TachyonFile.java:132) at tachyon.examples.BasicNonByteBufferOperations.read(BasicNonByteBufferOperations.java:53) at tachyon.examples.BasicNonByteBufferOperations.main(BasicNonByteBufferOperations.java:22) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134) 14/08/13 09:17:24 INFO : Try to find remote worker and read block 3221225472 from 0, with len 80 $ ls -l /Volumes/ramdisk/tachyonworker/3221225472 -rwxrwxrwx 1 dcapwell staff 11 Aug 13 09:17 /Volumes/ramdisk/tachyonworker/3221225472 Master UI bar 80.00 B 1024.00 MB 100% NO 08-13-2014 08:32:50:999 08-13-2014 09:17:24:828 When I click on 'bar' Error: File /foo/bar is not available Can not find the block /foo/bar 0 When I test locally, it seems this happens when you try to override the file. The write path never fails to accept anything thrown at it, but the read path will not see that data ever. Only way to resolve this, from the client side, is to delete the file. This will cause the metadata to be in-sync. Not looked into this yet, but it does seem that the worker's content gets written, but that the metadata never gets updated. Code that generated this issue. Write path: {code} TachyonFile file = getOrCreate(client, filePath); OutStream os = file.getOutStream(writeType); int length = 10; try { os.write(length); for (int i = 0; i < length; i ++) { os.write(i); } } finally { os.close(); } {code} Utils {code} private static TachyonFile getOrCreate(TachyonFS client, String filePath) throws IOException { TachyonFile file = client.getFile(filePath); if (file == null) { int fileId = client.createFile(filePath); file = client.getFile(fileId); } return file; } {code} Read path {code} TachyonFile file = client.getFile(filePath); InStream input = file.getInStream(readType); try { int length = input.read(); for (int i = 0; i < length; i ++) { System.out.println(input.read()); } } finally { input.close(); } {code}

    JIRA | 3 years ago | David Capwell
    java.io.IOException: Offset(0) plus length(80) is larger than file length(11)
  2. 0

    This was run on my mac. 14/08/13 09:17:24 WARN : Failed to read local file /Volumes/ramdisk/tachyonworker/3221225472 because: java.io.IOException: Offset(0) plus length(80) is larger than file length(11) at tachyon.client.TachyonFS.readLocalByteBuffer(TachyonFS.java:1152) at tachyon.client.TachyonFile.readLocalByteBuffer(TachyonFile.java:323) at tachyon.client.BlockInStream.get(BlockInStream.java:60) at tachyon.client.TachyonFile.getInStream(TachyonFile.java:132) at tachyon.examples.BasicNonByteBufferOperations.read(BasicNonByteBufferOperations.java:53) at tachyon.examples.BasicNonByteBufferOperations.main(BasicNonByteBufferOperations.java:22) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134) 14/08/13 09:17:24 INFO : Try to find remote worker and read block 3221225472 from 0, with len 80 $ ls -l /Volumes/ramdisk/tachyonworker/3221225472 -rwxrwxrwx 1 dcapwell staff 11 Aug 13 09:17 /Volumes/ramdisk/tachyonworker/3221225472 Master UI bar 80.00 B 1024.00 MB 100% NO 08-13-2014 08:32:50:999 08-13-2014 09:17:24:828 When I click on 'bar' Error: File /foo/bar is not available Can not find the block /foo/bar 0 When I test locally, it seems this happens when you try to override the file. The write path never fails to accept anything thrown at it, but the read path will not see that data ever. Only way to resolve this, from the client side, is to delete the file. This will cause the metadata to be in-sync. Not looked into this yet, but it does seem that the worker's content gets written, but that the metadata never gets updated. Code that generated this issue. Write path: {code} TachyonFile file = getOrCreate(client, filePath); OutStream os = file.getOutStream(writeType); int length = 10; try { os.write(length); for (int i = 0; i < length; i ++) { os.write(i); } } finally { os.close(); } {code} Utils {code} private static TachyonFile getOrCreate(TachyonFS client, String filePath) throws IOException { TachyonFile file = client.getFile(filePath); if (file == null) { int fileId = client.createFile(filePath); file = client.getFile(fileId); } return file; } {code} Read path {code} TachyonFile file = client.getFile(filePath); InStream input = file.getInStream(readType); try { int length = input.read(); for (int i = 0; i < length; i ++) { System.out.println(input.read()); } } finally { input.close(); } {code}

    JIRA | 3 years ago | David Capwell
    java.io.IOException: Offset(0) plus length(80) is larger than file length(11)

    Root Cause Analysis

    1. java.io.IOException

      Offset(0) plus length(80) is larger than file length(11)

      at tachyon.client.TachyonFS.readLocalByteBuffer()
    2. Tachyon Project Core
      BasicNonByteBufferOperations.main
      1. tachyon.client.TachyonFS.readLocalByteBuffer(TachyonFS.java:1152)
      2. tachyon.client.TachyonFile.readLocalByteBuffer(TachyonFile.java:323)
      3. tachyon.client.BlockInStream.get(BlockInStream.java:60)
      4. tachyon.client.TachyonFile.getInStream(TachyonFile.java:132)
      5. tachyon.examples.BasicNonByteBufferOperations.read(BasicNonByteBufferOperations.java:53)
      6. tachyon.examples.BasicNonByteBufferOperations.main(BasicNonByteBufferOperations.java:22)
      6 frames
    3. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:606)
      4 frames
    4. IDEA
      AppMain.main
      1. com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
      1 frame