java.io.IOException: Offset(0) plus length(80) is larger than file length(11)

JIRA | David Capwell | 2 years ago
  1. 0

    This was run on my mac. 14/08/13 09:17:24 WARN : Failed to read local file /Volumes/ramdisk/tachyonworker/3221225472 because: java.io.IOException: Offset(0) plus length(80) is larger than file length(11) at tachyon.client.TachyonFS.readLocalByteBuffer(TachyonFS.java:1152) at tachyon.client.TachyonFile.readLocalByteBuffer(TachyonFile.java:323) at tachyon.client.BlockInStream.get(BlockInStream.java:60) at tachyon.client.TachyonFile.getInStream(TachyonFile.java:132) at tachyon.examples.BasicNonByteBufferOperations.read(BasicNonByteBufferOperations.java:53) at tachyon.examples.BasicNonByteBufferOperations.main(BasicNonByteBufferOperations.java:22) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134) 14/08/13 09:17:24 INFO : Try to find remote worker and read block 3221225472 from 0, with len 80 $ ls -l /Volumes/ramdisk/tachyonworker/3221225472 -rwxrwxrwx 1 dcapwell staff 11 Aug 13 09:17 /Volumes/ramdisk/tachyonworker/3221225472 Master UI bar 80.00 B 1024.00 MB 100% NO 08-13-2014 08:32:50:999 08-13-2014 09:17:24:828 When I click on 'bar' Error: File /foo/bar is not available Can not find the block /foo/bar 0 When I test locally, it seems this happens when you try to override the file. The write path never fails to accept anything thrown at it, but the read path will not see that data ever. Only way to resolve this, from the client side, is to delete the file. This will cause the metadata to be in-sync. Not looked into this yet, but it does seem that the worker's content gets written, but that the metadata never gets updated. Code that generated this issue. Write path: {code} TachyonFile file = getOrCreate(client, filePath); OutStream os = file.getOutStream(writeType); int length = 10; try { os.write(length); for (int i = 0; i < length; i ++) { os.write(i); } } finally { os.close(); } {code} Utils {code} private static TachyonFile getOrCreate(TachyonFS client, String filePath) throws IOException { TachyonFile file = client.getFile(filePath); if (file == null) { int fileId = client.createFile(filePath); file = client.getFile(fileId); } return file; } {code} Read path {code} TachyonFile file = client.getFile(filePath); InStream input = file.getInStream(readType); try { int length = input.read(); for (int i = 0; i < length; i ++) { System.out.println(input.read()); } } finally { input.close(); } {code}

    JIRA | 2 years ago | David Capwell
    java.io.IOException: Offset(0) plus length(80) is larger than file length(11)
  2. 0

    This was run on my mac. 14/08/13 09:17:24 WARN : Failed to read local file /Volumes/ramdisk/tachyonworker/3221225472 because: java.io.IOException: Offset(0) plus length(80) is larger than file length(11) at tachyon.client.TachyonFS.readLocalByteBuffer(TachyonFS.java:1152) at tachyon.client.TachyonFile.readLocalByteBuffer(TachyonFile.java:323) at tachyon.client.BlockInStream.get(BlockInStream.java:60) at tachyon.client.TachyonFile.getInStream(TachyonFile.java:132) at tachyon.examples.BasicNonByteBufferOperations.read(BasicNonByteBufferOperations.java:53) at tachyon.examples.BasicNonByteBufferOperations.main(BasicNonByteBufferOperations.java:22) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134) 14/08/13 09:17:24 INFO : Try to find remote worker and read block 3221225472 from 0, with len 80 $ ls -l /Volumes/ramdisk/tachyonworker/3221225472 -rwxrwxrwx 1 dcapwell staff 11 Aug 13 09:17 /Volumes/ramdisk/tachyonworker/3221225472 Master UI bar 80.00 B 1024.00 MB 100% NO 08-13-2014 08:32:50:999 08-13-2014 09:17:24:828 When I click on 'bar' Error: File /foo/bar is not available Can not find the block /foo/bar 0 When I test locally, it seems this happens when you try to override the file. The write path never fails to accept anything thrown at it, but the read path will not see that data ever. Only way to resolve this, from the client side, is to delete the file. This will cause the metadata to be in-sync. Not looked into this yet, but it does seem that the worker's content gets written, but that the metadata never gets updated. Code that generated this issue. Write path: {code} TachyonFile file = getOrCreate(client, filePath); OutStream os = file.getOutStream(writeType); int length = 10; try { os.write(length); for (int i = 0; i < length; i ++) { os.write(i); } } finally { os.close(); } {code} Utils {code} private static TachyonFile getOrCreate(TachyonFS client, String filePath) throws IOException { TachyonFile file = client.getFile(filePath); if (file == null) { int fileId = client.createFile(filePath); file = client.getFile(fileId); } return file; } {code} Read path {code} TachyonFile file = client.getFile(filePath); InStream input = file.getInStream(readType); try { int length = input.read(); for (int i = 0; i < length; i ++) { System.out.println(input.read()); } } finally { input.close(); } {code}

    JIRA | 2 years ago | David Capwell
    java.io.IOException: Offset(0) plus length(80) is larger than file length(11)
  3. 0

    Frame size error

    Google Groups | 4 months ago | Unknown author
    java.io.IOException: Frame size (67108864) larger than max length (16777216)!
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Re: ClientContext.init() produces error

    Google Groups | 9 months ago | Gene Pang
    java.io.IOException: Frame size (1213486160) larger than max length (16384000)!
  6. 0

    [Help]How to load hive table to alluxio?

    Google Groups | 6 months ago | Leon Zhong
    java.lang.RuntimeException: java.io.IOException: Frame size (67108864) larger than max length (16777216)!

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.io.IOException

      Offset(0) plus length(80) is larger than file length(11)

      at tachyon.client.TachyonFS.readLocalByteBuffer()
    2. Tachyon Project Core
      BasicNonByteBufferOperations.main
      1. tachyon.client.TachyonFS.readLocalByteBuffer(TachyonFS.java:1152)
      2. tachyon.client.TachyonFile.readLocalByteBuffer(TachyonFile.java:323)
      3. tachyon.client.BlockInStream.get(BlockInStream.java:60)
      4. tachyon.client.TachyonFile.getInStream(TachyonFile.java:132)
      5. tachyon.examples.BasicNonByteBufferOperations.read(BasicNonByteBufferOperations.java:53)
      6. tachyon.examples.BasicNonByteBufferOperations.main(BasicNonByteBufferOperations.java:22)
      6 frames
    3. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:606)
      4 frames
    4. IDEA
      AppMain.main
      1. com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
      1 frame