java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz

JIRA | Arno Candel | 3 years ago
  1. 0

    master 5d7e1d5f5ef1974cbb Parse the attached file with Beta->ImportFiles2 (or with TestUtil.parseFromH2OFolder("irislarge.csv.gz"); Note that this test passes for an uncompressed file, or if ParseVA is used. How the file was made: cat the iris files together many times (Note: Will parse if using fewer than 16 in the command below: need to have > 2^23 rows, and probably requires a very small zip file (< 4MB?)) > cd smalldata/iris > cp iris.csv tmp; for i in {1..16}; do cat tmp tmp >> tmp2; mv tmp2 tmp; done; cat tmp | gzip > irislarge.csv.gz && rm tmp Stacktrace: 06:25:34.482 127.0.0.1:54321 # Session GET /2/Parse2.query source_key=nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz 06:25:35.403 127.0.0.1:54321 # Session GET /2/Parse2.html single_quotes=0 delete_on_done=1 header=0 separator=44 parser_type=CSV destination_key=irislarge.csv.hex source_key=nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz at water.DRemoteTask.block(DRemoteTask.java:81) at jsr166y.ForkJoinPool.managedBlock(ForkJoinPool.java:2803) at water.DRemoteTask.invoke(DRemoteTask.java:60) at water.fvec.ParseDataset2.parse_impl(ParseDataset2.java:288) at water.fvec.ParseDataset2.access$000(ParseDataset2.java:25) at water.fvec.ParseDataset2$ParserFJTask.compute2(ParseDataset2.java:87) at water.H2O$H2OCountedCompleter.compute(H2O.java:712) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) Caused by: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz at water.fvec.NewChunk.append2slow(NewChunk.java:117) at water.fvec.NewChunk.append2(NewChunk.java:88) at water.fvec.NewChunk.addNum(NewChunk.java:79) at water.fvec.ParseDataset2$FVecDataOut.addNumCol(ParseDataset2.java:630) at water.parser.CsvParser.parallelParse(CsvParser.java:276) at water.parser.CustomParser.streamParse(CustomParser.java:195) at water.fvec.ParseDataset2$MultiFileParseTask.streamParse(ParseDataset2.java:486) at water.fvec.ParseDataset2$MultiFileParseTask.map(ParseDataset2.java:438) at water.MRTask.lcompute(MRTask.java:66) at water.DRemoteTask.compute2(DRemoteTask.java:91) ... 6 more

    JIRA | 3 years ago | Arno Candel
    java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz
  2. 0

    master 5d7e1d5f5ef1974cbb Parse the attached file with Beta->ImportFiles2 (or with TestUtil.parseFromH2OFolder("irislarge.csv.gz"); Note that this test passes for an uncompressed file, or if ParseVA is used. How the file was made: cat the iris files together many times (Note: Will parse if using fewer than 16 in the command below: need to have > 2^23 rows, and probably requires a very small zip file (< 4MB?)) > cd smalldata/iris > cp iris.csv tmp; for i in {1..16}; do cat tmp tmp >> tmp2; mv tmp2 tmp; done; cat tmp | gzip > irislarge.csv.gz && rm tmp Stacktrace: 06:25:34.482 127.0.0.1:54321 # Session GET /2/Parse2.query source_key=nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz 06:25:35.403 127.0.0.1:54321 # Session GET /2/Parse2.html single_quotes=0 delete_on_done=1 header=0 separator=44 parser_type=CSV destination_key=irislarge.csv.hex source_key=nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz at water.DRemoteTask.block(DRemoteTask.java:81) at jsr166y.ForkJoinPool.managedBlock(ForkJoinPool.java:2803) at water.DRemoteTask.invoke(DRemoteTask.java:60) at water.fvec.ParseDataset2.parse_impl(ParseDataset2.java:288) at water.fvec.ParseDataset2.access$000(ParseDataset2.java:25) at water.fvec.ParseDataset2$ParserFJTask.compute2(ParseDataset2.java:87) at water.H2O$H2OCountedCompleter.compute(H2O.java:712) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) Caused by: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz at water.fvec.NewChunk.append2slow(NewChunk.java:117) at water.fvec.NewChunk.append2(NewChunk.java:88) at water.fvec.NewChunk.addNum(NewChunk.java:79) at water.fvec.ParseDataset2$FVecDataOut.addNumCol(ParseDataset2.java:630) at water.parser.CsvParser.parallelParse(CsvParser.java:276) at water.parser.CustomParser.streamParse(CustomParser.java:195) at water.fvec.ParseDataset2$MultiFileParseTask.streamParse(ParseDataset2.java:486) at water.fvec.ParseDataset2$MultiFileParseTask.map(ParseDataset2.java:438) at water.MRTask.lcompute(MRTask.java:66) at water.DRemoteTask.compute2(DRemoteTask.java:91) ... 6 more

    JIRA | 3 years ago | Arno Candel
    java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz
  3. 0

    Failed test_billion_rows_fvec.Basic.test_billion_rows_fvec (from nosetests) Failing for the past 3 builds (Since Unstable#897 ) Took 5.7 sec. add description Error Message check_sandbox_for_errors: Errors in sandbox stdout or stderr (or R stdout/stderr). Could have occurred at any prior time java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://home/0xdiag/home-0xdiag-datasets/standard/billion_rows.csv.gz 05:07:59.712 FJ-0-5 INFO WATER: at water.fvec.NewChunk.append2slow(NewChunk.java:325) 05:07:59.714 FJ-0-5 INFO WATER: at water.fvec.NewChunk.append2(NewChunk.java:270) 05:07:59.714 FJ-0-5 INFO WATER: at water.fvec.NewChunk.addNum(NewChunk.java:172) 05:07:59.715 FJ-0-5 INFO WATER: at water.fvec.ParseDataset2$FVecDataOut.addNumCol(ParseDataset2.java:847) 05:07:59.715 FJ-0-5 INFO WATER: at water.parser.CsvParser.parallelParse(CsvParser.java:274) 05:07:59.716 FJ-0-5 INFO WATER: at water.parser.CustomParser.streamParse2(CustomParser.java:233) 05:07:59.717 FJ-0-5 INFO WATER: at water.fvec.ParseDataset2$MultiFileParseTask.streamParse(ParseDataset2.java:623) 05:07:59.717 FJ-0-5 INFO WATER: at water.fvec.ParseDataset2$MultiFileParseTask.map(ParseDataset2.java:574) 05:07:59.718 FJ-0-5 INFO WATER: at water.MRTask.lcompute(MRTask.java:68) 05:07:59.719 FJ-0-5 INFO WATER: at water.DRemoteTask.compute2(DRemoteTask.java:91) 05:07:59.720 FJ-0-5 INFO WATER: at water.H2O$H2OCountedCompleter.compute(H2O.java:634) 05:07:59.720 FJ-0-5 INFO WATER: at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) 05:07:59.721 FJ-0-5 INFO WATER: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 05:07:59.722 FJ-0-5 INFO WATER: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) 05:07:59.722 FJ-0-5 INFO WATER: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 05:07:59.723 FJ-0-5 INFO WATER: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 05:07:59.725 FJ-0-7 ERRR WATER: Job $03010a89477532d4ffffffff$_8dfd51c33e26b0aebc1430f5c741489e(ParseDataset2) failed. 05:07:59.725 FJ-0-7 ERRR WATER: Got exception 'class java.lang.RuntimeException', with msg 'java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://home/0xdiag/home-0xdiag-datasets/standard/billion_rows.csv.gz' + java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://home/0xdiag/home-0xdiag-datasets/standard/billion_rows.csv.gz + at water.DRemoteTask.block(DRemoteTask.java:81) + at jsr166y.ForkJoinPool.managedBlock(ForkJoinPool.java:2803) + at water.DRemoteTask.invoke(DRemoteTask.java:60) + at water.fvec.ParseDataset2.parse_impl(ParseDataset2.java:410) + at water.fvec.ParseDataset2.access$000(ParseDataset2.java:24) + at water.fvec.ParseDataset2$ParserFJTask.compute2(ParseDataset2.java:120) + at water.H2O$H2OCountedCompleter.compute(H2O.java:634) + at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) + at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) + at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) + at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) + at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) + Caused by: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://home/0xdiag/home-0xdiag-datasets/standard/billion_rows.csv.gz + at water.fvec.NewChunk.append2slow(NewChunk.java:325) + at water.fvec.NewChunk.append2(NewChunk.java:270) + at water.fvec.NewChunk.addNum(NewChunk.java:172) + at water.fvec.ParseDataset2$FVecDataOut.addNumCol(ParseDataset2.java:847) + at water.parser.CsvParser.parallelParse(CsvParser.java:274) + at water.parser.CustomParser.streamParse2(CustomParser.java:233) + at water.fvec.ParseDataset2$MultiFileParseTask.streamParse(ParseDataset2.java:623) + at water.fvec.ParseDataset2$MultiFileParseTask.map(ParseDataset2.java:574) + at water.MRTask.lcompute(MRTask.java:68) + at water.DRemoteTask.compute2(DRemoteTask.java:91) + ... 6 more

    JIRA | 2 years ago | Tom Kraljevic
    java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://home/0xdiag/home-0xdiag-datasets/standard/billion_rows.csv.gz
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Failed test_billion_rows_fvec.Basic.test_billion_rows_fvec (from nosetests) Failing for the past 3 builds (Since Unstable#897 ) Took 5.7 sec. add description Error Message check_sandbox_for_errors: Errors in sandbox stdout or stderr (or R stdout/stderr). Could have occurred at any prior time java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://home/0xdiag/home-0xdiag-datasets/standard/billion_rows.csv.gz 05:07:59.712 FJ-0-5 INFO WATER: at water.fvec.NewChunk.append2slow(NewChunk.java:325) 05:07:59.714 FJ-0-5 INFO WATER: at water.fvec.NewChunk.append2(NewChunk.java:270) 05:07:59.714 FJ-0-5 INFO WATER: at water.fvec.NewChunk.addNum(NewChunk.java:172) 05:07:59.715 FJ-0-5 INFO WATER: at water.fvec.ParseDataset2$FVecDataOut.addNumCol(ParseDataset2.java:847) 05:07:59.715 FJ-0-5 INFO WATER: at water.parser.CsvParser.parallelParse(CsvParser.java:274) 05:07:59.716 FJ-0-5 INFO WATER: at water.parser.CustomParser.streamParse2(CustomParser.java:233) 05:07:59.717 FJ-0-5 INFO WATER: at water.fvec.ParseDataset2$MultiFileParseTask.streamParse(ParseDataset2.java:623) 05:07:59.717 FJ-0-5 INFO WATER: at water.fvec.ParseDataset2$MultiFileParseTask.map(ParseDataset2.java:574) 05:07:59.718 FJ-0-5 INFO WATER: at water.MRTask.lcompute(MRTask.java:68) 05:07:59.719 FJ-0-5 INFO WATER: at water.DRemoteTask.compute2(DRemoteTask.java:91) 05:07:59.720 FJ-0-5 INFO WATER: at water.H2O$H2OCountedCompleter.compute(H2O.java:634) 05:07:59.720 FJ-0-5 INFO WATER: at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) 05:07:59.721 FJ-0-5 INFO WATER: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 05:07:59.722 FJ-0-5 INFO WATER: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) 05:07:59.722 FJ-0-5 INFO WATER: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 05:07:59.723 FJ-0-5 INFO WATER: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 05:07:59.725 FJ-0-7 ERRR WATER: Job $03010a89477532d4ffffffff$_8dfd51c33e26b0aebc1430f5c741489e(ParseDataset2) failed. 05:07:59.725 FJ-0-7 ERRR WATER: Got exception 'class java.lang.RuntimeException', with msg 'java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://home/0xdiag/home-0xdiag-datasets/standard/billion_rows.csv.gz' + java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://home/0xdiag/home-0xdiag-datasets/standard/billion_rows.csv.gz + at water.DRemoteTask.block(DRemoteTask.java:81) + at jsr166y.ForkJoinPool.managedBlock(ForkJoinPool.java:2803) + at water.DRemoteTask.invoke(DRemoteTask.java:60) + at water.fvec.ParseDataset2.parse_impl(ParseDataset2.java:410) + at water.fvec.ParseDataset2.access$000(ParseDataset2.java:24) + at water.fvec.ParseDataset2$ParserFJTask.compute2(ParseDataset2.java:120) + at water.H2O$H2OCountedCompleter.compute(H2O.java:634) + at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) + at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) + at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) + at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) + at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) + Caused by: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://home/0xdiag/home-0xdiag-datasets/standard/billion_rows.csv.gz + at water.fvec.NewChunk.append2slow(NewChunk.java:325) + at water.fvec.NewChunk.append2(NewChunk.java:270) + at water.fvec.NewChunk.addNum(NewChunk.java:172) + at water.fvec.ParseDataset2$FVecDataOut.addNumCol(ParseDataset2.java:847) + at water.parser.CsvParser.parallelParse(CsvParser.java:274) + at water.parser.CustomParser.streamParse2(CustomParser.java:233) + at water.fvec.ParseDataset2$MultiFileParseTask.streamParse(ParseDataset2.java:623) + at water.fvec.ParseDataset2$MultiFileParseTask.map(ParseDataset2.java:574) + at water.MRTask.lcompute(MRTask.java:68) + at water.DRemoteTask.compute2(DRemoteTask.java:91) + ... 6 more

    JIRA | 2 years ago | Tom Kraljevic
    java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 while mapping key nfs://home/0xdiag/home-0xdiag-datasets/standard/billion_rows.csv.gz
  6. 0

    repros every time. single jvm create 9M vector with Rapids, then add it to itself test does some other Rapids first, but eventually fails here cd h2o-dev/py2/testdir_single_jvm python test_rapids_vec_fail1.py if connecting to existing localhost:54321 python test_rapids_vec_fail1.py -uc rapids parameters: {'funs': None, 'ast': '(= !v (c {(: #0 #8999999)})'} exec: { "ast": "(= !v (c {(: #0 #8999999)})", "col_names": [ "C1" ], "exception": null, "funs": null, "funstr": null, "key": { "name": "v" }, "num_cols": 1, "num_rows": 9000000, "result": "", "scalar": 0.0, "schema_name": "RapidsV1", "schema_type": "Rapids", "schema_version": 1, "string": null } rapids parameters: {'funs': None, 'ast': '(= !v (+ $v $v))'} ERROR 11-23 17:15:04.468 192.168.0.34:54333 10641 # Session ERRR: java.lang.RuntimeException: water.DException$DistributedException: from /192.168.0.34:54333; by class water.rapids.ASTBinOp$1; class java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608 at water.fvec.NewChunk.append2slowd(NewChunk.java:392) at water.fvec.NewChunk.addNum(NewChunk.java:239) at water.rapids.ASTBinOp$1.map(ASTOp.java:921) at water.MRTask.compute2(MRTask.java:426) at water.H2O$H2OCountedCompleter.compute(H2O.java:566) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) ERROR

    JIRA | 2 years ago | Kevin Normoyle
    java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 8388608

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.ArrayIndexOutOfBoundsException

      Array index out of range: 8388608 while mapping key nfs://Users/arno/h2o/./smalldata/iris/irislarge.csv.gz

      at water.fvec.NewChunk.append2slow()
    2. water.fvec
      ParseDataset2$FVecDataOut.addNumCol
      1. water.fvec.NewChunk.append2slow(NewChunk.java:117)
      2. water.fvec.NewChunk.append2(NewChunk.java:88)
      3. water.fvec.NewChunk.addNum(NewChunk.java:79)
      4. water.fvec.ParseDataset2$FVecDataOut.addNumCol(ParseDataset2.java:630)
      4 frames
    3. water.parser
      CustomParser.streamParse
      1. water.parser.CsvParser.parallelParse(CsvParser.java:276)
      2. water.parser.CustomParser.streamParse(CustomParser.java:195)
      2 frames
    4. water.fvec
      ParseDataset2$MultiFileParseTask.map
      1. water.fvec.ParseDataset2$MultiFileParseTask.streamParse(ParseDataset2.java:486)
      2. water.fvec.ParseDataset2$MultiFileParseTask.map(ParseDataset2.java:438)
      2 frames
    5. water
      H2O$H2OCountedCompleter.compute
      1. water.MRTask.lcompute(MRTask.java:66)
      2. water.DRemoteTask.compute2(DRemoteTask.java:91)
      3. water.H2O$H2OCountedCompleter.compute(H2O.java:712)
      3 frames
    6. jsr166y
      ForkJoinWorkerThread.run
      1. jsr166y.CountedCompleter.exec(CountedCompleter.java:429)
      2. jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
      3. jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
      4. jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
      5. jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
      5 frames