java.lang.OutOfMemoryError: Java heap space

Pentaho BI Platform Tracking | Daniel Einspanjer | 9 years ago
  1. 0

    The attached job was running against a large data set (6.5 billion records in 4.5 * 30 * 24 files) and it eventually bombed out around three fourths of the way through after outputting 4760533 rows (it was conducting a 0.001% sampling of the source set). The only thing that should have been retained in memory was the list of files to process and I would expect that memory constraint to be constant. The execution arguments were: export JAVAMAXMEM=4096 ./kitchen.sh -file 'zip:file:///home/etl/sample_2008.zip!/sample_all_2008.kjb' -level Minimal The tail of the console output was: INFO 21-05 10:37:12,972 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample] INFO 21-05 10:42:28,714 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-03.gz INFO 21-05 10:43:37,668 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file] INFO 21-05 12:11:07,905 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample] INFO 21-05 12:18:00,592 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-04.gz INFO 21-05 12:18:30,028 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file] INFO 21-05 14:45:46,102 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample] INFO 21-05 14:54:09,574 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-05.gz INFO 21-05 14:56:07,936 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file] INFO 21-05 20:51:43,483 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample] INFO 21-05 21:01:34,576 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-06.gz INFO 21-05 21:02:43,637 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file] ERROR 21-05 21:23:07,790 (LogWriter.java:println:436) -CSV file input.0 - UnexpectedError: java.lang.OutOfMemoryError: Java heap space ERROR 21-05 21:26:05,630 (LogWriter.java:println:436) -CSV file input.0 - java.lang.OutOfMemoryError: Java heap space at org.pentaho.di.trans.steps.csvinput.CsvInputData.resizeByteBuffer(CsvInputData.java:89) at org.pentaho.di.trans.steps.csvinput.CsvInput.readOneRow(CsvInput.java:465) at org.pentaho.di.trans.steps.csvinput.CsvInput.processRow(CsvInput.java:111) at org.pentaho.di.trans.step.BaseStep.runStepThread(BaseStep.java:2444) at org.pentaho.di.trans.steps.csvinput.CsvInput.run(CsvInput.java:641) INFO 21-05 21:37:59,000 (LogWriter.java:println:439) -sample_one_file - Transformation detected one or more steps with errors. ERROR 21-05 21:38:38,997 (LogWriter.java:println:436) -sample_one_file - Errors detected! INFO 21-05 21:39:18,442 (LogWriter.java:println:439) -sample_one_file - Transformation is killing the other steps! ERROR 21-05 21:40:37,335 (LogWriter.java:println:436) -sample_one_file - Errors detected! INFO 21-05 21:41:58,001 (LogWriter.java:println:439) -Kitchen - Finished! ERROR 21-05 21:41:58,002 (LogWriter.java:println:436) -Kitchen - Finished with errors INFO 21-05 21:41:58,002 (LogWriter.java:println:439) -Kitchen - Start=2008/05/19 17:10:47.428, Stop=2008/05/21 21:41:58.002 INFO 21-05 21:41:58,002 (LogWriter.java:println:439) -Kitchen - Processing ended after 189070 seconds.

    Pentaho BI Platform Tracking | 9 years ago | Daniel Einspanjer
    java.lang.OutOfMemoryError: Java heap space
  2. 0

    The attached job was running against a large data set (6.5 billion records in 4.5 * 30 * 24 files) and it eventually bombed out around three fourths of the way through after outputting 4760533 rows (it was conducting a 0.001% sampling of the source set). The only thing that should have been retained in memory was the list of files to process and I would expect that memory constraint to be constant. The execution arguments were: export JAVAMAXMEM=4096 ./kitchen.sh -file 'zip:file:///home/etl/sample_2008.zip!/sample_all_2008.kjb' -level Minimal The tail of the console output was: INFO 21-05 10:37:12,972 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample] INFO 21-05 10:42:28,714 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-03.gz INFO 21-05 10:43:37,668 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file] INFO 21-05 12:11:07,905 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample] INFO 21-05 12:18:00,592 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-04.gz INFO 21-05 12:18:30,028 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file] INFO 21-05 14:45:46,102 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample] INFO 21-05 14:54:09,574 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-05.gz INFO 21-05 14:56:07,936 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file] INFO 21-05 20:51:43,483 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample] INFO 21-05 21:01:34,576 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-06.gz INFO 21-05 21:02:43,637 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file] ERROR 21-05 21:23:07,790 (LogWriter.java:println:436) -CSV file input.0 - UnexpectedError: java.lang.OutOfMemoryError: Java heap space ERROR 21-05 21:26:05,630 (LogWriter.java:println:436) -CSV file input.0 - java.lang.OutOfMemoryError: Java heap space at org.pentaho.di.trans.steps.csvinput.CsvInputData.resizeByteBuffer(CsvInputData.java:89) at org.pentaho.di.trans.steps.csvinput.CsvInput.readOneRow(CsvInput.java:465) at org.pentaho.di.trans.steps.csvinput.CsvInput.processRow(CsvInput.java:111) at org.pentaho.di.trans.step.BaseStep.runStepThread(BaseStep.java:2444) at org.pentaho.di.trans.steps.csvinput.CsvInput.run(CsvInput.java:641) INFO 21-05 21:37:59,000 (LogWriter.java:println:439) -sample_one_file - Transformation detected one or more steps with errors. ERROR 21-05 21:38:38,997 (LogWriter.java:println:436) -sample_one_file - Errors detected! INFO 21-05 21:39:18,442 (LogWriter.java:println:439) -sample_one_file - Transformation is killing the other steps! ERROR 21-05 21:40:37,335 (LogWriter.java:println:436) -sample_one_file - Errors detected! INFO 21-05 21:41:58,001 (LogWriter.java:println:439) -Kitchen - Finished! ERROR 21-05 21:41:58,002 (LogWriter.java:println:436) -Kitchen - Finished with errors INFO 21-05 21:41:58,002 (LogWriter.java:println:439) -Kitchen - Start=2008/05/19 17:10:47.428, Stop=2008/05/21 21:41:58.002 INFO 21-05 21:41:58,002 (LogWriter.java:println:439) -Kitchen - Processing ended after 189070 seconds.

    Pentaho BI Platform Tracking | 9 years ago | Daniel Einspanjer
    java.lang.OutOfMemoryError: Java heap space
  3. 0

    OutOfMemory Exception : java heap space

    Google Groups | 6 months ago | deepak singh
    java.lang.OutOfMemoryError: Java heap space* *java.lang.OutOfMemoryError: Java heap space*
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    OutOfMemory Exception : java heap space

    Google Groups | 6 months ago | deepak singh
    java.lang.OutOfMemoryError: Java heap space* *java.lang.OutOfMemoryError: Java heap space*
  6. 0

    [exoplatform] [explatform] Memory requirements

    ow2.org | 1 year ago
    java.lang.OutOfMemoryError: Java heap space>java.lang.OutOfMemoryError: Java heap space

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.OutOfMemoryError

      Java heap space

      at org.pentaho.di.trans.steps.csvinput.CsvInputData.resizeByteBuffer()
    2. org.pentaho.di
      CsvInput.run
      1. org.pentaho.di.trans.steps.csvinput.CsvInputData.resizeByteBuffer(CsvInputData.java:89)
      2. org.pentaho.di.trans.steps.csvinput.CsvInput.readOneRow(CsvInput.java:465)
      3. org.pentaho.di.trans.steps.csvinput.CsvInput.processRow(CsvInput.java:111)
      4. org.pentaho.di.trans.step.BaseStep.runStepThread(BaseStep.java:2444)
      5. org.pentaho.di.trans.steps.csvinput.CsvInput.run(CsvInput.java:641)
      5 frames