java.lang.OutOfMemoryError: Java heap space

Pentaho BI Platform Tracking | Sean Pieper | 9 years ago
  1. 0

    This may be related to PDI-605, but the crash is happening on execution of a javascript step, so I think it is different-- I can try updating to a more recent release next week if that seems likely to fix the problem, but my co-worker says he has seen similar problems with version 3.0.2 on Windows. I have a job consisting of two transformations and a shell script. The first transformation loads a number of lines from a database, wraps the fields in xml tags, and then invokes a javascript step to concatenate the lines before hitting a blocking step. The javascript looks like this: var call_array; var fixed_xml=new String(xmlvaluename.getString()); fixed_xml=fixed_xml.replace(/CallRecord/g, "ns1:CallRecord"); fixed_xml=fixed_xml.replace(/<ns1:ServiceStart><\/ns1:ServiceStart><ns1:ServiceEnd><\/ns1:ServiceEnd>/, "<ns1:ServiceStart>"+QueueEnd.getString()+ "</ns1:ServiceStart><ns1:ServiceEnd>"+ QueueEnd.getString()+ "</ns1:ServiceEnd>"); if(call_array==undefined) { call_array=fixed_xml; } else { call_array=call_array+fixed_xml; } The output of this transformation is saved in a file, which is then loaded by a PHP script which uploads the data to a webservice, and the final transformation logs the last successfully uploaded row in a database table. I had this job configured to repeat with 0 second delay between iterations, and the files generated by the first transformation are about 580K in size. After about 150 iterations, I started getting the following error: 2008/03/13 16:44:30 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : Unexpected error : 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : java.lang.OutOfMemoryError: Java heap space 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at java.lang.String.concat(String.java:1831) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.ScriptRuntime.add(ScriptRuntime.java:2296) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.gen.c209._c0(script:14) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.gen.c209.call(script) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:340) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:2758) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.gen.c209.call(script) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.gen.c209.exec(script) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.addValues(ScriptValuesMod.java:356) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.processRow(ScriptValuesMod.java:738) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.run(ScriptValuesMod.java:798) 2008/03/13 16:44:46 - AC2 File Create - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : Errors detected! 2008/03/13 16:44:46 - AC2 File Create - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : Errors detected! 2008/03/13 16:44:46 - AC2 File Create - Transformation detected 1 steps with errors! 2008/03/13 16:44:46 - AC2 File Create - Transformation is killing the other steps! but then things would clear up and execute ok for a bit. 2008/03/13 16:44:49 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:45:13 - AC2 Log Upload - Dispatching started for transformation [AC2 Log Upload] 2008/03/13 16:45:20 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:45:43 - AC2 Log Upload - Dispatching started for transformation [AC2 Log Upload] 2008/03/13 16:45:52 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:46:19 - AC2 Log Upload - Dispatching started for transformation [AC2 Log Upload] 2008/03/13 16:46:28 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:46:56 - AC2 Log Upload - Dispatching started for transformation [AC2 Log Upload] 2008/03/13 16:47:05 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:47:30 - AC2 Log Upload - Dispatching started for transformation [AC2 Log Upload] and then I'd get the same out of heap errors again. My co-worker says he has seen similar transient problems with version 3.0.2 on Windows. I'm guessing this has something to do with how memory being used by the Java scripts is being de-allocated and garbage collected, and I may be punishing the system by having no delay between job iterations, but it makes me concerned because we would like to continuously run this job at a lower pace.

    Pentaho BI Platform Tracking | 9 years ago | Sean Pieper
    java.lang.OutOfMemoryError: Java heap space
  2. 0

    This may be related to PDI-605, but the crash is happening on execution of a javascript step, so I think it is different-- I can try updating to a more recent release next week if that seems likely to fix the problem, but my co-worker says he has seen similar problems with version 3.0.2 on Windows. I have a job consisting of two transformations and a shell script. The first transformation loads a number of lines from a database, wraps the fields in xml tags, and then invokes a javascript step to concatenate the lines before hitting a blocking step. The javascript looks like this: var call_array; var fixed_xml=new String(xmlvaluename.getString()); fixed_xml=fixed_xml.replace(/CallRecord/g, "ns1:CallRecord"); fixed_xml=fixed_xml.replace(/<ns1:ServiceStart><\/ns1:ServiceStart><ns1:ServiceEnd><\/ns1:ServiceEnd>/, "<ns1:ServiceStart>"+QueueEnd.getString()+ "</ns1:ServiceStart><ns1:ServiceEnd>"+ QueueEnd.getString()+ "</ns1:ServiceEnd>"); if(call_array==undefined) { call_array=fixed_xml; } else { call_array=call_array+fixed_xml; } The output of this transformation is saved in a file, which is then loaded by a PHP script which uploads the data to a webservice, and the final transformation logs the last successfully uploaded row in a database table. I had this job configured to repeat with 0 second delay between iterations, and the files generated by the first transformation are about 580K in size. After about 150 iterations, I started getting the following error: 2008/03/13 16:44:30 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : Unexpected error : 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : java.lang.OutOfMemoryError: Java heap space 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at java.lang.String.concat(String.java:1831) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.ScriptRuntime.add(ScriptRuntime.java:2296) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.gen.c209._c0(script:14) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.gen.c209.call(script) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:340) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:2758) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.gen.c209.call(script) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.mozilla.javascript.gen.c209.exec(script) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.addValues(ScriptValuesMod.java:356) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.processRow(ScriptValuesMod.java:738) 2008/03/13 16:44:46 - concatenate xml.0 - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : at org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.run(ScriptValuesMod.java:798) 2008/03/13 16:44:46 - AC2 File Create - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : Errors detected! 2008/03/13 16:44:46 - AC2 File Create - ERROR (version 3.0.1, build 534 from 2007/12/12 12:28:23) : Errors detected! 2008/03/13 16:44:46 - AC2 File Create - Transformation detected 1 steps with errors! 2008/03/13 16:44:46 - AC2 File Create - Transformation is killing the other steps! but then things would clear up and execute ok for a bit. 2008/03/13 16:44:49 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:45:13 - AC2 Log Upload - Dispatching started for transformation [AC2 Log Upload] 2008/03/13 16:45:20 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:45:43 - AC2 Log Upload - Dispatching started for transformation [AC2 Log Upload] 2008/03/13 16:45:52 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:46:19 - AC2 Log Upload - Dispatching started for transformation [AC2 Log Upload] 2008/03/13 16:46:28 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:46:56 - AC2 Log Upload - Dispatching started for transformation [AC2 Log Upload] 2008/03/13 16:47:05 - AC2 File Create - Dispatching started for transformation [AC2 File Create] 2008/03/13 16:47:30 - AC2 Log Upload - Dispatching started for transformation [AC2 Log Upload] and then I'd get the same out of heap errors again. My co-worker says he has seen similar transient problems with version 3.0.2 on Windows. I'm guessing this has something to do with how memory being used by the Java scripts is being de-allocated and garbage collected, and I may be punishing the system by having no delay between job iterations, but it makes me concerned because we would like to continuously run this job at a lower pace.

    Pentaho BI Platform Tracking | 9 years ago | Sean Pieper
    java.lang.OutOfMemoryError: Java heap space
  3. 0

    how to avoid out of memory with infinite loop

    Google Groups | 8 years ago | Rhino user
    java.lang.OutOfMemoryError: Java heap space
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    6 Common Errors in Setting Java Heap Size - Paul Gu|blog

    paulgu.com | 3 weeks ago
    java.lang.OutOfMemoryError: Java heap space
  6. 0

    Java How To ...: 6 Common Errors in Setting Java Heap Size

    blogspot.com | 8 months ago
    java.lang.OutOfMemoryError: Java heap space

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.OutOfMemoryError

      Java heap space

      at java.lang.String.concat()
    2. Java RT
      String.concat
      1. java.lang.String.concat(String.java:1831)
      1 frame
    3. Rhino
      ScriptRuntime.add
      1. org.mozilla.javascript.ScriptRuntime.add(ScriptRuntime.java:2296)
      1 frame
    4. org.mozilla.javascript
      c209.call
      1. org.mozilla.javascript.gen.c209._c0(script:14)
      2. org.mozilla.javascript.gen.c209.call(script)
      2 frames
    5. Rhino
      ScriptRuntime.doTopCall
      1. org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:340)
      2. org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:2758)
      2 frames
    6. org.mozilla.javascript
      c209.exec
      1. org.mozilla.javascript.gen.c209.call(script)
      2. org.mozilla.javascript.gen.c209.exec(script)
      2 frames
    7. org.pentaho.di
      ScriptValuesMod.run
      1. org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.addValues(ScriptValuesMod.java:356)
      2. org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.processRow(ScriptValuesMod.java:738)
      3. org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.run(ScriptValuesMod.java:798)
      3 frames