java.lang.OutOfMemoryError: GC overhead limit exceeded

Pentaho BI Platform Tracking | Alexander Schurman | 5 months ago
  1. 0

    When EXCEL Input step is used with NON Local system location, in other words, with VFS paths it fails. 2 cases below When Streaming Mode is used, the most likely error is "Zip File is closed". This was tested with VFS: * ZIP * TMP * HDFS Error for ZIP Closed: {code} 2016/07/06 13:02:11 - Spoon - Transformation opened. 2016/07/06 13:02:11 - Spoon - Launching transformation [excel]... 2016/07/06 13:02:11 - Spoon - Started the transformation execution. 2016/07/06 13:02:11 - excel - Dispatching started for transformation [excel] 2016/07/06 13:02:12 - Microsoft Excel Input.0 - ERROR (version 6.1.0.3-223, build 1 from 2016-06-16 15.11.37 by buildguy) : Error processing row from Excel file [zip:file:///C:/tmp/DB/sample file.zip!/sample file.xlsx] : org.pentaho.di.core.exception.KettleException: 2016/07/06 13:02:12 - Microsoft Excel Input.0 - java.lang.IllegalStateException: Zip File is closed 2016/07/06 13:02:12 - Microsoft Excel Input.0 - Zip File is closed 2016/07/06 13:02:12 - Microsoft Excel Input.0 - ERROR (version 6.1.0.3-223, build 1 from 2016-06-16 15.11.37 by buildguy) : org.pentaho.di.core.exception.KettleException: 2016/07/06 13:02:12 - Microsoft Excel Input.0 - java.lang.IllegalStateException: Zip File is closed 2016/07/06 13:02:12 - Microsoft Excel Input.0 - Zip File is closed 2016/07/06 13:02:12 - Microsoft Excel Input.0 - 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at org.pentaho.di.trans.steps.excelinput.staxpoi.StaxPoiWorkbook.<init>(StaxPoiWorkbook.java:81) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at org.pentaho.di.trans.steps.excelinput.WorkbookFactory.getWorkbook(WorkbookFactory.java:43) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at org.pentaho.di.trans.steps.excelinput.ExcelInput.getRowFromWorkbooks(ExcelInput.java:552) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at org.pentaho.di.trans.steps.excelinput.ExcelInput.processRow(ExcelInput.java:430) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at java.lang.Thread.run(Unknown Source) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - Caused by: java.lang.IllegalStateException: Zip File is closed 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at org.apache.poi.openxml4j.util.ZipFileZipEntrySource.getEntries(ZipFileZipEntrySource.java:45) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at org.apache.poi.openxml4j.opc.ZipPackage.getPartsImpl(ZipPackage.java:186) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at org.apache.poi.openxml4j.opc.OPCPackage.getParts(OPCPackage.java:684) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at org.apache.poi.openxml4j.opc.OPCPackage.open(OPCPackage.java:227) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at org.apache.poi.openxml4j.opc.OPCPackage.open(OPCPackage.java:187) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - at org.pentaho.di.trans.steps.excelinput.staxpoi.StaxPoiWorkbook.<init>(StaxPoiWorkbook.java:78) 2016/07/06 13:02:12 - Microsoft Excel Input.0 - ... 5 more 2016/07/06 13:02:12 - Microsoft Excel Input.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=1) 2016/07/06 13:02:12 - excel - ERROR (version 6.1.0.3-223, build 1 from 2016-06-16 15.11.37 by buildguy) : Errors detected! 2016/07/06 13:02:12 - Spoon - The transformation has finished!! 2016/07/06 13:02:12 - excel - ERROR (version 6.1.0.3-223, build 1 from 2016-06-16 15.11.37 by buildguy) : Errors detected! 2016/07/06 13:02:12 - excel - ERROR (version 6.1.0.3-223, build 1 from 2016-06-16 15.11.37 by buildguy) : Errors detected! 2016/07/06 13:02:12 - excel - Transformation detected one or more steps with errors. 2016/07/06 13:02:12 - excel - Transformation is killing the other steps! {code} When Non Streaming Mode is used, the amount of MEMORY is SO HIGH that normally an OUT OF MEMORY ERROR is thrown. This happens on "LARGE" XML file over *15 Mb*. User case is about 80Mb files {code} 2016/07/06 13:13:32 - Spoon - Transformation opened. 2016/07/06 13:13:32 - Spoon - Launching transformation [excel]... 2016/07/06 13:13:32 - Spoon - Started the transformation execution. 2016/07/06 13:13:32 - excel - Dispatching started for transformation [excel] 2016/07/06 13:14:39 - Microsoft Excel Input.0 - ERROR (version 6.1.0.3-223, build 1 from 2016-06-16 15.11.37 by buildguy) : UnexpectedError: 2016/07/06 13:14:39 - Microsoft Excel Input.0 - ERROR (version 6.1.0.3-223, build 1 from 2016-06-16 15.11.37 by buildguy) : java.lang.OutOfMemoryError: GC overhead limit exceeded 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at java.lang.String.substring(Unknown Source) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.ss.util.CellReference.separateRefParts(CellReference.java:361) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.ss.util.CellReference.<init>(CellReference.java:92) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.xssf.usermodel.XSSFCell.<init>(XSSFCell.java:107) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.xssf.usermodel.XSSFRow.<init>(XSSFRow.java:68) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.xssf.usermodel.XSSFSheet.initRows(XSSFSheet.java:238) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.xssf.usermodel.XSSFSheet.read(XSSFSheet.java:201) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.xssf.usermodel.XSSFSheet.onDocumentRead(XSSFSheet.java:188) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.xssf.usermodel.XSSFWorkbook.onDocumentRead(XSSFWorkbook.java:378) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:166) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.xssf.usermodel.XSSFWorkbook.<init>(XSSFWorkbook.java:241) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.ss.usermodel.WorkbookFactory.create(WorkbookFactory.java:175) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.apache.poi.ss.usermodel.WorkbookFactory.create(WorkbookFactory.java:143) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.pentaho.di.trans.steps.excelinput.poi.PoiWorkbook.<init>(PoiWorkbook.java:78) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.pentaho.di.trans.steps.excelinput.WorkbookFactory.getWorkbook(WorkbookFactory.java:41) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.pentaho.di.trans.steps.excelinput.ExcelInput.getRowFromWorkbooks(ExcelInput.java:552) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.pentaho.di.trans.steps.excelinput.ExcelInput.processRow(ExcelInput.java:430) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - at java.lang.Thread.run(Unknown Source) 2016/07/06 13:14:39 - Microsoft Excel Input.0 - Finished processing (I=2193, O=0, R=0, W=2193, U=0, E=1) 2016/07/06 13:14:39 - Delay row.0 - Finished processing (I=0, O=0, R=2193, W=2193, U=0, E=0) 2016/07/06 13:14:39 - Dummy (do nothing).0 - Finished processing (I=0, O=0, R=2193, W=2193, U=0, E=0) 2016/07/06 13:14:39 - excel - Transformation detected one or more steps with errors. 2016/07/06 13:14:39 - excel - Transformation is killing the other steps! 2016/07/06 13:14:39 - excel - ERROR (version 6.1.0.3-223, build 1 from 2016-06-16 15.11.37 by buildguy) : Errors detected! 2016/07/06 13:14:39 - Spoon - The transformation has finished!! 2016/07/06 13:14:39 - excel - ERROR (version 6.1.0.3-223, build 1 from 2016-06-16 15.11.37 by buildguy) : Errors detected! 2016/07/06 13:14:40 - excel - ERROR (version 6.1.0.3-223, build 1 from 2016-06-16 15.11.37 by buildguy) : Errors detected! {code}

    Pentaho BI Platform Tracking | 5 months ago | Alexander Schurman
    java.lang.OutOfMemoryError: GC overhead limit exceeded
  2. 0

    Memory problems when calling Taro.readxl a second time

    GitHub | 2 years ago | mbeltagy
    java.lang.OutOfMemoryError: GC overhead limit exceeded
  3. 0

    GC overhead limit exceeded

    GitHub | 3 years ago | navneetccna
    java.lang.OutOfMemoryError: GC overhead limit exceeded
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Alfresco GC overhead limit exceeded

    Stack Overflow | 2 years ago | OzkanCiftci
    java.lang.OutOfMemoryError: GC overhead limit exceeded

  1. silex 2 times, last 2 months ago
1 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.lang.OutOfMemoryError

    GC overhead limit exceeded

    at java.lang.String.substring()
  2. Java RT
    String.substring
    1. java.lang.String.substring(Unknown Source)
    1 frame
  3. POI
    CellReference.<init>
    1. org.apache.poi.ss.util.CellReference.separateRefParts(CellReference.java:361)
    2. org.apache.poi.ss.util.CellReference.<init>(CellReference.java:92)
    2 frames
  4. Apache POI
    XSSFWorkbook.onDocumentRead
    1. org.apache.poi.xssf.usermodel.XSSFCell.<init>(XSSFCell.java:107)
    2. org.apache.poi.xssf.usermodel.XSSFRow.<init>(XSSFRow.java:68)
    3. org.apache.poi.xssf.usermodel.XSSFSheet.initRows(XSSFSheet.java:238)
    4. org.apache.poi.xssf.usermodel.XSSFSheet.read(XSSFSheet.java:201)
    5. org.apache.poi.xssf.usermodel.XSSFSheet.onDocumentRead(XSSFSheet.java:188)
    6. org.apache.poi.xssf.usermodel.XSSFWorkbook.onDocumentRead(XSSFWorkbook.java:378)
    6 frames
  5. POI
    POIXMLDocument.load
    1. org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:166)
    1 frame
  6. Apache POI
    XSSFWorkbook.<init>
    1. org.apache.poi.xssf.usermodel.XSSFWorkbook.<init>(XSSFWorkbook.java:241)
    1 frame
  7. POI
    WorkbookFactory.create
    1. org.apache.poi.ss.usermodel.WorkbookFactory.create(WorkbookFactory.java:175)
    2. org.apache.poi.ss.usermodel.WorkbookFactory.create(WorkbookFactory.java:143)
    2 frames
  8. org.pentaho.di
    RunThread.run
    1. org.pentaho.di.trans.steps.excelinput.poi.PoiWorkbook.<init>(PoiWorkbook.java:78)
    2. org.pentaho.di.trans.steps.excelinput.WorkbookFactory.getWorkbook(WorkbookFactory.java:41)
    3. org.pentaho.di.trans.steps.excelinput.ExcelInput.getRowFromWorkbooks(ExcelInput.java:552)
    4. org.pentaho.di.trans.steps.excelinput.ExcelInput.processRow(ExcelInput.java:430)
    5. org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
    5 frames
  9. Java RT
    Thread.run
    1. java.lang.Thread.run(Unknown Source)
    1 frame