org.ofbiz.core.entity.GenericDataSourceException: SQL Exception occurred on commit (Connection org.hsqldb.jdbc.jdbcConnection@d1b458d is closed.)

Atlassian JIRA | Chris Fuller | 11 months ago
  1. 0

    There is a race condition in Atlassian's OfBiz Entity Engine fork that can result in connections being detected incorrectly as abandoned when they are actually still in use. This is due to certain allowed changes to object visibility as a result of just-in-time compiler optimizations, with the result that the connection is enqueued as abandoned just before the thread that is using it closes it. The most common manifestation of this is a warning message about an abandoned SQL processor: {code} !!! ABANDONED SQLProcessor DETECTED !!! This probably means that somebody forgot to close an EntityListIterator. Connection: org.ofbiz.core.entity.jdbc.interceptors.connection.ConnectionTracker$DelegatingConnectionImpl@13af30e {code} This is particularly likely under heavy use (such as when reindexing) and when the connection is opened and closed in a small function that is likely to inline these operations, such as this from JCHART (leading to [JCHART-462|https://ecosystem.atlassian.net/browse/JCHART-462] as a result): {code} try { sqlProcessor = new SQLProcessor(ENTITY_DS); } finally { ... sqlProcessor.close(); ... } ... {code} In the most extreme cases (typically under heavy load) it is even possible for the connection to be closed by another thread before its connection commits, with actual failed operations as a result: {code} Exception in thread "TestThread[59]" org.ofbiz.core.entity.GenericDataSourceException: SQL Exception occurred on commit (Connection org.hsqldb.jdbc.jdbcConnection@d1b458d is closed.) at org.ofbiz.core.entity.jdbc.SQLProcessor.commit(SQLProcessor.java:211) at org.ofbiz.core.entity.jdbc.SQLProcessor.smartCommit(SQLProcessor.java:362) at org.ofbiz.core.entity.jdbc.SQLProcessor.close(SQLProcessor.java:264) ... Caused by: java.sql.SQLException: Connection org.hsqldb.jdbc.jdbcConnection@d1b458d is closed. at org.apache.commons.dbcp2.DelegatingConnection.checkOpen(DelegatingConnection.java:605) at org.apache.commons.dbcp2.DelegatingConnection.commit(DelegatingConnection.java:362) at org.apache.commons.dbcp2.DelegatingConnection.commit(DelegatingConnection.java:364) at org.ofbiz.core.entity.jdbc.interceptors.connection.DelegatingConnection.commit(DelegatingConnection.java:66) at org.ofbiz.core.entity.jdbc.SQLProcessor.commit(SQLProcessor.java:205) ... {code} This is why using a ReadOnlySQLProcessor works around the problem; the read-only connections do not attempt to commit, so the fact that it was already closed by another thread makes no difference. For the technically savvy, there is a reasonably good and detailed explanation of this rather subtle problem in [this blog post|http://www.infoq.com/articles/Fatal-Flaw-Finalizers-Phantoms]. The fix is to reorder the volatile writes and the release of the connection guard in such a way as to disallow the visibility change.

    Atlassian JIRA | 11 months ago | Chris Fuller
    org.ofbiz.core.entity.GenericDataSourceException: SQL Exception occurred on commit (Connection org.hsqldb.jdbc.jdbcConnection@d1b458d is closed.)
  2. 0

    There is a race condition in Atlassian's OfBiz Entity Engine fork that can result in connections being detected incorrectly as abandoned when they are actually still in use. This is due to certain allowed changes to object visibility as a result of just-in-time compiler optimizations, with the result that the connection is enqueued as abandoned just before the thread that is using it closes it. The most common manifestation of this is a warning message about an abandoned SQL processor: {code} !!! ABANDONED SQLProcessor DETECTED !!! This probably means that somebody forgot to close an EntityListIterator. Connection: org.ofbiz.core.entity.jdbc.interceptors.connection.ConnectionTracker$DelegatingConnectionImpl@13af30e {code} This is particularly likely under heavy use (such as when reindexing) and when the connection is opened and closed in a small function that is likely to inline these operations, such as this from JCHART (leading to [JCHART-462|https://ecosystem.atlassian.net/browse/JCHART-462] as a result): {code} try { sqlProcessor = new SQLProcessor(ENTITY_DS); } finally { ... sqlProcessor.close(); ... } ... {code} In the most extreme cases (typically under heavy load) it is even possible for the connection to be closed by another thread before its connection commits, with actual failed operations as a result: {code} Exception in thread "TestThread[59]" org.ofbiz.core.entity.GenericDataSourceException: SQL Exception occurred on commit (Connection org.hsqldb.jdbc.jdbcConnection@d1b458d is closed.) at org.ofbiz.core.entity.jdbc.SQLProcessor.commit(SQLProcessor.java:211) at org.ofbiz.core.entity.jdbc.SQLProcessor.smartCommit(SQLProcessor.java:362) at org.ofbiz.core.entity.jdbc.SQLProcessor.close(SQLProcessor.java:264) ... Caused by: java.sql.SQLException: Connection org.hsqldb.jdbc.jdbcConnection@d1b458d is closed. at org.apache.commons.dbcp2.DelegatingConnection.checkOpen(DelegatingConnection.java:605) at org.apache.commons.dbcp2.DelegatingConnection.commit(DelegatingConnection.java:362) at org.apache.commons.dbcp2.DelegatingConnection.commit(DelegatingConnection.java:364) at org.ofbiz.core.entity.jdbc.interceptors.connection.DelegatingConnection.commit(DelegatingConnection.java:66) at org.ofbiz.core.entity.jdbc.SQLProcessor.commit(SQLProcessor.java:205) ... {code} This is why using a ReadOnlySQLProcessor works around the problem; the read-only connections do not attempt to commit, so the fact that it was already closed by another thread makes no difference. For the technically savvy, there is a reasonably good and detailed explanation of this rather subtle problem in [this blog post|http://www.infoq.com/articles/Fatal-Flaw-Finalizers-Phantoms]. The fix is to reorder the volatile writes and the release of the connection guard in such a way as to disallow the visibility change.

    Atlassian JIRA | 11 months ago | Chris Fuller
    org.ofbiz.core.entity.GenericDataSourceException: SQL Exception occurred on commit (Connection org.hsqldb.jdbc.jdbcConnection@d1b458d is closed.)
  3. 0

    [JRA-7972] Workflow migrations hang - Atlassian JIRA

    atlassian.com | 12 months ago
    org.ofbiz.core.entity.GenericDataSourceException: SQL Exception occurred on commit (No operations allowed after connection closed.)
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    [JRA-7972] Workflow migrations hang - Atlassian JIRA

    atlassian.com | 12 months ago
    org.ofbiz.core.entity.GenericDataSourceException: SQL Exception occurred on commit (No operations allowed after connection closed.)
  6. 0

    The FogBugz importer is blowing up near the end of a large import: {noformat} 2006-04-06 14:55:04,326 INFO [imports.importer.impl.ImportLogger] Rewritten comment for issue LOGI75-145 (105435.017s)^M 2006-04-06 15:02:56,451 INFO [imports.importer.impl.ImportLogger] Rewritten comment for issue IDEDEV-94 (105907.142s)^M 2006-04-06 15:19:42,591 INFO [imports.importer.impl.ImportLogger] FAILED: Unexpected failure occurred. Importer will stop immediately. Data maybe in an unstable state (106913.282s)^M 2006-04-06 15:19:42,779 INFO [imports.importer.impl.ImportLogger] com.atlassian.jira.exception.DataAccessException: org.ofbiz.core.entity.GenericDataSourceException: SQL Exception while executing the following:SELECT ID, issueid, AUTHOR, actiontype, actionlevel, actionbody, CREATED, actionnum FROM jiraaction WHERE issueid=? AND actiontype=? (I/O Error: Connection reset)^M at com.atlassian.jira.imports.importer.impl.DefaultJiraDataImporter.rewriteOldIssueKeys(DefaultJiraDataImporter.java:652)^M at com.atlassian.jira.imports.importer.impl.DefaultJiraDataImporter.doImport(DefaultJiraDataImporter.java:116)^M at com.atlassian.jira.imports.importer.impl.ImporterThread.run(ImporterThread.java:21)^M Caused by: org.ofbiz.core.entity.GenericDataSourceException: SQL Exception while executing the following:SELECT ID, issueid, AUTHOR, actiontype, actionlevel, actionbody, CREATED, actionnum FROM jiraaction WHERE issueid=? AND actiontype=? (I/O Error: Connection reset)^M at org.ofbiz.core.entity.jdbc.SQLProcessor.executeQuery(SQLProcessor.java:345)^M at org.ofbiz.core.entity.GenericDAO.selectListIteratorByCondition(GenericDAO.java:1050)^M at org.ofbiz.core.entity.GenericDAO.selectByAnd(GenericDAO.java:595)^M at org.ofbiz.core.entity.GenericHelperDAO.findByAnd(GenericHelperDAO.java:134)^M at org.ofbiz.core.entity.GenericDelegator.findByAnd(GenericDelegator.java:782)^M at org.ofbiz.core.entity.GenericDelegator.findByAnd(GenericDelegator.java:767)^M at org.ofbiz.core.entity.GenericDelegator.findByAnd(GenericDelegator.java:744)^M at com.atlassian.jira.imports.importer.impl.DefaultJiraDataImporter.rewriteOldIssueKeys(DefaultJiraDataImporter.java:631)^M ... 2 more^M (106913.47s)^M 2006-04-06 15:19:42,966 INFO [imports.importer.impl.ImportLogger] Flushing issue cache (106913.657s)^M 2006-04-06 15:21:28,576 WARN [core.entity.jdbc.SQLProcessor] Already closed.^M 2006-04-06 15:21:28,623 WARN [core.entity.jdbc.SQLProcessor] [SQLProcessor.rollback]: SQL Exception while rolling back insert. Error was:java.sql.SQLException: Invalid state, the Connection object is closed.^M 2006-04-06 15:21:28,623 WARN [NoModule] org.ofbiz.core.entity.jdbc.SQLProcessor^M java.sql.SQLException: Invalid state, the Connection object is closed.^M at net.sourceforge.jtds.jdbc.ConnectionJDBC2.checkOpen(ConnectionJDBC2.java:1494)^M at net.sourceforge.jtds.jdbc.ConnectionJDBC2.rollback(ConnectionJDBC2.java:1889)^M at org.apache.tomcat.dbcp.dbcp.DelegatingConnection.rollback(DelegatingConnection.java:265)^M at org.apache.tomcat.dbcp.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.rollback(PoolingDataSource.java:288)^M at org.ofbiz.core.entity.jdbc.SQLProcessor.rollback(SQLProcessor.java:131)^M at org.ofbiz.core.entity.jdbc.SQLProcessor.commit(SQLProcessor.java:114)^M at org.ofbiz.core.entity.jdbc.SQLProcessor.close(SQLProcessor.java:153)^M at org.ofbiz.core.entity.jdbc.SQLProcessor.finalize(SQLProcessor.java:702)^M at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)^M at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)^M at java.lang.ref.Finalizer.access$100(Finalizer.java:14)^M at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)^M 2006-04-06 15:21:28,623 WARN [NoModule] [SQLProcessor.commit]: SQL Exception occurred on commit. Error was:java.sql.SQLException: Invalid state, the Connection object is closed.^M 2006-04-06 15:21:28,638 ERROR [NoModule] Error closing the result, connection, etc in finalize EntityListIterator^M org.ofbiz.core.entity.GenericDataSourceException: SQL Exception occurred on commit (Invalid state, the Connection object is closed.)^M at org.ofbiz.core.entity.jdbc.SQLProcessor.commit(SQLProcessor.java:116)^M at org.ofbiz.core.entity.jdbc.SQLProcessor.close(SQLProcessor.java:153)^M at org.ofbiz.core.entity.jdbc.SQLProcessor.finalize(SQLProcessor.java:702)^M at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)^M at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)^M at java.lang.ref.Finalizer.access$100(Finalizer.java:14)^M at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)^M java.sql.SQLException: Invalid state, the Connection object is closed.^M at net.sourceforge.jtds.jdbc.ConnectionJDBC2.checkOpen(ConnectionJDBC2.java:1494)^M at net.sourceforge.jtds.jdbc.ConnectionJDBC2.commit(ConnectionJDBC2.java:1874)^M at org.apache.tomcat.dbcp.dbcp.DelegatingConnection.commit(DelegatingConnection.java:238)^M at org.apache.tomcat.dbcp.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.commit(PoolingDataSource.java:199)^M at org.ofbiz.core.entity.jdbc.SQLProcessor.commit(SQLProcessor.java:112)^M at org.ofbiz.core.entity.jdbc.SQLProcessor.close(SQLProcessor.java:153)^M at org.ofbiz.core.entity.jdbc.SQLProcessor.finalize(SQLProcessor.java:702)^M at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)^M at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)^M at java.lang.ref.Finalizer.access$100(Finalizer.java:14)^M at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)^M {noformat}

    Atlassian JIRA | 1 decade ago | Jeff Turner
    org.ofbiz.core.entity.GenericDataSourceException: SQL Exception occurred on commit (Invalid state, the Connection object is closed.)

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. org.ofbiz.core.entity.GenericDataSourceException

      SQL Exception occurred on commit (Connection org.hsqldb.jdbc.jdbcConnection@d1b458d is closed.)

      at org.ofbiz.core.entity.jdbc.SQLProcessor.commit()
    2. org.ofbiz.core
      SQLProcessor.close
      1. org.ofbiz.core.entity.jdbc.SQLProcessor.commit(SQLProcessor.java:211)
      2. org.ofbiz.core.entity.jdbc.SQLProcessor.smartCommit(SQLProcessor.java:362)
      3. org.ofbiz.core.entity.jdbc.SQLProcessor.close(SQLProcessor.java:264)
      3 frames