java.sql.BatchUpdateException: Batch entry 0 DELETE FROM resource_params_cache WHERE NOT EXISTS (SELECT * FROM catalog_resources cr WHERE cr.resource=resource_params_cache.resource) was aborted. Call getNextException to see the cause.

JIRA | Wyatt Alt | 2 years ago
  1. 0

    Some of the the slaves in PDB-1124 (e.g 2) seem to hit this every time they attempt to gc the master database. {code} 2015-01-04 05:23:24,952 INFO [c.p.p.command] [b9d19827-c06b-4396-ac07-0afb6707e89f] [replace catalog] 2015-01-04 05:23:32,427 INFO [c.p.p.c.services] Starting sweep of stale reports (threshold: 14 days) 2015-01-04 05:23:32,439 INFO [c.p.p.c.services] Finished sweep of stale reports (threshold: 14 days) 2015-01-04 05:23:32,439 INFO [c.p.p.c.services] Starting database garbage collection 2015-01-04 05:23:35,208 ERROR [c.p.p.c.services] Error during garbage collection java.sql.BatchUpdateException: Batch entry 0 DELETE FROM resource_params_cache WHERE NOT EXISTS (SELECT * FROM catalog_resources cr WHERE cr.resource=resource_params_cache.resource) was aborted. Call getNextException to see the cause. at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2746) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl$1.handleError(QueryExecutorImpl.java:457) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1887) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:405) ~[puppetdb.jar:na] at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2893) ~[puppetdb.jar:na] at com.jolbox.bonecp.StatementHandle.executeBatch(StatementHandle.java:469) ~[puppetdb.jar:na] at clojure.java.jdbc.internal$do_prepared_STAR_$fn__6350.invoke(internal.clj:356) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223) ~[na:na] at clojure.java.jdbc.internal$do_prepared_STAR_.doInvoke(internal.clj:356) ~[na:na] at clojure.lang.RestFn.invoke(RestFn.java:423) [puppetdb.jar:na] at clojure.java.jdbc$delete_rows.invoke(jdbc.clj:297) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$delete_unassociated_params_BANG_$fn__9696.invoke(storage.clj:740) ~[na:na] at com.puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na] at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.scf.storage$delete_unassociated_params_BANG_.invoke(storage.clj:739) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_$fn__9708$fn__9709.invoke(storage.clj:767) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_$fn__9708.invoke(storage.clj:766) ~[na:na] at com.puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na] at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_.invoke(storage.clj:765) ~[na:na] at com.puppetlabs.puppetdb.cli.services$garbage_collect_BANG_$fn__19190.invoke(services.clj:165) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__6761$fn__6762$fn__6763.invoke(jdbc.clj:290) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:204) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__6761$fn__6762.invoke(jdbc.clj:290) ~[na:na] at clojure.java.jdbc.internal$with_connection_STAR_.invoke(internal.clj:186) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__6761.invoke(jdbc.clj:287) ~[na:na] at com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740$fn__6741$fn__6742.invoke(jdbc.clj:259) ~[na:na] at com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740$fn__6741.invoke(jdbc.clj:258) ~[na:na] at com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740.invoke(jdbc.clj:250) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn.invoke(jdbc.clj:286) ~[na:na] at com.puppetlabs.puppetdb.cli.services$garbage_collect_BANG_.invoke(services.clj:164) ~[na:na] at com.puppetlabs.puppetdb.cli.services$perform_db_maintenance_BANG_.doInvoke(services.clj:177) [na:na] at clojure.lang.RestFn.applyTo(RestFn.java:139) [puppetdb.jar:na] at clojure.core$apply.invoke(core.clj:626) [puppetdb.jar:na] at com.puppetlabs.puppetdb.cli.services$start_puppetdb$fn__19276.invoke(services.clj:347) [na:na] at clojure.lang.AFn.run(AFn.java:22) [puppetdb.jar:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_71] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [na:1.7.0_71] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_71] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71] {code} That is from host 2's puppetdb.log-20150111, which is in the ticket. This is also present in the master puppetdb logs for PE-7392 on 2014-12-25, but that is the only time it occurs for them.

    JIRA | 2 years ago | Wyatt Alt
    java.sql.BatchUpdateException: Batch entry 0 DELETE FROM resource_params_cache WHERE NOT EXISTS (SELECT * FROM catalog_resources cr WHERE cr.resource=resource_params_cache.resource) was aborted. Call getNextException to see the cause.
  2. 0

    Some of the the slaves in PDB-1124 (e.g 2) seem to hit this every time they attempt to gc the master database. {code} 2015-01-04 05:23:24,952 INFO [c.p.p.command] [b9d19827-c06b-4396-ac07-0afb6707e89f] [replace catalog] 2015-01-04 05:23:32,427 INFO [c.p.p.c.services] Starting sweep of stale reports (threshold: 14 days) 2015-01-04 05:23:32,439 INFO [c.p.p.c.services] Finished sweep of stale reports (threshold: 14 days) 2015-01-04 05:23:32,439 INFO [c.p.p.c.services] Starting database garbage collection 2015-01-04 05:23:35,208 ERROR [c.p.p.c.services] Error during garbage collection java.sql.BatchUpdateException: Batch entry 0 DELETE FROM resource_params_cache WHERE NOT EXISTS (SELECT * FROM catalog_resources cr WHERE cr.resource=resource_params_cache.resource) was aborted. Call getNextException to see the cause. at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2746) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl$1.handleError(QueryExecutorImpl.java:457) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1887) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:405) ~[puppetdb.jar:na] at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2893) ~[puppetdb.jar:na] at com.jolbox.bonecp.StatementHandle.executeBatch(StatementHandle.java:469) ~[puppetdb.jar:na] at clojure.java.jdbc.internal$do_prepared_STAR_$fn__6350.invoke(internal.clj:356) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223) ~[na:na] at clojure.java.jdbc.internal$do_prepared_STAR_.doInvoke(internal.clj:356) ~[na:na] at clojure.lang.RestFn.invoke(RestFn.java:423) [puppetdb.jar:na] at clojure.java.jdbc$delete_rows.invoke(jdbc.clj:297) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$delete_unassociated_params_BANG_$fn__9696.invoke(storage.clj:740) ~[na:na] at com.puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na] at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.scf.storage$delete_unassociated_params_BANG_.invoke(storage.clj:739) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_$fn__9708$fn__9709.invoke(storage.clj:767) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_$fn__9708.invoke(storage.clj:766) ~[na:na] at com.puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na] at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_.invoke(storage.clj:765) ~[na:na] at com.puppetlabs.puppetdb.cli.services$garbage_collect_BANG_$fn__19190.invoke(services.clj:165) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__6761$fn__6762$fn__6763.invoke(jdbc.clj:290) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:204) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__6761$fn__6762.invoke(jdbc.clj:290) ~[na:na] at clojure.java.jdbc.internal$with_connection_STAR_.invoke(internal.clj:186) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__6761.invoke(jdbc.clj:287) ~[na:na] at com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740$fn__6741$fn__6742.invoke(jdbc.clj:259) ~[na:na] at com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740$fn__6741.invoke(jdbc.clj:258) ~[na:na] at com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740.invoke(jdbc.clj:250) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn.invoke(jdbc.clj:286) ~[na:na] at com.puppetlabs.puppetdb.cli.services$garbage_collect_BANG_.invoke(services.clj:164) ~[na:na] at com.puppetlabs.puppetdb.cli.services$perform_db_maintenance_BANG_.doInvoke(services.clj:177) [na:na] at clojure.lang.RestFn.applyTo(RestFn.java:139) [puppetdb.jar:na] at clojure.core$apply.invoke(core.clj:626) [puppetdb.jar:na] at com.puppetlabs.puppetdb.cli.services$start_puppetdb$fn__19276.invoke(services.clj:347) [na:na] at clojure.lang.AFn.run(AFn.java:22) [puppetdb.jar:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_71] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [na:1.7.0_71] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_71] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71] {code} That is from host 2's puppetdb.log-20150111, which is in the ticket. This is also present in the master puppetdb logs for PE-7392 on 2014-12-25, but that is the only time it occurs for them.

    JIRA | 2 years ago | Wyatt Alt
    java.sql.BatchUpdateException: Batch entry 0 DELETE FROM resource_params_cache WHERE NOT EXISTS (SELECT * FROM catalog_resources cr WHERE cr.resource=resource_params_cache.resource) was aborted. Call getNextException to see the cause.
  3. 0

    Hi, We're seeing an issue with our puppetdb since it was upgraded to 2.3.1, with some nodes not updating. We get errors like the following the the puppetdb.log {noformat} 015-04-23 14:45:10,223 ERROR [c.p.p.command] [b7ba3fa5-c61b-4461-a2b5-4317d00a92c8] [replace facts] Retrying after attempt 4, due to: java.sql.BatchUpdateException: Batch entry 0 DELETE FROM fact_values fv WHERE fv.id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND NOT EXISTS (SELECT 1 FROM facts f WHERE f.fact_value_id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND f.fact_value_id = fv.id AND (f.factset_id, f.fact_path_id) NOT in ((36,820),(36,44),(36,279),(36,67),(36,924),(36,452),(36,812),(36,105),(36,616),(36,139),(36,1019),(36,487),(36,231),(36,65),(36,15),(36,48),(36,88),(36,655),(36,118),(36,87),(36,230),(36,647),(36,493),(36,991),(36,857),(36,81),(36,2),(36,1),(36,79),(36,249),(36,911),(36,349),(36,63),(36,548),(36,141),(36,324),(36,1006),(36,40),(36,27),(36,612),(36,364),(36,530),(36,832),(36,339),(36,25),(36,526),(36,92),(36,797),(36,85),(36,642),(36,131),(36,86),(36,383),(36,58),(36,103),(36,13),(36,871),(36,68),(36,136),(36,536),(36,127),(36,624),(36,7),(36,128),(36,543),(36,519),(36,214),(36,4),(36,46),(36,482),(36,897),(36,49),(36,809),(36,704),(36,811))) was aborted. Call getNextException to see the cause. java.sql.BatchUpdateException: Batch entry 0 DELETE FROM fact_values fv WHERE fv.id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND NOT EXISTS (SELECT 1 FROM facts f WHERE f.fact_value_id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND f.fact_value_id = fv.id AND (f.factset_id, f.fact_path_id) NOT in ((36,820),(36,44),(36,279),(36,67),(36,924),(36,452),(36,812),(36,105),(36,616),(36,139),(36,1019),(36,487),(36,231),(36,65),(36,15),(36,48),(36,88),(36,655),(36,118),(36,87),(36,230),(36,647),(36,493),(36,991),(36,857),(36,81),(36,2),(36,1),(36,79),(36,249),(36,911),(36,349),(36,63),(36,548),(36,141),(36,324),(36,1006),(36,40),(36,27),(36,612),(36,364),(36,530),(36,832),(36,339),(36,25),(36,526),(36,92),(36,797),(36,85),(36,642),(36,131),(36,86),(36,383),(36,58),(36,103),(36,13),(36,871),(36,68),(36,136),(36,536),(36,127),(36,624),(36,7),(36,128),(36,543),(36,519),(36,214),(36,4),(36,46),(36,482),(36,897),(36,49),(36,809),(36,704),(36,811))) was aborted. Call getNextException to see the cause. at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2746) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1887) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:405) ~[puppetdb.jar:na] at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2893) ~[puppetdb.jar:na] at com.jolbox.bonecp.StatementHandle.executeBatch(StatementHandle.java:469) ~[puppetdb.jar:na] at clojure.java.jdbc.internal$do_prepared_STAR_$fn__7259.invoke(internal.clj:356) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223) ~[na:na] at clojure.java.jdbc.internal$do_prepared_STAR_.doInvoke(internal.clj:356) ~[na:na] at clojure.lang.RestFn.applyTo(RestFn.java:139) ~[puppetdb.jar:na] at clojure.core$apply.invoke(core.clj:626) ~[puppetdb.jar:na] at clojure.java.jdbc$do_prepared.doInvoke(jdbc.clj:206) ~[na:na] at clojure.lang.RestFn.invoke(RestFn.java:423) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.scf.storage$eval10818$delete_pending_value_id_orphans_BANG___10819$fn__10820.invoke(storage.clj:797) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval10818$delete_pending_value_id_orphans_BANG___10819.invoke(storage.clj:777) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval10995$update_facts_BANG___10996$fn__11000$fn__11002.invoke(storage.clj:993) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval10995$update_facts_BANG___10996$fn__11000.invoke(storage.clj:956) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval10995$update_facts_BANG___10996.invoke(storage.clj:949) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval11197$replace_facts_BANG___11198$fn__11202$fn__11204.invoke(storage.clj:1212) ~[na:na] at com.puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na] at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.scf.storage$eval11197$replace_facts_BANG___11198$fn__11202.invoke(storage.clj:1209) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval11197$replace_facts_BANG___11198.invoke(storage.clj:1202) ~[na:na] at com.puppetlabs.puppetdb.command$eval13392$fn__13395$fn__13408.invoke(command.clj:381) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__7670$fn__7671$fn__7672.invoke(jdbc.clj:290) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:204) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__7670$fn__7671.invoke(jdbc.clj:290) ~[na:na] at clojure.java.jdbc.internal$with_connection_STAR_.invoke(internal.clj:186) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__7670.invoke(jdbc.clj:287) ~[na:na] at com.puppetlabs.jdbc$eval7648$retry_sql_STAR___7649$fn__7650$fn__7651.invoke(jdbc.clj:259) ~[na:na] at com.puppetlabs.jdbc$eval7648$retry_sql_STAR___7649$fn__7650.invoke(jdbc.clj:258) ~[na:na] at com.puppetlabs.jdbc$eval7648$retry_sql_STAR___7649.invoke(jdbc.clj:250) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn.invoke(jdbc.clj:286) ~[na:na] at com.puppetlabs.puppetdb.command$eval13392$fn__13395.invoke(command.clj:379) ~[na:na] at clojure.lang.MultiFn.invoke(MultiFn.java:231) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.command$produce_message_handler$fn__13564.invoke(command.clj:647) ~[na:na] at com.puppetlabs.puppetdb.command$wrap_with_discard$fn__13513$fn__13517.invoke(command.clj:554) ~[na:na] at com.puppetlabs.puppetdb.command.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na] at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.command$wrap_with_discard$fn__13513.invoke(command.clj:553) ~[na:na] at com.puppetlabs.puppetdb.command$wrap_with_exception_handling$fn__13498$fn__13499.invoke(command.clj:507) ~[na:na] at com.puppetlabs.puppetdb.command.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na] at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.command$wrap_with_exception_handling$fn__13498.invoke(command.clj:506) ~[na:na] at com.puppetlabs.puppetdb.command$wrap_with_command_parser$fn__13508.invoke(command.clj:529) [na:na] at com.puppetlabs.puppetdb.command$wrap_with_meter$fn__13488.invoke(command.clj:467) [na:na] at com.puppetlabs.puppetdb.command$wrap_with_thread_name$fn__13522.invoke(command.clj:569) [na:na] at com.puppetlabs.mq$create_message_listener$reify__11783.onMessage(mq.clj:270) [na:na] at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:560) [puppetdb.jar:na] at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:498) [puppetdb.jar:na] at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467) [puppetdb.jar:na] at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:325) [puppetdb.jar:na] at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263) [puppetdb.jar:na] at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058) [puppetdb.jar:na] at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1050) [puppetdb.jar:na] at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:947) [puppetdb.jar:na] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75] {noformat} and the corresponding error in the postgres.log: {noformat} RROR: update or delete on table "fact_values" violates foreign key constraint "fact_value_id_fk" on table "facts" DETAIL: Key (id)=(1524348) is still referenced from table "facts". STATEMENT: DELETE FROM fact_values fv WHERE fv.id in ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50,$51,$52,$53,$54,$55,$56,$57,$58,$59,$60,$61,$62,$63,$64,$65,$66,$67,$68,$69,$70,$71,$72,$73,$74,$75) AND NOT EXISTS (SELECT 1 FROM facts f WHERE f.fact_value_id in ($76,$77,$78,$79,$80,$81,$82,$83,$84,$85,$86,$87,$88,$89,$90,$91,$92,$93,$94,$95,$96,$97,$98,$99,$100,$101,$102,$103,$104,$105,$106,$107,$108,$109,$110,$111,$112,$113,$114,$115,$116,$117,$118,$119,$120,$121,$122,$123,$124,$125,$126,$127,$128,$129,$130,$131,$132,$133,$134,$135,$136,$137,$138,$139,$140,$141,$142,$143,$144,$145,$146,$147,$148,$149,$150) AND f.fact_value_id = fv.id AND (f.factset_id, f.fact_path_id) NOT in (($151,$152),($153,$154),($155,$156),($157,$158),($159,$160),($161,$162),($163,$164),($165,$166),($167,$168),($169,$170),($171,$172),($173,$174),($175,$176),($177,$178),($179,$180),($181,$182),($183,$184),($185,$186),($187,$188),($189,$190),($191,$192),($193,$194),($195,$196),($197,$198),($199,$200),($201,$202),($203,$204),($205,$206),($207,$208),($209,$210),($211,$212),($213,$214),($215,$216),($217,$218),($219,$220),($221,$222),($223,$224),($225,$226),($227,$228),($229,$230),($231,$232),($233,$234),($235,$236),($237,$238),($239,$240),($241,$242),($243,$244),($245,$246),($247,$248),($249,$250),($251,$252),($253,$254),($255,$256),($257,$258),($259,$260),($261,$262),($263,$264),($265,$266),($267,$268),($269,$270),($271,$272),($273,$274),($275,$276),($277,$278),($279,$280),($281,$282),($283,$284),($285,$286),($287,$288),($289,$290),($291,$292),($293,$294),($295,$296),($297,$298),($299,$300))) {noformat} I've been able to get things moving along by performing {noformat} delete from facts where fact_value_id in ( <list of numbers from failing fact_values delete>) {noformat} and I've updated to 2.3.3, but new 'factsets' keep developing the same error. We're on CentOS 7 and postgres 9.2.7. We have about 110 puppet agents.

    JIRA | 2 years ago | Simon Oxwell
    java.sql.BatchUpdateException: Batch entry 0 DELETE FROM fact_values fv WHERE fv.id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND NOT EXISTS (SELECT 1 FROM facts f WHERE f.fact_value_id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND f.fact_value_id = fv.id AND (f.factset_id, f.fact_path_id) NOT in ((36,820),(36,44),(36,279),(36,67),(36,924),(36,452),(36,812),(36,105),(36,616),(36,139),(36,1019),(36,487),(36,231),(36,65),(36,15),(36,48),(36,88),(36,655),(36,118),(36,87),(36,230),(36,647),(36,493),(36,991),(36,857),(36,81),(36,2),(36,1),(36,79),(36,249),(36,911),(36,349),(36,63),(36,548),(36,141),(36,324),(36,1006),(36,40),(36,27),(36,612),(36,364),(36,530),(36,832),(36,339),(36,25),(36,526),(36,92),(36,797),(36,85),(36,642),(36,131),(36,86),(36,383),(36,58),(36,103),(36,13),(36,871),(36,68),(36,136),(36,536),(36,127),(36,624),(36,7),(36,128),(36,543),(36,519),(36,214),(36,4),(36,46),(36,482),(36,897),(36,49),(36,809),(36,704),(36,811))) was aborted. Call getNextException to see the cause.
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Hi, We're seeing an issue with our puppetdb since it was upgraded to 2.3.1, with some nodes not updating. We get errors like the following the the puppetdb.log {noformat} 015-04-23 14:45:10,223 ERROR [c.p.p.command] [b7ba3fa5-c61b-4461-a2b5-4317d00a92c8] [replace facts] Retrying after attempt 4, due to: java.sql.BatchUpdateException: Batch entry 0 DELETE FROM fact_values fv WHERE fv.id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND NOT EXISTS (SELECT 1 FROM facts f WHERE f.fact_value_id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND f.fact_value_id = fv.id AND (f.factset_id, f.fact_path_id) NOT in ((36,820),(36,44),(36,279),(36,67),(36,924),(36,452),(36,812),(36,105),(36,616),(36,139),(36,1019),(36,487),(36,231),(36,65),(36,15),(36,48),(36,88),(36,655),(36,118),(36,87),(36,230),(36,647),(36,493),(36,991),(36,857),(36,81),(36,2),(36,1),(36,79),(36,249),(36,911),(36,349),(36,63),(36,548),(36,141),(36,324),(36,1006),(36,40),(36,27),(36,612),(36,364),(36,530),(36,832),(36,339),(36,25),(36,526),(36,92),(36,797),(36,85),(36,642),(36,131),(36,86),(36,383),(36,58),(36,103),(36,13),(36,871),(36,68),(36,136),(36,536),(36,127),(36,624),(36,7),(36,128),(36,543),(36,519),(36,214),(36,4),(36,46),(36,482),(36,897),(36,49),(36,809),(36,704),(36,811))) was aborted. Call getNextException to see the cause. java.sql.BatchUpdateException: Batch entry 0 DELETE FROM fact_values fv WHERE fv.id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND NOT EXISTS (SELECT 1 FROM facts f WHERE f.fact_value_id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND f.fact_value_id = fv.id AND (f.factset_id, f.fact_path_id) NOT in ((36,820),(36,44),(36,279),(36,67),(36,924),(36,452),(36,812),(36,105),(36,616),(36,139),(36,1019),(36,487),(36,231),(36,65),(36,15),(36,48),(36,88),(36,655),(36,118),(36,87),(36,230),(36,647),(36,493),(36,991),(36,857),(36,81),(36,2),(36,1),(36,79),(36,249),(36,911),(36,349),(36,63),(36,548),(36,141),(36,324),(36,1006),(36,40),(36,27),(36,612),(36,364),(36,530),(36,832),(36,339),(36,25),(36,526),(36,92),(36,797),(36,85),(36,642),(36,131),(36,86),(36,383),(36,58),(36,103),(36,13),(36,871),(36,68),(36,136),(36,536),(36,127),(36,624),(36,7),(36,128),(36,543),(36,519),(36,214),(36,4),(36,46),(36,482),(36,897),(36,49),(36,809),(36,704),(36,811))) was aborted. Call getNextException to see the cause. at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2746) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1887) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:405) ~[puppetdb.jar:na] at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2893) ~[puppetdb.jar:na] at com.jolbox.bonecp.StatementHandle.executeBatch(StatementHandle.java:469) ~[puppetdb.jar:na] at clojure.java.jdbc.internal$do_prepared_STAR_$fn__7259.invoke(internal.clj:356) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223) ~[na:na] at clojure.java.jdbc.internal$do_prepared_STAR_.doInvoke(internal.clj:356) ~[na:na] at clojure.lang.RestFn.applyTo(RestFn.java:139) ~[puppetdb.jar:na] at clojure.core$apply.invoke(core.clj:626) ~[puppetdb.jar:na] at clojure.java.jdbc$do_prepared.doInvoke(jdbc.clj:206) ~[na:na] at clojure.lang.RestFn.invoke(RestFn.java:423) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.scf.storage$eval10818$delete_pending_value_id_orphans_BANG___10819$fn__10820.invoke(storage.clj:797) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval10818$delete_pending_value_id_orphans_BANG___10819.invoke(storage.clj:777) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval10995$update_facts_BANG___10996$fn__11000$fn__11002.invoke(storage.clj:993) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval10995$update_facts_BANG___10996$fn__11000.invoke(storage.clj:956) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval10995$update_facts_BANG___10996.invoke(storage.clj:949) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval11197$replace_facts_BANG___11198$fn__11202$fn__11204.invoke(storage.clj:1212) ~[na:na] at com.puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na] at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.scf.storage$eval11197$replace_facts_BANG___11198$fn__11202.invoke(storage.clj:1209) ~[na:na] at com.puppetlabs.puppetdb.scf.storage$eval11197$replace_facts_BANG___11198.invoke(storage.clj:1202) ~[na:na] at com.puppetlabs.puppetdb.command$eval13392$fn__13395$fn__13408.invoke(command.clj:381) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__7670$fn__7671$fn__7672.invoke(jdbc.clj:290) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:204) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__7670$fn__7671.invoke(jdbc.clj:290) ~[na:na] at clojure.java.jdbc.internal$with_connection_STAR_.invoke(internal.clj:186) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn$fn__7670.invoke(jdbc.clj:287) ~[na:na] at com.puppetlabs.jdbc$eval7648$retry_sql_STAR___7649$fn__7650$fn__7651.invoke(jdbc.clj:259) ~[na:na] at com.puppetlabs.jdbc$eval7648$retry_sql_STAR___7649$fn__7650.invoke(jdbc.clj:258) ~[na:na] at com.puppetlabs.jdbc$eval7648$retry_sql_STAR___7649.invoke(jdbc.clj:250) ~[na:na] at com.puppetlabs.jdbc$with_transacted_connection_fn.invoke(jdbc.clj:286) ~[na:na] at com.puppetlabs.puppetdb.command$eval13392$fn__13395.invoke(command.clj:379) ~[na:na] at clojure.lang.MultiFn.invoke(MultiFn.java:231) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.command$produce_message_handler$fn__13564.invoke(command.clj:647) ~[na:na] at com.puppetlabs.puppetdb.command$wrap_with_discard$fn__13513$fn__13517.invoke(command.clj:554) ~[na:na] at com.puppetlabs.puppetdb.command.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na] at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.command$wrap_with_discard$fn__13513.invoke(command.clj:553) ~[na:na] at com.puppetlabs.puppetdb.command$wrap_with_exception_handling$fn__13498$fn__13499.invoke(command.clj:507) ~[na:na] at com.puppetlabs.puppetdb.command.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na] at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] at com.puppetlabs.puppetdb.command$wrap_with_exception_handling$fn__13498.invoke(command.clj:506) ~[na:na] at com.puppetlabs.puppetdb.command$wrap_with_command_parser$fn__13508.invoke(command.clj:529) [na:na] at com.puppetlabs.puppetdb.command$wrap_with_meter$fn__13488.invoke(command.clj:467) [na:na] at com.puppetlabs.puppetdb.command$wrap_with_thread_name$fn__13522.invoke(command.clj:569) [na:na] at com.puppetlabs.mq$create_message_listener$reify__11783.onMessage(mq.clj:270) [na:na] at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:560) [puppetdb.jar:na] at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:498) [puppetdb.jar:na] at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467) [puppetdb.jar:na] at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:325) [puppetdb.jar:na] at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263) [puppetdb.jar:na] at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058) [puppetdb.jar:na] at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1050) [puppetdb.jar:na] at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:947) [puppetdb.jar:na] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75] {noformat} and the corresponding error in the postgres.log: {noformat} RROR: update or delete on table "fact_values" violates foreign key constraint "fact_value_id_fk" on table "facts" DETAIL: Key (id)=(1524348) is still referenced from table "facts". STATEMENT: DELETE FROM fact_values fv WHERE fv.id in ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50,$51,$52,$53,$54,$55,$56,$57,$58,$59,$60,$61,$62,$63,$64,$65,$66,$67,$68,$69,$70,$71,$72,$73,$74,$75) AND NOT EXISTS (SELECT 1 FROM facts f WHERE f.fact_value_id in ($76,$77,$78,$79,$80,$81,$82,$83,$84,$85,$86,$87,$88,$89,$90,$91,$92,$93,$94,$95,$96,$97,$98,$99,$100,$101,$102,$103,$104,$105,$106,$107,$108,$109,$110,$111,$112,$113,$114,$115,$116,$117,$118,$119,$120,$121,$122,$123,$124,$125,$126,$127,$128,$129,$130,$131,$132,$133,$134,$135,$136,$137,$138,$139,$140,$141,$142,$143,$144,$145,$146,$147,$148,$149,$150) AND f.fact_value_id = fv.id AND (f.factset_id, f.fact_path_id) NOT in (($151,$152),($153,$154),($155,$156),($157,$158),($159,$160),($161,$162),($163,$164),($165,$166),($167,$168),($169,$170),($171,$172),($173,$174),($175,$176),($177,$178),($179,$180),($181,$182),($183,$184),($185,$186),($187,$188),($189,$190),($191,$192),($193,$194),($195,$196),($197,$198),($199,$200),($201,$202),($203,$204),($205,$206),($207,$208),($209,$210),($211,$212),($213,$214),($215,$216),($217,$218),($219,$220),($221,$222),($223,$224),($225,$226),($227,$228),($229,$230),($231,$232),($233,$234),($235,$236),($237,$238),($239,$240),($241,$242),($243,$244),($245,$246),($247,$248),($249,$250),($251,$252),($253,$254),($255,$256),($257,$258),($259,$260),($261,$262),($263,$264),($265,$266),($267,$268),($269,$270),($271,$272),($273,$274),($275,$276),($277,$278),($279,$280),($281,$282),($283,$284),($285,$286),($287,$288),($289,$290),($291,$292),($293,$294),($295,$296),($297,$298),($299,$300))) {noformat} I've been able to get things moving along by performing {noformat} delete from facts where fact_value_id in ( <list of numbers from failing fact_values delete>) {noformat} and I've updated to 2.3.3, but new 'factsets' keep developing the same error. We're on CentOS 7 and postgres 9.2.7. We have about 110 puppet agents.

    JIRA | 2 years ago | Simon Oxwell
    java.sql.BatchUpdateException: Batch entry 0 DELETE FROM fact_values fv WHERE fv.id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND NOT EXISTS (SELECT 1 FROM facts f WHERE f.fact_value_id in (90,180,2414,2667,2590,1264,2436,1561878,2568,1524348,90,2516,2436,2468,1561273,1561275,33,1210,1236,2678,90,33,2320,2478,2494,2452,90,2678,1551663,33,26,2660,1561269,896,33,578,33,1561277,1551664,90,4641,180,2590,90,83,578,1551663,83,2452,2451,1561876,1561877,180,1561272,1561876,2474,90,1551664,180,2516,1561277,2462,1561271,33,2568,26,2494,2667,1524348,2414,180,33,33,90,4616) AND f.fact_value_id = fv.id AND (f.factset_id, f.fact_path_id) NOT in ((36,820),(36,44),(36,279),(36,67),(36,924),(36,452),(36,812),(36,105),(36,616),(36,139),(36,1019),(36,487),(36,231),(36,65),(36,15),(36,48),(36,88),(36,655),(36,118),(36,87),(36,230),(36,647),(36,493),(36,991),(36,857),(36,81),(36,2),(36,1),(36,79),(36,249),(36,911),(36,349),(36,63),(36,548),(36,141),(36,324),(36,1006),(36,40),(36,27),(36,612),(36,364),(36,530),(36,832),(36,339),(36,25),(36,526),(36,92),(36,797),(36,85),(36,642),(36,131),(36,86),(36,383),(36,58),(36,103),(36,13),(36,871),(36,68),(36,136),(36,536),(36,127),(36,624),(36,7),(36,128),(36,543),(36,519),(36,214),(36,4),(36,46),(36,482),(36,897),(36,49),(36,809),(36,704),(36,811))) was aborted. Call getNextException to see the cause.
  6. 0

    Hi Gang, I just upgraded to PuppetDB 2.3.1 from 2.3.0 in my development environment and PuppetDB crashed on restart. If you need more information let me know. The resulting log messages. 2015-04-01 08:44:51,294 INFO [o.e.j.u.log] Logging initialized @24441ms 2015-04-01 08:44:52,110 INFO [p.t.s.w.jetty9-core] Removing buggy security provider SunPKCS11-NSS version 1.7 2015-04-01 08:44:52,636 INFO [p.t.s.w.jetty9-service] Initializing web server(s). 2015-04-01 08:44:52,642 INFO [p.t.s.w.jetty9-service] Starting web server(s). 2015-04-01 08:44:52,818 INFO [p.t.s.w.jetty9-core] Starting web server. 2015-04-01 08:44:52,822 INFO [o.e.j.s.Server] jetty-9.2.z-SNAPSHOT 2015-04-01 08:44:52,874 INFO [o.e.j.s.ServerConnector] Started ServerConnector@66ace155{HTTP/1.1}{localhost:8080} 2015-04-01 08:44:53,150 INFO [o.e.j.s.ServerConnector] Started ServerConnector@7f17e429{SSL-HTTP/1.1}{0.0.0.0:8081} 2015-04-01 08:44:53,151 INFO [o.e.j.s.Server] Started @26300ms 2015-04-01 08:44:53,228 INFO [c.p.p.c.services] PuppetDB version 2.3.1 2015-04-01 08:44:53,362 INFO [c.p.p.s.migrate] Applying database migration version 28 2015-04-01 08:44:53,396 ERROR [c.p.p.s.migrate] Caught SQLException during migration java.sql.BatchUpdateException: Batch entry 5 DELETE FROM fact_paths t1 WHERE t1.id <> (SELECT MIN(t2.id) FROM fact_paths t2 WHERE t1.path = t2.path) was aborted. Call getNextException to see the cause. at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2746) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl$1.handleError(QueryExecutorImpl.java:457) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1887) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:405) ~[puppetdb.jar:na] at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2893) ~[puppetdb.jar:na] at com.jolbox.bonecp.StatementHandle.executeBatch(StatementHandle.java:469) ~[puppetdb.jar:na] at clojure.java.jdbc$do_commands$fn__7301.invoke(jdbc.clj:188) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223) [na:na] at clojure.java.jdbc$do_commands.doInvoke(jdbc.clj:187) ~[na:na] at clojure.lang.RestFn.invoke(RestFn.java:3894) [puppetdb.jar:na] at com.puppetlabs.puppetdb.scf.migrate$lift_fact_paths_into_facts.invoke(migrate.clj:968) ~[na:na] at com.puppetlabs.puppetdb.scf.migrate$migrate_BANG_$fn__20902$fn__20915.invoke(migrate.clj:1063) ~[na:na] at com.puppetlabs.puppetdb.scf.migrate$migrate_BANG_$fn__20902.invoke(migrate.clj:1062) [na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:204) [na:na] at com.puppetlabs.puppetdb.scf.migrate$migrate_BANG_.invoke(migrate.clj:1059) [na:na] at com.puppetlabs.puppetdb.cli.services$start_puppetdb$fn__21109.invoke(services.clj:292) [na:na] at clojure.java.jdbc.internal$with_connection_STAR_.invoke(internal.clj:186) [na:na] at com.puppetlabs.puppetdb.cli.services$start_puppetdb.invoke(services.clj:290) [na:na] at com.puppetlabs.puppetdb.cli.services$reify__21157$service_fnk__18232__auto___positional$reify__21168.start(services.clj:366) [na:na] at puppetlabs.trapperkeeper.services$eval18068$fn__18082$G__18058__18085.invoke(services.clj:10) [na:na] at puppetlabs.trapperkeeper.services$eval18068$fn__18082$G__18057__18089.invoke(services.clj:10) [na:na] at puppetlabs.trapperkeeper.internal$run_lifecycle_fn_BANG_.invoke(internal.clj:154) [na:na] at puppetlabs.trapperkeeper.internal$run_lifecycle_fns.invoke(internal.clj:182) [na:na] at puppetlabs.trapperkeeper.internal$build_app_STAR_$reify__18905.start(internal.clj:449) [na:na] at puppetlabs.trapperkeeper.internal$boot_services_STAR_$fn__18917.invoke(internal.clj:473) [na:na] at puppetlabs.trapperkeeper.internal$boot_services_STAR_.invoke(internal.clj:471) [na:na] at puppetlabs.trapperkeeper.core$boot_with_cli_data.invoke(core.clj:113) [na:na] at puppetlabs.trapperkeeper.core$run.invoke(core.clj:144) [na:na] at puppetlabs.trapperkeeper.core$main.doInvoke(core.clj:159) [na:na] at clojure.lang.RestFn.applyTo(RestFn.java:137) [puppetdb.jar:na] at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na] at com.puppetlabs.puppetdb.cli.services$_main.doInvoke(services.clj:373) [na:na] at clojure.lang.RestFn.invoke(RestFn.java:421) [puppetdb.jar:na] at clojure.lang.Var.invoke(Var.java:383) [puppetdb.jar:na] at clojure.lang.AFn.applyToHelper(AFn.java:156) [puppetdb.jar:na] at clojure.lang.Var.applyTo(Var.java:700) [puppetdb.jar:na] at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na] at com.puppetlabs.puppetdb.core$run_command.invoke(core.clj:87) [na:na] at com.puppetlabs.puppetdb.core$_main.doInvoke(core.clj:95) [na:na] at clojure.lang.RestFn.invoke(RestFn.java:436) [puppetdb.jar:na] at clojure.lang.Var.invoke(Var.java:388) [puppetdb.jar:na] at clojure.lang.AFn.applyToHelper(AFn.java:160) [puppetdb.jar:na] at clojure.lang.Var.applyTo(Var.java:700) [puppetdb.jar:na] at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na] at clojure.main$main_opt.invoke(main.clj:315) [puppetdb.jar:na] at clojure.main$main.doInvoke(main.clj:420) [puppetdb.jar:na] at clojure.lang.RestFn.invoke(RestFn.java:482) [puppetdb.jar:na] at clojure.lang.Var.invoke(Var.java:401) [puppetdb.jar:na] at clojure.lang.AFn.applyToHelper(AFn.java:171) [puppetdb.jar:na] at clojure.lang.Var.applyTo(Var.java:700) [puppetdb.jar:na] at clojure.main.main(main.java:37) [puppetdb.jar:na] 2015-04-01 08:44:53,398 ERROR [c.p.p.s.migrate] Unravelled exception org.postgresql.util.PSQLException: ERROR: update or delete on table "fact_paths" violates foreign key constraint "fact_values_path_id_fk" on table "fact_values" Detail: Key (id)=(452) is still referenced from table "fact_values". at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2157) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1886) ~[puppetdb.jar:na] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:405) ~[puppetdb.jar:na] at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2893) ~[puppetdb.jar:na] at com.jolbox.bonecp.StatementHandle.executeBatch(StatementHandle.java:469) ~[puppetdb.jar:na] at clojure.java.jdbc$do_commands$fn__7301.invoke(jdbc.clj:188) ~[na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223) [na:na] at clojure.java.jdbc$do_commands.doInvoke(jdbc.clj:187) ~[na:na] at clojure.lang.RestFn.invoke(RestFn.java:3894) [puppetdb.jar:na] at com.puppetlabs.puppetdb.scf.migrate$lift_fact_paths_into_facts.invoke(migrate.clj:968) ~[na:na] at com.puppetlabs.puppetdb.scf.migrate$migrate_BANG_$fn__20902$fn__20915.invoke(migrate.clj:1063) ~[na:na] at com.puppetlabs.puppetdb.scf.migrate$migrate_BANG_$fn__20902.invoke(migrate.clj:1062) [na:na] at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:204) [na:na] at com.puppetlabs.puppetdb.scf.migrate$migrate_BANG_.invoke(migrate.clj:1059) [na:na] at com.puppetlabs.puppetdb.cli.services$start_puppetdb$fn__21109.invoke(services.clj:292) [na:na] at clojure.java.jdbc.internal$with_connection_STAR_.invoke(internal.clj:186) [na:na] at com.puppetlabs.puppetdb.cli.services$start_puppetdb.invoke(services.clj:290) [na:na] at com.puppetlabs.puppetdb.cli.services$reify__21157$service_fnk__18232__auto___positional$reify__21168.start(services.clj:366) [na:na] at puppetlabs.trapperkeeper.services$eval18068$fn__18082$G__18058__18085.invoke(services.clj:10) [na:na] at puppetlabs.trapperkeeper.services$eval18068$fn__18082$G__18057__18089.invoke(services.clj:10) [na:na] at puppetlabs.trapperkeeper.internal$run_lifecycle_fn_BANG_.invoke(internal.clj:154) [na:na] at puppetlabs.trapperkeeper.internal$run_lifecycle_fns.invoke(internal.clj:182) [na:na] at puppetlabs.trapperkeeper.internal$build_app_STAR_$reify__18905.start(internal.clj:449) [na:na] at puppetlabs.trapperkeeper.internal$boot_services_STAR_$fn__18917.invoke(internal.clj:473) [na:na] at puppetlabs.trapperkeeper.internal$boot_services_STAR_.invoke(internal.clj:471) [na:na] at puppetlabs.trapperkeeper.core$boot_with_cli_data.invoke(core.clj:113) [na:na] at puppetlabs.trapperkeeper.core$run.invoke(core.clj:144) [na:na] at puppetlabs.trapperkeeper.core$main.doInvoke(core.clj:159) [na:na] at clojure.lang.RestFn.applyTo(RestFn.java:137) [puppetdb.jar:na] at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na] at com.puppetlabs.puppetdb.cli.services$_main.doInvoke(services.clj:373) [na:na] at clojure.lang.RestFn.invoke(RestFn.java:421) [puppetdb.jar:na] at clojure.lang.Var.invoke(Var.java:383) [puppetdb.jar:na] at clojure.lang.AFn.applyToHelper(AFn.java:156) [puppetdb.jar:na] at clojure.lang.Var.applyTo(Var.java:700) [puppetdb.jar:na] at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na] at com.puppetlabs.puppetdb.core$run_command.invoke(core.clj:87) [na:na] at com.puppetlabs.puppetdb.core$_main.doInvoke(core.clj:95) [na:na] at clojure.lang.RestFn.invoke(RestFn.java:436) [puppetdb.jar:na] at clojure.lang.Var.invoke(Var.java:388) [puppetdb.jar:na] at clojure.lang.AFn.applyToHelper(AFn.java:160) [puppetdb.jar:na] at clojure.lang.Var.applyTo(Var.java:700) [puppetdb.jar:na] at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na] at clojure.main$main_opt.invoke(main.clj:315) [puppetdb.jar:na] at clojure.main$main.doInvoke(main.clj:420) [puppetdb.jar:na] at clojure.lang.RestFn.invoke(RestFn.java:482) [puppetdb.jar:na] at clojure.lang.Var.invoke(Var.java:401) [puppetdb.jar:na] at clojure.lang.AFn.applyToHelper(AFn.java:171) [puppetdb.jar:na] at clojure.lang.Var.applyTo(Var.java:700) [puppetdb.jar:na] at clojure.main.main(main.java:37) [puppetdb.jar:na] 2015-04-01 08:44:53,403 INFO [p.t.internal] Shutting down due to JVM shutdown hook. 2015-04-01 08:44:53,404 INFO [p.t.internal] Beginning shutdown sequence 2015-04-01 08:44:53,406 INFO [c.p.p.c.services] Shutdown request received; puppetdb exiting. 2015-04-01 08:44:53,406 INFO [p.t.s.w.jetty9-service] Shutting down web server(s). 2015-04-01 08:44:53,407 INFO [p.t.s.w.jetty9-core] Shutting down web server. 2015-04-01 08:44:53,414 INFO [o.e.j.s.ServerConnector] Stopped ServerConnector@66ace155{HTTP/1.1}{localhost:8080} 2015-04-01 08:44:53,420 INFO [o.e.j.s.ServerConnector] Stopped ServerConnector@7f17e429{SSL-HTTP/1.1}{0.0.0.0:8081} 2015-04-01 08:44:53,422 INFO [p.t.s.w.jetty9-core] Web server shutdown 2015-04-01 08:44:53,423 INFO [p.t.internal] Finished shutdown sequence ---- h3. QA Risk Analysis | Probability | Low | | Severity | Med (puppetdb crash | | Risk Level | Low | | Test Level | Unit |

    JIRA | 2 years ago | Pete Brown
    java.sql.BatchUpdateException: Batch entry 5 DELETE FROM fact_paths t1 WHERE t1.id <> (SELECT MIN(t2.id) FROM fact_paths t2 WHERE t1.path = t2.path) was aborted. Call getNextException to see the cause.

    10 unregistered visitors
    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.sql.BatchUpdateException

      Batch entry 0 DELETE FROM resource_params_cache WHERE NOT EXISTS (SELECT * FROM catalog_resources cr WHERE cr.resource=resource_params_cache.resource) was aborted. Call getNextException to see the cause.

      at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError()
    2. PostgreSQL JDBC Driver
      AbstractJdbc2Statement.executeBatch
      1. org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2746)[puppetdb.jar:na]
      2. org.postgresql.core.v3.QueryExecutorImpl$1.handleError(QueryExecutorImpl.java:457)[puppetdb.jar:na]
      3. org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1887)[puppetdb.jar:na]
      4. org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:405)[puppetdb.jar:na]
      5. org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2893)[puppetdb.jar:na]
      5 frames
    3. BoneCP :: Core Library
      StatementHandle.executeBatch
      1. com.jolbox.bonecp.StatementHandle.executeBatch(StatementHandle.java:469)[puppetdb.jar:na]
      1 frame
    4. notificationagent
      internal$do_prepared_STAR_.doInvoke
      1. clojure.java.jdbc.internal$do_prepared_STAR_$fn__6350.invoke(internal.clj:356)[na:na]
      2. clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223)[na:na]
      3. clojure.java.jdbc.internal$do_prepared_STAR_.doInvoke(internal.clj:356)[na:na]
      3 frames
    5. Clojure
      jdbc$delete_rows.invoke
      1. clojure.lang.RestFn.invoke(RestFn.java:423)[puppetdb.jar:na]
      2. clojure.java.jdbc$delete_rows.invoke(jdbc.clj:297)[na:na]
      2 frames
    6. com.puppetlabs.puppetdb
      Object$Callable$7da976d4.call
      1. com.puppetlabs.puppetdb.scf.storage$delete_unassociated_params_BANG_$fn__9696.invoke(storage.clj:740)[na:na]
      2. com.puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source)[na:na]
      2 frames
    7. Apache Giraph Core
      Timer.time
      1. com.yammer.metrics.core.Timer.time(Timer.java:91)[puppetdb.jar:na]
      1 frame
    8. com.puppetlabs.puppetdb
      storage$garbage_collect_BANG_$fn__9708$fn__9709.invoke
      1. com.puppetlabs.puppetdb.scf.storage$delete_unassociated_params_BANG_.invoke(storage.clj:739)[na:na]
      2. com.puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_$fn__9708$fn__9709.invoke(storage.clj:767)[na:na]
      2 frames
    9. notificationagent
      internal$transaction_STAR_.invoke
      1. clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:223)[na:na]
      1 frame
    10. com.puppetlabs.puppetdb
      Object$Callable$7da976d4.call
      1. com.puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_$fn__9708.invoke(storage.clj:766)[na:na]
      2. com.puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source)[na:na]
      2 frames
    11. Apache Giraph Core
      Timer.time
      1. com.yammer.metrics.core.Timer.time(Timer.java:91)[puppetdb.jar:na]
      1 frame
    12. com.puppetlabs.puppetdb
      services$garbage_collect_BANG_$fn__19190.invoke
      1. com.puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_.invoke(storage.clj:765)[na:na]
      2. com.puppetlabs.puppetdb.cli.services$garbage_collect_BANG_$fn__19190.invoke(services.clj:165)[na:na]
      2 frames
    13. com.puppetlabs
      jdbc$with_transacted_connection_fn$fn__6761$fn__6762$fn__6763.invoke
      1. com.puppetlabs.jdbc$with_transacted_connection_fn$fn__6761$fn__6762$fn__6763.invoke(jdbc.clj:290)[na:na]
      1 frame
    14. notificationagent
      internal$transaction_STAR_.invoke
      1. clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:204)[na:na]
      1 frame
    15. com.puppetlabs
      jdbc$with_transacted_connection_fn$fn__6761$fn__6762.invoke
      1. com.puppetlabs.jdbc$with_transacted_connection_fn$fn__6761$fn__6762.invoke(jdbc.clj:290)[na:na]
      1 frame
    16. notificationagent
      internal$with_connection_STAR_.invoke
      1. clojure.java.jdbc.internal$with_connection_STAR_.invoke(internal.clj:186)[na:na]
      1 frame
    17. com.puppetlabs
      jdbc$with_transacted_connection_fn.invoke
      1. com.puppetlabs.jdbc$with_transacted_connection_fn$fn__6761.invoke(jdbc.clj:287)[na:na]
      2. com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740$fn__6741$fn__6742.invoke(jdbc.clj:259)[na:na]
      3. com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740$fn__6741.invoke(jdbc.clj:258)[na:na]
      4. com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740.invoke(jdbc.clj:250)[na:na]
      5. com.puppetlabs.jdbc$with_transacted_connection_fn.invoke(jdbc.clj:286)[na:na]
      5 frames
    18. com.puppetlabs.puppetdb
      services$perform_db_maintenance_BANG_.doInvoke
      1. com.puppetlabs.puppetdb.cli.services$garbage_collect_BANG_.invoke(services.clj:164)[na:na]
      2. com.puppetlabs.puppetdb.cli.services$perform_db_maintenance_BANG_.doInvoke(services.clj:177)[na:na]
      2 frames
    19. Clojure
      core$apply.invoke
      1. clojure.lang.RestFn.applyTo(RestFn.java:139)[puppetdb.jar:na]
      2. clojure.core$apply.invoke(core.clj:626)[puppetdb.jar:na]
      2 frames
    20. com.puppetlabs.puppetdb
      services$start_puppetdb$fn__19276.invoke
      1. com.puppetlabs.puppetdb.cli.services$start_puppetdb$fn__19276.invoke(services.clj:347)[na:na]
      1 frame
    21. Clojure
      AFn.run
      1. clojure.lang.AFn.run(AFn.java:22)[puppetdb.jar:na]
      1 frame
    22. Java RT
      Thread.run
      1. java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[na:1.7.0_71]
      2. java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)[na:1.7.0_71]
      3. java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)[na:1.7.0_71]
      4. java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)[na:1.7.0_71]
      5. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[na:1.7.0_71]
      6. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[na:1.7.0_71]
      7. java.lang.Thread.run(Thread.java:745)[na:1.7.0_71]
      7 frames