org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373

JIRA | Keiran Sweet | 6 months ago
  1. 0

    Hi There, I have been directed here by Ben Ford via IRC I’ve been building a Puppet 2016.1.2 environment on a client site for a PoC with the following configuration. Unfortunately they have not purchased licences yet, so I can’t put in a support call for this issue, however, I think it could be worth having someone look into this as i think there is something up with this release in the code manager and file sync space. The deployment is as follows * PE 2016.1.2 * RHEL 7 on VMWare * Deployment is an All in one + 2 additional compile masters. * All Puppet modules reside in an on premise Bitbucket deployment * Code Manager is used for manifest management and git repositories are pulled in using ssh:// paths to the repos * Install is done without code manager initially enabled, then i follow the documentation to set it up and configure against my control repo. I’ve encountered a couple of issues while doing this POC build that I’d like to raise as I think they need to be looked into. 1. File sync crash with LargeObjectException I’ve encountered this issue with fresh installs of 2016.1.2 and it is resolved via the settings in the below support document, I think it is worth updating the below link to detail that it is not specific to upgrades, but also fresh installs too. It would also be worth identifying if this needs to be set on all compile masters as well, or just the MoM. https://support.puppet.com/hc/en-us/articles/219376108-Resolving-a-file-sync-crash-and-org-eclipse-jgit-errors-LargeObjectException-error-after-upgrading-to-Puppet-Enterprise-2016-1-2 2. stdlib module deployed via code deploy appears to make changes to deployed code, resulting in a dirty repository. As part of my Puppetfile I pull in a version of puppetlabs-stdlib for use in the environments. I found that when referencing stdlib in my Puppetfile, code manager would build the environment and promote it to the code/ directory, however it would then report the modules as dirty as per the below: 2016-05-25 16:16:39,798 WARN [async-dispatch-2] [p.e.s.f.file-sync-client-core] Repo ':puppet-code' at commit '289ec3c20b100443a300af0455e9d7095307d735' is dirty. Detailed repo status follows. {:clean false, :modified [], :missing [], :untracked [], :permissions-modified [], :dirty-submodules {:master {:clean false, :permissions-modified [], :modified ["modules/stdlib/spec/fixtures/modules/stdlib/lib" "modules/stdlib/spec/fixtures/modules/stdlib/manifests"], :missing [], :untracked []}, :production {:clean false, :permissions-modified [], :modified ["modules/stdlib/spec/fixtures/modules/stdlib/lib" "modules/stdlib/spec/fixtures/modules/stdlib/manifests"], :missing [], :untracked []}}} If I go into each of these environments in the code directory, and run git status, it reports that there are changes made to the following paths: modules/stdlib/spec/fixtures/modules/stdlib/lib modules/stdlib/spec/fixtures/modules/stdlib/manifests If I run a git diff in these environments, It reports that these 2 files have different line endings, further inspection shows that these files are actually symlinks that are broken, thus making the issue very confusing indeed. (Apologies I don’t have the output, i’m in the process of rebuilding this platform) One thing I did try, which resolves the issue to a degree, is create a new branch of the stdlib module and remove the spec/ directory and update a test environments Puppetfile to reference that, once done, that particular environment deploys without the above symptoms, so it looks like something in the spec/ directory can cause issues with code manager. I haven’t been able to reproduce this with any other modules, just stdlib. I have also remirrored the repository to see if there there was any corruption and this has not solved the issue. 3. File Sync from MoM to Compile Master fails, bringing Puppetserver offline. The next issue, I encounter is when adding compile masters to the all in one install. I follow the documentation exactly here: https://docs.puppet.com/pe/latest/install_multimaster.html and all things seem to go OK, however once filesync commences, it creates the initial environment directories, however the Puppet server then fails with the below: ##### Example 1 2016-05-25 15:20:17,931 INFO [main] [o.e.j.u.log] Logging initialized @13289ms 2016-05-25 15:20:20,967 INFO [async-dispatch-2] [p.t.s.w.jetty9-service] Initializing web server(s). 2016-05-25 15:20:21,038 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-puppet-profiler service 2016-05-25 15:20:21,045 INFO [async-dispatch-2] [p.s.j.jruby-puppet-service] Initializing the JRuby service 2016-05-25 15:20:21,088 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-jruby-metrics service 2016-05-25 15:20:21,107 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-internal] Creating JRuby instance with id 1. 2016-05-25 15:20:35,111 INFO [clojure-agent-send-pool-0] [puppet-server] Puppet Puppet settings initialized; run mode: master 2016-05-25 15:20:35,957 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-agents] Finished creating JRubyPuppet instance 1 of 1 2016-05-25 15:20:35,970 INFO [async-dispatch-2] [p.s.c.puppet-server-config-core] Not overriding webserver settings with values from core Puppet 2016-05-25 15:20:35,972 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Initializing file sync client service 2016-05-25 15:20:35,977 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for file-sync-client-service service 2016-05-25 15:20:35,980 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Registering file sync client HTTP API 2016-05-25 15:20:36,119 ERROR [async-dispatch-2] [p.t.internal] Error during service init!!! org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:154) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005$fn__22006.invoke(jgit_utils.clj:537) ~[na:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005.invoke(jgit_utils.clj:534) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278$fn__25279.invoke(file_sync_client_core.clj:1247) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278.invoke(file_sync_client_core.clj:1245) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303$fn__25304.invoke(file_sync_client_core.clj:1259) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303.invoke(file_sync_client_core.clj:1249) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_service$reify__25535$service_fnk__5254__auto___positional$reify__25554.get_live_dir_status(file_sync_client_service.clj:101) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25434__25486.invoke(file_sync_client.clj:3) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25433__25490.invoke(file_sync_client.clj:3) ~[na:na] at clojure.core$partial$fn__4527.invoke(core.clj:2493) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262$get_working_dir_commits__28263.invoke(file_sync_versioned_code_service.clj:33) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715$fn__25716.invoke(file_sync_versioned_code_core.clj:66) ~[na:na] at clojure.core.protocols$iter_reduce.invoke(protocols.clj:49) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6510.invoke(protocols.clj:112) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13) ~[puppet-server-release.jar:na] at clojure.core$reduce.invoke(core.clj:6519) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715.invoke(file_sync_versioned_code_core.clj:64) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714.invoke(file_sync_versioned_code_core.clj:54) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262.init(file_sync_versioned_code_service.clj:39) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5048__5064.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5047__5068.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736$fn__13737.invoke(internal.clj:175) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736.invoke(internal.clj:158) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763$fn__13764.invoke(internal.clj:209) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763.invoke(internal.clj:186) [na:na] at puppetlabs.trapperkeeper.internal$eval14229$build_app_STAR___14238$fn$reify__14248.init(internal.clj:559) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283$fn__14285.invoke(internal.clj:585) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283.invoke(internal.clj:584) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282.invoke(internal.clj:578) [na:na] at clojure.core$partial$fn__4529.invoke(core.clj:2499) [puppet-server-release.jar:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897$fn__13899.invoke(internal.clj:226) [na:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897.invoke(internal.clj:226) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:1011) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:1015) [na:na] at clojure.core.async$ioc_alts_BANG_$fn__11963.invoke(async.clj:378) [na:na] at clojure.core.async$do_alts$fn__11909$fn__11912.invoke(async.clj:247) [na:na] at clojure.core.async.impl.channels.ManyToManyChannel$fn__6730$fn__6731.invoke(channels.clj:95) [na:na] at clojure.lang.AFn.run(AFn.java:22) [puppet-server-release.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] Caused by: org.eclipse.jgit.errors.MissingObjectException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:145) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.ObjectReader.open(ObjectReader.java:226) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseAny(RevWalk.java:859) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseTree(RevWalk.java:799) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.<init>(IndexDiff.java:314) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:551) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:376) ~[puppet-server-release.jar:na] at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:148) ~[puppet-server-release.jar:na] ... 41 common frames omitted 2016-05-25 15:20:36,126 INFO [main] [p.t.internal] Beginning shutdown sequence 2016-05-25 15:20:36,146 INFO [async-dispatch-3] [p.e.s.f.file-sync-client-service] Stopping file sync client service 2016-05-25 15:20:36,153 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopping metrics sampler job 2016-05-25 15:20:36,154 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopped metrics sampler job 2016-05-25 15:20:36,160 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Flush request received; creating new JRuby pool. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Replacing old JRuby pool with new instance. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Swapped JRuby pools, beginning cleanup of old pool. 2016-05-25 15:20:36,181 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-internal] Cleaned up old JRuby instance with id 1. 2016-05-25 15:20:36,193 INFO [async-dispatch-3] [p.t.s.w.jetty9-service] Shutting down web server(s). 2016-05-25 15:20:36,202 INFO [main] [p.t.internal] Finished shutdown sequence ##### Example 2 2016-05-25 15:20:17,931 INFO [main] [o.e.j.u.log] Logging initialized @13289ms 2016-05-25 15:20:20,967 INFO [async-dispatch-2] [p.t.s.w.jetty9-service] Initializing web server(s). 2016-05-25 15:20:21,038 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-puppet-profiler service 2016-05-25 15:20:21,045 INFO [async-dispatch-2] [p.s.j.jruby-puppet-service] Initializing the JRuby service 2016-05-25 15:20:21,088 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-jruby-metrics service 2016-05-25 15:20:21,107 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-internal] Creating JRuby instance with id 1. 2016-05-25 15:20:35,111 INFO [clojure-agent-send-pool-0] [puppet-server] Puppet Puppet settings initialized; run mode: master 2016-05-25 15:20:35,957 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-agents] Finished creating JRubyPuppet instance 1 of 1 2016-05-25 15:20:35,970 INFO [async-dispatch-2] [p.s.c.puppet-server-config-core] Not overriding webserver settings with values from core Puppet 2016-05-25 15:20:35,972 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Initializing file sync client service 2016-05-25 15:20:35,977 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for file-sync-client-service service 2016-05-25 15:20:35,980 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Registering file sync client HTTP API 2016-05-25 15:20:36,119 ERROR [async-dispatch-2] [p.t.internal] Error during service init!!! org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:154) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005$fn__22006.invoke(jgit_utils.clj:537) ~[na:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005.invoke(jgit_utils.clj:534) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278$fn__25279.invoke(file_sync_client_core.clj:1247) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278.invoke(file_sync_client_core.clj:1245) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303$fn__25304.invoke(file_sync_client_core.clj:1259) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303.invoke(file_sync_client_core.clj:1249) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_service$reify__25535$service_fnk__5254__auto___positional$reify__25554.get_live_dir_status(file_sync_client_service.clj:101) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25434__25486.invoke(file_sync_client.clj:3) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25433__25490.invoke(file_sync_client.clj:3) ~[na:na] at clojure.core$partial$fn__4527.invoke(core.clj:2493) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262$get_working_dir_commits__28263.invoke(file_sync_versioned_code_service.clj:33) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715$fn__25716.invoke(file_sync_versioned_code_core.clj:66) ~[na:na] at clojure.core.protocols$iter_reduce.invoke(protocols.clj:49) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6510.invoke(protocols.clj:112) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13) ~[puppet-server-release.jar:na] at clojure.core$reduce.invoke(core.clj:6519) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715.invoke(file_sync_versioned_code_core.clj:64) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714.invoke(file_sync_versioned_code_core.clj:54) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262.init(file_sync_versioned_code_service.clj:39) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5048__5064.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5047__5068.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736$fn__13737.invoke(internal.clj:175) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736.invoke(internal.clj:158) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763$fn__13764.invoke(internal.clj:209) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763.invoke(internal.clj:186) [na:na] at puppetlabs.trapperkeeper.internal$eval14229$build_app_STAR___14238$fn$reify__14248.init(internal.clj:559) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283$fn__14285.invoke(internal.clj:585) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283.invoke(internal.clj:584) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282.invoke(internal.clj:578) [na:na] at clojure.core$partial$fn__4529.invoke(core.clj:2499) [puppet-server-release.jar:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897$fn__13899.invoke(internal.clj:226) [na:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897.invoke(internal.clj:226) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:1011) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:1015) [na:na] at clojure.core.async$ioc_alts_BANG_$fn__11963.invoke(async.clj:378) [na:na] at clojure.core.async$do_alts$fn__11909$fn__11912.invoke(async.clj:247) [na:na] at clojure.core.async.impl.channels.ManyToManyChannel$fn__6730$fn__6731.invoke(channels.clj:95) [na:na] at clojure.lang.AFn.run(AFn.java:22) [puppet-server-release.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] Caused by: org.eclipse.jgit.errors.MissingObjectException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:145) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.ObjectReader.open(ObjectReader.java:226) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseAny(RevWalk.java:859) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseTree(RevWalk.java:799) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.<init>(IndexDiff.java:314) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:551) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:376) ~[puppet-server-release.jar:na] at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:148) ~[puppet-server-release.jar:na] ... 41 common frames omitted 2016-05-25 15:20:36,126 INFO [main] [p.t.internal] Beginning shutdown sequence 2016-05-25 15:20:36,146 INFO [async-dispatch-3] [p.e.s.f.file-sync-client-service] Stopping file sync client service 2016-05-25 15:20:36,153 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopping metrics sampler job 2016-05-25 15:20:36,154 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopped metrics sampler job 2016-05-25 15:20:36,160 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Flush request received; creating new JRuby pool. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Replacing old JRuby pool with new instance. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Swapped JRuby pools, beginning cleanup of old pool. 2016-05-25 15:20:36,181 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-internal] Cleaned up old JRuby instance with id 1. 2016-05-25 15:20:36,193 INFO [async-dispatch-3] [p.t.s.w.jetty9-service] Shutting down web server(s). 2016-05-25 15:20:36,202 INFO [main] [p.t.internal] Finished shutdown sequence I worked with some of the guys on IRC and we found that if you disable filesync, the master comes up OK, however code (obviously) does not get sync’d from the master. I’m not quite sure if these issues are related, however I didn’t encounter these with 2015.x, so I’ll likely redeploy this environment with a previous version as I’m running out of time to get this up and working. Let me know if you need any further information. Thanks, Keiran.Sweet@sourcedgroup.com

    JIRA | 6 months ago | Keiran Sweet
    org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373
  2. 0

    Hi There, I have been directed here by Ben Ford via IRC I’ve been building a Puppet 2016.1.2 environment on a client site for a PoC with the following configuration. Unfortunately they have not purchased licences yet, so I can’t put in a support call for this issue, however, I think it could be worth having someone look into this as i think there is something up with this release in the code manager and file sync space. The deployment is as follows * PE 2016.1.2 * RHEL 7 on VMWare * Deployment is an All in one + 2 additional compile masters. * All Puppet modules reside in an on premise Bitbucket deployment * Code Manager is used for manifest management and git repositories are pulled in using ssh:// paths to the repos * Install is done without code manager initially enabled, then i follow the documentation to set it up and configure against my control repo. I’ve encountered a couple of issues while doing this POC build that I’d like to raise as I think they need to be looked into. 1. File sync crash with LargeObjectException I’ve encountered this issue with fresh installs of 2016.1.2 and it is resolved via the settings in the below support document, I think it is worth updating the below link to detail that it is not specific to upgrades, but also fresh installs too. It would also be worth identifying if this needs to be set on all compile masters as well, or just the MoM. https://support.puppet.com/hc/en-us/articles/219376108-Resolving-a-file-sync-crash-and-org-eclipse-jgit-errors-LargeObjectException-error-after-upgrading-to-Puppet-Enterprise-2016-1-2 2. stdlib module deployed via code deploy appears to make changes to deployed code, resulting in a dirty repository. As part of my Puppetfile I pull in a version of puppetlabs-stdlib for use in the environments. I found that when referencing stdlib in my Puppetfile, code manager would build the environment and promote it to the code/ directory, however it would then report the modules as dirty as per the below: 2016-05-25 16:16:39,798 WARN [async-dispatch-2] [p.e.s.f.file-sync-client-core] Repo ':puppet-code' at commit '289ec3c20b100443a300af0455e9d7095307d735' is dirty. Detailed repo status follows. {:clean false, :modified [], :missing [], :untracked [], :permissions-modified [], :dirty-submodules {:master {:clean false, :permissions-modified [], :modified ["modules/stdlib/spec/fixtures/modules/stdlib/lib" "modules/stdlib/spec/fixtures/modules/stdlib/manifests"], :missing [], :untracked []}, :production {:clean false, :permissions-modified [], :modified ["modules/stdlib/spec/fixtures/modules/stdlib/lib" "modules/stdlib/spec/fixtures/modules/stdlib/manifests"], :missing [], :untracked []}}} If I go into each of these environments in the code directory, and run git status, it reports that there are changes made to the following paths: modules/stdlib/spec/fixtures/modules/stdlib/lib modules/stdlib/spec/fixtures/modules/stdlib/manifests If I run a git diff in these environments, It reports that these 2 files have different line endings, further inspection shows that these files are actually symlinks that are broken, thus making the issue very confusing indeed. (Apologies I don’t have the output, i’m in the process of rebuilding this platform) One thing I did try, which resolves the issue to a degree, is create a new branch of the stdlib module and remove the spec/ directory and update a test environments Puppetfile to reference that, once done, that particular environment deploys without the above symptoms, so it looks like something in the spec/ directory can cause issues with code manager. I haven’t been able to reproduce this with any other modules, just stdlib. I have also remirrored the repository to see if there there was any corruption and this has not solved the issue. 3. File Sync from MoM to Compile Master fails, bringing Puppetserver offline. The next issue, I encounter is when adding compile masters to the all in one install. I follow the documentation exactly here: https://docs.puppet.com/pe/latest/install_multimaster.html and all things seem to go OK, however once filesync commences, it creates the initial environment directories, however the Puppet server then fails with the below: ##### Example 1 2016-05-25 15:20:17,931 INFO [main] [o.e.j.u.log] Logging initialized @13289ms 2016-05-25 15:20:20,967 INFO [async-dispatch-2] [p.t.s.w.jetty9-service] Initializing web server(s). 2016-05-25 15:20:21,038 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-puppet-profiler service 2016-05-25 15:20:21,045 INFO [async-dispatch-2] [p.s.j.jruby-puppet-service] Initializing the JRuby service 2016-05-25 15:20:21,088 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-jruby-metrics service 2016-05-25 15:20:21,107 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-internal] Creating JRuby instance with id 1. 2016-05-25 15:20:35,111 INFO [clojure-agent-send-pool-0] [puppet-server] Puppet Puppet settings initialized; run mode: master 2016-05-25 15:20:35,957 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-agents] Finished creating JRubyPuppet instance 1 of 1 2016-05-25 15:20:35,970 INFO [async-dispatch-2] [p.s.c.puppet-server-config-core] Not overriding webserver settings with values from core Puppet 2016-05-25 15:20:35,972 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Initializing file sync client service 2016-05-25 15:20:35,977 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for file-sync-client-service service 2016-05-25 15:20:35,980 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Registering file sync client HTTP API 2016-05-25 15:20:36,119 ERROR [async-dispatch-2] [p.t.internal] Error during service init!!! org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:154) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005$fn__22006.invoke(jgit_utils.clj:537) ~[na:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005.invoke(jgit_utils.clj:534) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278$fn__25279.invoke(file_sync_client_core.clj:1247) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278.invoke(file_sync_client_core.clj:1245) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303$fn__25304.invoke(file_sync_client_core.clj:1259) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303.invoke(file_sync_client_core.clj:1249) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_service$reify__25535$service_fnk__5254__auto___positional$reify__25554.get_live_dir_status(file_sync_client_service.clj:101) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25434__25486.invoke(file_sync_client.clj:3) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25433__25490.invoke(file_sync_client.clj:3) ~[na:na] at clojure.core$partial$fn__4527.invoke(core.clj:2493) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262$get_working_dir_commits__28263.invoke(file_sync_versioned_code_service.clj:33) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715$fn__25716.invoke(file_sync_versioned_code_core.clj:66) ~[na:na] at clojure.core.protocols$iter_reduce.invoke(protocols.clj:49) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6510.invoke(protocols.clj:112) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13) ~[puppet-server-release.jar:na] at clojure.core$reduce.invoke(core.clj:6519) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715.invoke(file_sync_versioned_code_core.clj:64) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714.invoke(file_sync_versioned_code_core.clj:54) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262.init(file_sync_versioned_code_service.clj:39) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5048__5064.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5047__5068.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736$fn__13737.invoke(internal.clj:175) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736.invoke(internal.clj:158) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763$fn__13764.invoke(internal.clj:209) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763.invoke(internal.clj:186) [na:na] at puppetlabs.trapperkeeper.internal$eval14229$build_app_STAR___14238$fn$reify__14248.init(internal.clj:559) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283$fn__14285.invoke(internal.clj:585) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283.invoke(internal.clj:584) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282.invoke(internal.clj:578) [na:na] at clojure.core$partial$fn__4529.invoke(core.clj:2499) [puppet-server-release.jar:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897$fn__13899.invoke(internal.clj:226) [na:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897.invoke(internal.clj:226) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:1011) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:1015) [na:na] at clojure.core.async$ioc_alts_BANG_$fn__11963.invoke(async.clj:378) [na:na] at clojure.core.async$do_alts$fn__11909$fn__11912.invoke(async.clj:247) [na:na] at clojure.core.async.impl.channels.ManyToManyChannel$fn__6730$fn__6731.invoke(channels.clj:95) [na:na] at clojure.lang.AFn.run(AFn.java:22) [puppet-server-release.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] Caused by: org.eclipse.jgit.errors.MissingObjectException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:145) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.ObjectReader.open(ObjectReader.java:226) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseAny(RevWalk.java:859) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseTree(RevWalk.java:799) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.<init>(IndexDiff.java:314) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:551) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:376) ~[puppet-server-release.jar:na] at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:148) ~[puppet-server-release.jar:na] ... 41 common frames omitted 2016-05-25 15:20:36,126 INFO [main] [p.t.internal] Beginning shutdown sequence 2016-05-25 15:20:36,146 INFO [async-dispatch-3] [p.e.s.f.file-sync-client-service] Stopping file sync client service 2016-05-25 15:20:36,153 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopping metrics sampler job 2016-05-25 15:20:36,154 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopped metrics sampler job 2016-05-25 15:20:36,160 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Flush request received; creating new JRuby pool. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Replacing old JRuby pool with new instance. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Swapped JRuby pools, beginning cleanup of old pool. 2016-05-25 15:20:36,181 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-internal] Cleaned up old JRuby instance with id 1. 2016-05-25 15:20:36,193 INFO [async-dispatch-3] [p.t.s.w.jetty9-service] Shutting down web server(s). 2016-05-25 15:20:36,202 INFO [main] [p.t.internal] Finished shutdown sequence ##### Example 2 2016-05-25 15:20:17,931 INFO [main] [o.e.j.u.log] Logging initialized @13289ms 2016-05-25 15:20:20,967 INFO [async-dispatch-2] [p.t.s.w.jetty9-service] Initializing web server(s). 2016-05-25 15:20:21,038 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-puppet-profiler service 2016-05-25 15:20:21,045 INFO [async-dispatch-2] [p.s.j.jruby-puppet-service] Initializing the JRuby service 2016-05-25 15:20:21,088 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-jruby-metrics service 2016-05-25 15:20:21,107 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-internal] Creating JRuby instance with id 1. 2016-05-25 15:20:35,111 INFO [clojure-agent-send-pool-0] [puppet-server] Puppet Puppet settings initialized; run mode: master 2016-05-25 15:20:35,957 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-agents] Finished creating JRubyPuppet instance 1 of 1 2016-05-25 15:20:35,970 INFO [async-dispatch-2] [p.s.c.puppet-server-config-core] Not overriding webserver settings with values from core Puppet 2016-05-25 15:20:35,972 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Initializing file sync client service 2016-05-25 15:20:35,977 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for file-sync-client-service service 2016-05-25 15:20:35,980 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Registering file sync client HTTP API 2016-05-25 15:20:36,119 ERROR [async-dispatch-2] [p.t.internal] Error during service init!!! org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:154) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005$fn__22006.invoke(jgit_utils.clj:537) ~[na:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005.invoke(jgit_utils.clj:534) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278$fn__25279.invoke(file_sync_client_core.clj:1247) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278.invoke(file_sync_client_core.clj:1245) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303$fn__25304.invoke(file_sync_client_core.clj:1259) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303.invoke(file_sync_client_core.clj:1249) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_service$reify__25535$service_fnk__5254__auto___positional$reify__25554.get_live_dir_status(file_sync_client_service.clj:101) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25434__25486.invoke(file_sync_client.clj:3) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25433__25490.invoke(file_sync_client.clj:3) ~[na:na] at clojure.core$partial$fn__4527.invoke(core.clj:2493) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262$get_working_dir_commits__28263.invoke(file_sync_versioned_code_service.clj:33) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715$fn__25716.invoke(file_sync_versioned_code_core.clj:66) ~[na:na] at clojure.core.protocols$iter_reduce.invoke(protocols.clj:49) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6510.invoke(protocols.clj:112) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13) ~[puppet-server-release.jar:na] at clojure.core$reduce.invoke(core.clj:6519) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715.invoke(file_sync_versioned_code_core.clj:64) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714.invoke(file_sync_versioned_code_core.clj:54) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262.init(file_sync_versioned_code_service.clj:39) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5048__5064.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5047__5068.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736$fn__13737.invoke(internal.clj:175) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736.invoke(internal.clj:158) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763$fn__13764.invoke(internal.clj:209) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763.invoke(internal.clj:186) [na:na] at puppetlabs.trapperkeeper.internal$eval14229$build_app_STAR___14238$fn$reify__14248.init(internal.clj:559) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283$fn__14285.invoke(internal.clj:585) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283.invoke(internal.clj:584) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282.invoke(internal.clj:578) [na:na] at clojure.core$partial$fn__4529.invoke(core.clj:2499) [puppet-server-release.jar:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897$fn__13899.invoke(internal.clj:226) [na:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897.invoke(internal.clj:226) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:1011) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:1015) [na:na] at clojure.core.async$ioc_alts_BANG_$fn__11963.invoke(async.clj:378) [na:na] at clojure.core.async$do_alts$fn__11909$fn__11912.invoke(async.clj:247) [na:na] at clojure.core.async.impl.channels.ManyToManyChannel$fn__6730$fn__6731.invoke(channels.clj:95) [na:na] at clojure.lang.AFn.run(AFn.java:22) [puppet-server-release.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] Caused by: org.eclipse.jgit.errors.MissingObjectException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:145) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.ObjectReader.open(ObjectReader.java:226) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseAny(RevWalk.java:859) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseTree(RevWalk.java:799) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.<init>(IndexDiff.java:314) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:551) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:376) ~[puppet-server-release.jar:na] at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:148) ~[puppet-server-release.jar:na] ... 41 common frames omitted 2016-05-25 15:20:36,126 INFO [main] [p.t.internal] Beginning shutdown sequence 2016-05-25 15:20:36,146 INFO [async-dispatch-3] [p.e.s.f.file-sync-client-service] Stopping file sync client service 2016-05-25 15:20:36,153 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopping metrics sampler job 2016-05-25 15:20:36,154 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopped metrics sampler job 2016-05-25 15:20:36,160 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Flush request received; creating new JRuby pool. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Replacing old JRuby pool with new instance. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Swapped JRuby pools, beginning cleanup of old pool. 2016-05-25 15:20:36,181 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-internal] Cleaned up old JRuby instance with id 1. 2016-05-25 15:20:36,193 INFO [async-dispatch-3] [p.t.s.w.jetty9-service] Shutting down web server(s). 2016-05-25 15:20:36,202 INFO [main] [p.t.internal] Finished shutdown sequence I worked with some of the guys on IRC and we found that if you disable filesync, the master comes up OK, however code (obviously) does not get sync’d from the master. I’m not quite sure if these issues are related, however I didn’t encounter these with 2015.x, so I’ll likely redeploy this environment with a previous version as I’m running out of time to get this up and working. Let me know if you need any further information. Thanks, Keiran.Sweet@sourcedgroup.com

    JIRA | 6 months ago | Keiran Sweet
    org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373
  3. 0

    Hi There, I have been directed here by Ben Ford via IRC I’ve been building a Puppet 2016.1.2 environment on a client site for a PoC with the following configuration. Unfortunately they have not purchased licences yet, so I can’t put in a support call for this issue, however, I think it could be worth having someone look into this as i think there is something up with this release in the code manager and file sync space. The deployment is as follows * PE 2016.1.2 * RHEL 7 on VMWare * Deployment is an All in one + 2 additional compile masters. * All Puppet modules reside in an on premise Bitbucket deployment * Code Manager is used for manifest management and git repositories are pulled in using ssh:// paths to the repos * Install is done without code manager initially enabled, then i follow the documentation to set it up and configure against my control repo. I’ve encountered a couple of issues while doing this POC build that I’d like to raise as I think they need to be looked into. 1. File sync crash with LargeObjectException I’ve encountered this issue with fresh installs of 2016.1.2 and it is resolved via the settings in the below support document, I think it is worth updating the below link to detail that it is not specific to upgrades, but also fresh installs too. It would also be worth identifying if this needs to be set on all compile masters as well, or just the MoM. https://support.puppet.com/hc/en-us/articles/219376108-Resolving-a-file-sync-crash-and-org-eclipse-jgit-errors-LargeObjectException-error-after-upgrading-to-Puppet-Enterprise-2016-1-2 2. stdlib module deployed via code deploy appears to make changes to deployed code, resulting in a dirty repository. As part of my Puppetfile I pull in a version of puppetlabs-stdlib for use in the environments. I found that when referencing stdlib in my Puppetfile, code manager would build the environment and promote it to the code/ directory, however it would then report the modules as dirty as per the below: 2016-05-25 16:16:39,798 WARN [async-dispatch-2] [p.e.s.f.file-sync-client-core] Repo ':puppet-code' at commit '289ec3c20b100443a300af0455e9d7095307d735' is dirty. Detailed repo status follows. {:clean false, :modified [], :missing [], :untracked [], :permissions-modified [], :dirty-submodules {:master {:clean false, :permissions-modified [], :modified ["modules/stdlib/spec/fixtures/modules/stdlib/lib" "modules/stdlib/spec/fixtures/modules/stdlib/manifests"], :missing [], :untracked []}, :production {:clean false, :permissions-modified [], :modified ["modules/stdlib/spec/fixtures/modules/stdlib/lib" "modules/stdlib/spec/fixtures/modules/stdlib/manifests"], :missing [], :untracked []}}} If I go into each of these environments in the code directory, and run git status, it reports that there are changes made to the following paths: modules/stdlib/spec/fixtures/modules/stdlib/lib modules/stdlib/spec/fixtures/modules/stdlib/manifests If I run a git diff in these environments, It reports that these 2 files have different line endings, further inspection shows that these files are actually symlinks that are broken, thus making the issue very confusing indeed. (Apologies I don’t have the output, i’m in the process of rebuilding this platform) One thing I did try, which resolves the issue to a degree, is create a new branch of the stdlib module and remove the spec/ directory and update a test environments Puppetfile to reference that, once done, that particular environment deploys without the above symptoms, so it looks like something in the spec/ directory can cause issues with code manager. I haven’t been able to reproduce this with any other modules, just stdlib. I have also remirrored the repository to see if there there was any corruption and this has not solved the issue. 3. File Sync from MoM to Compile Master fails, bringing Puppetserver offline. The next issue, I encounter is when adding compile masters to the all in one install. I follow the documentation exactly here: https://docs.puppet.com/pe/latest/install_multimaster.html and all things seem to go OK, however once filesync commences, it creates the initial environment directories, however the Puppet server then fails with the below: ##### Example 1 2016-05-25 15:20:17,931 INFO [main] [o.e.j.u.log] Logging initialized @13289ms 2016-05-25 15:20:20,967 INFO [async-dispatch-2] [p.t.s.w.jetty9-service] Initializing web server(s). 2016-05-25 15:20:21,038 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-puppet-profiler service 2016-05-25 15:20:21,045 INFO [async-dispatch-2] [p.s.j.jruby-puppet-service] Initializing the JRuby service 2016-05-25 15:20:21,088 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-jruby-metrics service 2016-05-25 15:20:21,107 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-internal] Creating JRuby instance with id 1. 2016-05-25 15:20:35,111 INFO [clojure-agent-send-pool-0] [puppet-server] Puppet Puppet settings initialized; run mode: master 2016-05-25 15:20:35,957 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-agents] Finished creating JRubyPuppet instance 1 of 1 2016-05-25 15:20:35,970 INFO [async-dispatch-2] [p.s.c.puppet-server-config-core] Not overriding webserver settings with values from core Puppet 2016-05-25 15:20:35,972 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Initializing file sync client service 2016-05-25 15:20:35,977 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for file-sync-client-service service 2016-05-25 15:20:35,980 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Registering file sync client HTTP API 2016-05-25 15:20:36,119 ERROR [async-dispatch-2] [p.t.internal] Error during service init!!! org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:154) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005$fn__22006.invoke(jgit_utils.clj:537) ~[na:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005.invoke(jgit_utils.clj:534) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278$fn__25279.invoke(file_sync_client_core.clj:1247) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278.invoke(file_sync_client_core.clj:1245) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303$fn__25304.invoke(file_sync_client_core.clj:1259) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303.invoke(file_sync_client_core.clj:1249) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_service$reify__25535$service_fnk__5254__auto___positional$reify__25554.get_live_dir_status(file_sync_client_service.clj:101) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25434__25486.invoke(file_sync_client.clj:3) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25433__25490.invoke(file_sync_client.clj:3) ~[na:na] at clojure.core$partial$fn__4527.invoke(core.clj:2493) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262$get_working_dir_commits__28263.invoke(file_sync_versioned_code_service.clj:33) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715$fn__25716.invoke(file_sync_versioned_code_core.clj:66) ~[na:na] at clojure.core.protocols$iter_reduce.invoke(protocols.clj:49) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6510.invoke(protocols.clj:112) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13) ~[puppet-server-release.jar:na] at clojure.core$reduce.invoke(core.clj:6519) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715.invoke(file_sync_versioned_code_core.clj:64) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714.invoke(file_sync_versioned_code_core.clj:54) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262.init(file_sync_versioned_code_service.clj:39) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5048__5064.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5047__5068.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736$fn__13737.invoke(internal.clj:175) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736.invoke(internal.clj:158) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763$fn__13764.invoke(internal.clj:209) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763.invoke(internal.clj:186) [na:na] at puppetlabs.trapperkeeper.internal$eval14229$build_app_STAR___14238$fn$reify__14248.init(internal.clj:559) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283$fn__14285.invoke(internal.clj:585) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283.invoke(internal.clj:584) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282.invoke(internal.clj:578) [na:na] at clojure.core$partial$fn__4529.invoke(core.clj:2499) [puppet-server-release.jar:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897$fn__13899.invoke(internal.clj:226) [na:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897.invoke(internal.clj:226) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:1011) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:1015) [na:na] at clojure.core.async$ioc_alts_BANG_$fn__11963.invoke(async.clj:378) [na:na] at clojure.core.async$do_alts$fn__11909$fn__11912.invoke(async.clj:247) [na:na] at clojure.core.async.impl.channels.ManyToManyChannel$fn__6730$fn__6731.invoke(channels.clj:95) [na:na] at clojure.lang.AFn.run(AFn.java:22) [puppet-server-release.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] Caused by: org.eclipse.jgit.errors.MissingObjectException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:145) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.ObjectReader.open(ObjectReader.java:226) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseAny(RevWalk.java:859) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseTree(RevWalk.java:799) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.<init>(IndexDiff.java:314) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:551) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:376) ~[puppet-server-release.jar:na] at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:148) ~[puppet-server-release.jar:na] ... 41 common frames omitted 2016-05-25 15:20:36,126 INFO [main] [p.t.internal] Beginning shutdown sequence 2016-05-25 15:20:36,146 INFO [async-dispatch-3] [p.e.s.f.file-sync-client-service] Stopping file sync client service 2016-05-25 15:20:36,153 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopping metrics sampler job 2016-05-25 15:20:36,154 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopped metrics sampler job 2016-05-25 15:20:36,160 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Flush request received; creating new JRuby pool. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Replacing old JRuby pool with new instance. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Swapped JRuby pools, beginning cleanup of old pool. 2016-05-25 15:20:36,181 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-internal] Cleaned up old JRuby instance with id 1. 2016-05-25 15:20:36,193 INFO [async-dispatch-3] [p.t.s.w.jetty9-service] Shutting down web server(s). 2016-05-25 15:20:36,202 INFO [main] [p.t.internal] Finished shutdown sequence ##### Example 2 2016-05-25 15:20:17,931 INFO [main] [o.e.j.u.log] Logging initialized @13289ms 2016-05-25 15:20:20,967 INFO [async-dispatch-2] [p.t.s.w.jetty9-service] Initializing web server(s). 2016-05-25 15:20:21,038 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-puppet-profiler service 2016-05-25 15:20:21,045 INFO [async-dispatch-2] [p.s.j.jruby-puppet-service] Initializing the JRuby service 2016-05-25 15:20:21,088 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-jruby-metrics service 2016-05-25 15:20:21,107 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-internal] Creating JRuby instance with id 1. 2016-05-25 15:20:35,111 INFO [clojure-agent-send-pool-0] [puppet-server] Puppet Puppet settings initialized; run mode: master 2016-05-25 15:20:35,957 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-agents] Finished creating JRubyPuppet instance 1 of 1 2016-05-25 15:20:35,970 INFO [async-dispatch-2] [p.s.c.puppet-server-config-core] Not overriding webserver settings with values from core Puppet 2016-05-25 15:20:35,972 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Initializing file sync client service 2016-05-25 15:20:35,977 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for file-sync-client-service service 2016-05-25 15:20:35,980 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Registering file sync client HTTP API 2016-05-25 15:20:36,119 ERROR [async-dispatch-2] [p.t.internal] Error during service init!!! org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:154) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005$fn__22006.invoke(jgit_utils.clj:537) ~[na:na] at puppetlabs.enterprise.jgit_utils$eval22000$status__22005.invoke(jgit_utils.clj:534) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278$fn__25279.invoke(file_sync_client_core.clj:1247) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278.invoke(file_sync_client_core.clj:1245) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303$fn__25304.invoke(file_sync_client_core.clj:1259) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303.invoke(file_sync_client_core.clj:1249) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_service$reify__25535$service_fnk__5254__auto___positional$reify__25554.get_live_dir_status(file_sync_client_service.clj:101) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25434__25486.invoke(file_sync_client.clj:3) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25433__25490.invoke(file_sync_client.clj:3) ~[na:na] at clojure.core$partial$fn__4527.invoke(core.clj:2493) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262$get_working_dir_commits__28263.invoke(file_sync_versioned_code_service.clj:33) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715$fn__25716.invoke(file_sync_versioned_code_core.clj:66) ~[na:na] at clojure.core.protocols$iter_reduce.invoke(protocols.clj:49) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6510.invoke(protocols.clj:112) ~[puppet-server-release.jar:na] at clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13) ~[puppet-server-release.jar:na] at clojure.core$reduce.invoke(core.clj:6519) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715.invoke(file_sync_versioned_code_core.clj:64) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714.invoke(file_sync_versioned_code_core.clj:54) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262.init(file_sync_versioned_code_service.clj:39) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5048__5064.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5047__5068.invoke(services.clj:8) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736$fn__13737.invoke(internal.clj:175) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736.invoke(internal.clj:158) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763$fn__13764.invoke(internal.clj:209) ~[na:na] at puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763.invoke(internal.clj:186) [na:na] at puppetlabs.trapperkeeper.internal$eval14229$build_app_STAR___14238$fn$reify__14248.init(internal.clj:559) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283$fn__14285.invoke(internal.clj:585) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283.invoke(internal.clj:584) [na:na] at puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282.invoke(internal.clj:578) [na:na] at clojure.core$partial$fn__4529.invoke(core.clj:2499) [puppet-server-release.jar:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897$fn__13899.invoke(internal.clj:226) [na:na] at puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897.invoke(internal.clj:226) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:1011) [na:na] at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:1015) [na:na] at clojure.core.async$ioc_alts_BANG_$fn__11963.invoke(async.clj:378) [na:na] at clojure.core.async$do_alts$fn__11909$fn__11912.invoke(async.clj:247) [na:na] at clojure.core.async.impl.channels.ManyToManyChannel$fn__6730$fn__6731.invoke(channels.clj:95) [na:na] at clojure.lang.AFn.run(AFn.java:22) [puppet-server-release.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] Caused by: org.eclipse.jgit.errors.MissingObjectException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373 at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:145) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.ObjectReader.open(ObjectReader.java:226) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseAny(RevWalk.java:859) ~[puppet-server-release.jar:na] at org.eclipse.jgit.revwalk.RevWalk.parseTree(RevWalk.java:799) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.<init>(IndexDiff.java:314) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:551) ~[puppet-server-release.jar:na] at org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:376) ~[puppet-server-release.jar:na] at org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:148) ~[puppet-server-release.jar:na] ... 41 common frames omitted 2016-05-25 15:20:36,126 INFO [main] [p.t.internal] Beginning shutdown sequence 2016-05-25 15:20:36,146 INFO [async-dispatch-3] [p.e.s.f.file-sync-client-service] Stopping file sync client service 2016-05-25 15:20:36,153 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopping metrics sampler job 2016-05-25 15:20:36,154 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopped metrics sampler job 2016-05-25 15:20:36,160 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Flush request received; creating new JRuby pool. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Replacing old JRuby pool with new instance. 2016-05-25 15:20:36,168 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Swapped JRuby pools, beginning cleanup of old pool. 2016-05-25 15:20:36,181 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-internal] Cleaned up old JRuby instance with id 1. 2016-05-25 15:20:36,193 INFO [async-dispatch-3] [p.t.s.w.jetty9-service] Shutting down web server(s). 2016-05-25 15:20:36,202 INFO [main] [p.t.internal] Finished shutdown sequence I worked with some of the guys on IRC and we found that if you disable filesync, the master comes up OK, however code (obviously) does not get sync’d from the master. I’m not quite sure if these issues are related, however I didn’t encounter these with 2015.x, so I’ll likely redeploy this environment with a previous version as I’m running out of time to get this up and working. Let me know if you need any further information. Thanks, Keiran.Sweet@sourcedgroup.com

    JIRA | 6 months ago | Keiran Sweet
    org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 0000000000000000000000000000000000000000

    GitHub | 10 months ago | mslinn
    org.eclipse.jgit.api.errors.JGitInternalException: Missing unknown 0000000000000000000000000000000000000000
  6. 0

    Could not read an object while parsing commit

    GitHub | 4 months ago | crosser
    org.eclipse.jgit.api.errors.JGitInternalException: Could not read an object while parsing commit d36838975f0c14d43e6aa119874a8536c6a281c3

    1 unregistered visitors
    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. org.eclipse.jgit.errors.MissingObjectException

      Missing unknown 71411e686b9ac9429915c470b68d753cedcd4373

      at org.eclipse.jgit.internal.storage.file.WindowCursor.open()
    2. JGit - Core
      StatusCommand.call
      1. org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:145)[puppet-server-release.jar:na]
      2. org.eclipse.jgit.lib.ObjectReader.open(ObjectReader.java:226)[puppet-server-release.jar:na]
      3. org.eclipse.jgit.revwalk.RevWalk.parseAny(RevWalk.java:859)[puppet-server-release.jar:na]
      4. org.eclipse.jgit.revwalk.RevWalk.parseTree(RevWalk.java:799)[puppet-server-release.jar:na]
      5. org.eclipse.jgit.lib.IndexDiff.<init>(IndexDiff.java:314)[puppet-server-release.jar:na]
      6. org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:551)[puppet-server-release.jar:na]
      7. org.eclipse.jgit.lib.IndexDiff.diff(IndexDiff.java:376)[puppet-server-release.jar:na]
      8. org.eclipse.jgit.api.StatusCommand.call(StatusCommand.java:148)[puppet-server-release.jar:na]
      8 frames
    3. puppetlabs.enterprise
      jgit_utils$eval22000$status__22005.invoke
      1. puppetlabs.enterprise.jgit_utils$eval22000$status__22005$fn__22006.invoke(jgit_utils.clj:537)[na:na]
      2. puppetlabs.enterprise.jgit_utils$eval22000$status__22005.invoke(jgit_utils.clj:534)[na:na]
      2 frames
    4. puppetlabs.enterprise.services
      file_sync_client$eval25446$fn__25483$G__25433__25490.invoke
      1. puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278$fn__25279.invoke(file_sync_client_core.clj:1247)[na:na]
      2. puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25273$live_dir_status_info__25278.invoke(file_sync_client_core.clj:1245)[na:na]
      3. puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303$fn__25304.invoke(file_sync_client_core.clj:1259)[na:na]
      4. puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval25298$get_live_dir_status__25303.invoke(file_sync_client_core.clj:1249)[na:na]
      5. puppetlabs.enterprise.services.file_sync_client.file_sync_client_service$reify__25535$service_fnk__5254__auto___positional$reify__25554.get_live_dir_status(file_sync_client_service.clj:101)[na:na]
      6. puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25434__25486.invoke(file_sync_client.clj:3)[na:na]
      7. puppetlabs.enterprise.services.protocols.file_sync_client$eval25446$fn__25483$G__25433__25490.invoke(file_sync_client.clj:3)[na:na]
      7 frames
    5. Clojure
      core$partial$fn__4527.invoke
      1. clojure.core$partial$fn__4527.invoke(core.clj:2493)[puppet-server-release.jar:na]
      1 frame
    6. puppetlabs.enterprise.services
      file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715$fn__25716.invoke
      1. puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262$get_working_dir_commits__28263.invoke(file_sync_versioned_code_service.clj:33)[na:na]
      2. puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715$fn__25716.invoke(file_sync_versioned_code_core.clj:66)[na:na]
      2 frames
    7. Clojure
      core$reduce.invoke
      1. clojure.core.protocols$iter_reduce.invoke(protocols.clj:49)[puppet-server-release.jar:na]
      2. clojure.core.protocols$fn__6510.invoke(protocols.clj:112)[puppet-server-release.jar:na]
      3. clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13)[puppet-server-release.jar:na]
      4. clojure.core$reduce.invoke(core.clj:6519)[puppet-server-release.jar:na]
      4 frames
    8. puppetlabs.enterprise.services
      file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262.init
      1. puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714$fn__25715.invoke(file_sync_versioned_code_core.clj:64)[na:na]
      2. puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_core$eval25709$live_dir_commits__25714.invoke(file_sync_versioned_code_core.clj:54)[na:na]
      3. puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__28251$service_fnk__5254__auto___positional$reify__28262.init(file_sync_versioned_code_service.clj:39)[na:na]
      3 frames
    9. puppetlabs.trapperkeeper
      internal$eval14275$boot_services_for_app_STAR__STAR___14282.invoke
      1. puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5048__5064.invoke(services.clj:8)[na:na]
      2. puppetlabs.trapperkeeper.services$eval5060$fn__5061$G__5047__5068.invoke(services.clj:8)[na:na]
      3. puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736$fn__13737.invoke(internal.clj:175)[na:na]
      4. puppetlabs.trapperkeeper.internal$eval13729$run_lifecycle_fn_BANG___13736.invoke(internal.clj:158)[na:na]
      5. puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763$fn__13764.invoke(internal.clj:209)[na:na]
      6. puppetlabs.trapperkeeper.internal$eval13758$run_lifecycle_fns__13763.invoke(internal.clj:186)[na:na]
      7. puppetlabs.trapperkeeper.internal$eval14229$build_app_STAR___14238$fn$reify__14248.init(internal.clj:559)[na:na]
      8. puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283$fn__14285.invoke(internal.clj:585)[na:na]
      9. puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282$fn__14283.invoke(internal.clj:584)[na:na]
      10. puppetlabs.trapperkeeper.internal$eval14275$boot_services_for_app_STAR__STAR___14282.invoke(internal.clj:578)[na:na]
      10 frames
    10. Clojure
      core$partial$fn__4529.invoke
      1. clojure.core$partial$fn__4529.invoke(core.clj:2499)[puppet-server-release.jar:na]
      1 frame
    11. puppetlabs.trapperkeeper
      internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897.invoke
      1. puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897$fn__13899.invoke(internal.clj:226)[na:na]
      2. puppetlabs.trapperkeeper.internal$eval13797$initialize_lifecycle_worker__13808$fn__13809$fn__13896$state_machine__11795__auto____13897.invoke(internal.clj:226)[na:na]
      2 frames
    12. pigpen
      ioc_macros$run_state_machine_wrapped.invoke
      1. clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:1011)[na:na]
      2. clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:1015)[na:na]
      2 frames
    13. Clojure
      async$do_alts$fn__11909$fn__11912.invoke
      1. clojure.core.async$ioc_alts_BANG_$fn__11963.invoke(async.clj:378)[na:na]
      2. clojure.core.async$do_alts$fn__11909$fn__11912.invoke(async.clj:247)[na:na]
      2 frames
    14. pigpen
      ManyToManyChannel$fn__6730$fn__6731.invoke
      1. clojure.core.async.impl.channels.ManyToManyChannel$fn__6730$fn__6731.invoke(channels.clj:95)[na:na]
      1 frame
    15. Clojure
      AFn.run
      1. clojure.lang.AFn.run(AFn.java:22)[puppet-server-release.jar:na]
      1 frame
    16. Java RT
      Thread.run
      1. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[na:1.8.0_91]
      2. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[na:1.8.0_91]
      3. java.lang.Thread.run(Thread.java:745)[na:1.8.0_91]
      3 frames