java.lang.NullPointerExceptionjava.lang.NullPointerException

JIRA | Nidhi Mehta | 3 months ago
  1. 0

    H2O Build git hash d651ad244a9510d286c6b6f0bc4a8d23b2f485ed Running a gbm grid search (with cross validation) on a couple of datasets and hit a few NPEs and java assertion error due to leaked frames and missing chunks. Could not repro the errors when ran them again. here are the stack traces- {code:java} [1] "/home/nidhi/auto_sklearn_csv/dataset_22_mfeat-zernike.arff.txt" Hyper-parameter: col_sample_rate_per_tree, 0.6 Hyper-parameter: learn_rate, 0.5 Hyper-parameter: max_depth, 1 Hyper-parameter: min_rows, 100 Hyper-parameter: min_split_improvement, 1e-04 Hyper-parameter: ntrees, 10000 Hyper-parameter: sample_rate, 0.8 [2016-09-05 07:06:27] failure_details: DistributedException from /172.17.2.208:28555 [2016-09-05 07:06:27] failure_stack_traces: DistributedException from /172.17.2.208:28555, caused by java.lang.NullPointerException at water.MRTask.getResult(MRTask.java:477) at water.MRTask.getResult(MRTask.java:485) at water.MRTask.doAll(MRTask.java:389) at water.MRTask.doAll(MRTask.java:383) at hex.tree.gbm.GBM$GBMDriver.buildNextKTrees(GBM.java:444) at hex.tree.SharedTree$Driver.scoreAndBuildTrees(SharedTree.java:344) at hex.tree.SharedTree$Driver.computeImpl(SharedTree.java:284) at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:168) at water.H2O$H2OCountedCompleter.compute(H2O.java:1198) at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) Caused by: java.lang.NullPointerException at water.fvec.NewChunk.chunkD(NewChunk.java:1395) at water.fvec.NewChunk.compress2(NewChunk.java:1114) at water.fvec.NewChunk.compress(NewChunk.java:1013) at water.fvec.NewChunk.new_close(NewChunk.java:876) at water.fvec.Chunk.close(Chunk.java:561) at water.MRTask.compute2(MRTask.java:650) at water.H2O$H2OCountedCompleter.compute1(H2O.java:1201) at hex.tree.gbm.GBM$GBMDriver$ComputePredAndRes$Icer.compute1(GBM$GBMDriver$ComputePredAndRes$Icer.java) at water.H2O$H2OCountedCompleter.compute(H2O.java:1197) ... 5 more {code} {code:java} ##when rerun runs fine [1] "/home/nidhi/auto_sklearn_csv/oh10.wc.arff.txt" |======================================================================| 100% |====== | 8% java.lang.AssertionError: Coldata -2.0 out of range glycogen:0.0-6.0 step=1.0 nbins=6 isInt=1 cnt=0.000000, [0.000000 - 1.000000], mean/var= 0.00/ 0.00, cnt=0.000000, [1.000000 - 2.000000], mean/var= 0.00/ 0.00, cnt=0.000000, [2.000000 - 3.000000], mean/var= 0.00/ 0.00, cnt=0.000000, [3.000000 - 4.000000], mean/var= 0.00/ 0.00, cnt=0.000000, [4.000000 - 5.000000], mean/var= 0.00/ 0.00, cnt=0.000000, [5.000000 - 6.000000], mean/var= 0.00/ 0.00, Error: java.lang.AssertionError: Coldata -2.0 out of range glycogen:0.0-6.0 step=1.0 nbins=6 isInt=1 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at hex.tree.DHistogram.bin(DHistogram.java:172) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at hex.tree.DHistogram.updateSharedHistosAndReset(DHistogram.java:617) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at hex.tree.ScoreBuildHistogram.accum_all(ScoreBuildHistogram.java:256) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at hex.tree.ScoreBuildHistogram.map(ScoreBuildHistogram.java:123) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at water.MRTask.compute2(MRTask.java:630) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at water.H2O$H2OCountedCompleter.compute1(H2O.java:1201) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at hex.tree.ScoreBuildHistogram$Icer.compute1(ScoreBuildHistogram$Icer.java) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at water.H2O$H2OCountedCompleter.compute(H2O.java:1197) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-3-9 ERRR: Error: Missing chunk 9 for $04ff2e020080ffffffff$nfs://home/nidhi/auto_sklearn_csv/oh10.wc.arff.txt 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-3-21 ERRR: Error: Missing chunk 15 for $04ff2e020080ffffffff$nfs://home/nidhi/auto_sklearn_csv/oh10.wc.arff.txt ################ {code} {code:java} ##COULD NOT REPRO ON MASTER 18 fails check nohupR.out rollup error 08-31 03:59:44.789 172.16.2.188:23555 8819 FJ-2-171 INFO: {"_train":{"name":"dataset_16_mfeat-karhunen.arff.txt_model_8_cv_5_train","type":"Key"},"_valid":{"name":"dataset_16_mfeat-karhunen.arff.txt_model_8_cv_5_valid","type":"Key"},"_nfolds":0,"_keep_cross_validation_predictions":false,"_keep_cross_validation_fold_assignment":false,"_parallelize_cross_validation":true,"_auto_rebalance":true,"_seed":4563563636689,"_fold_assignment":"AUTO","_categorical_encoding":"AUTO","_distribution":"AUTO","_tweedie_power":1.5,"_quantile_alpha":0.5,"_huber_alpha":0.9,"_ignored_columns":[""],"_ignore_const_cols":true,"_weights_column":"__internal_cv_weights__","_offset_column":null,"_fold_column":null,"_is_cv_model":true,"_score_each_iteration":false,"_max_runtime_secs":0.0,"_stopping_rounds":2,"_stopping_metric":"AUTO","_stopping_tolerance":0.001,"_response_column":"class","_balance_classes":false,"_max_after_balance_size":5.0,"_class_sampling_factors":null,"_max_confusion_matrix_size":20,"_checkpoint":null,"_pretrained_autoencoder":null,"_ntrees":10000,"_max_depth":3,"_min_rows":1.0,"_nbins":20,"_nbins_cats":1024,"_min_split_improvement":1.0E-5,"_histogram_type":"AUTO","_r2_stopping":1.7976931348623157E308,"_nbins_top_level":1024,"_build_tree_one_node":false,"_score_tree_interval":20,"_initial_score_interval":4000,"_score_interval":4000,"_sample_rate":1.0,"_sample_rate_per_class":null,"_col_sample_rate_change_per_level":1.0,"_col_sample_rate_per_tree":1.0,"_learn_rate":0.01,"_learn_rate_annealing":1.0,"_col_sample_rate":1.0,"_max_abs_leafnode_pred":1.7976931348623157E308} 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: Grid search: model builder for parameters hex.tree.gbm.GBMModel$GBMParameters@127558f failed! Exception: java.lang.NullPointerExceptionjava.lang.NullPointerException 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.fvec.NewChunk.bufB(NewChunk.java:1455) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.fvec.NewChunk.compress2(NewChunk.java:1209) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.fvec.NewChunk.compress(NewChunk.java:1013) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.fvec.NewChunk.new_close(NewChunk.java:876) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.fvec.Chunk.close(Chunk.java:561) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.MRTask.compute2(MRTask.java:650) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.H2O$H2OCountedCompleter.compute1(H2O.java:1201) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at hex.tree.ScoreBuildHistogram$Icer.compute1(ScoreBuildHistogram$Icer.java) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.H2O$H2OCountedCompleter.compute(H2O.java:1197) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.ForkJoinPool$WorkQueue.pollAndExecAll(ForkJoinPool.java:914) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:979) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 INFO: Windowed averages (window size 2) of grid's best 3 logloss metrics: [0.20842527609394138, 0.18711026295815913, 0.16014838264852577] 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 INFO: Checking convergence with logloss metric: 0.20842527609394138 --> 0.16014838264852577 (still improving). 08-31 03:59:44.902 172.16.2.188:23555 8819 FJ-1-43 INFO: Creating 5 cross-validation splits with random number seed: 4563563636689 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: java.lang.AssertionError: null vec: $04ff1e000000ffffffff$nfs://home/nidhi/auto_sklearn_csv/dataset_16_mfeat-karhunen.arff.txt 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at water.fvec.Frame.<init>(Frame.java:113) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at water.fvec.Frame.<init>(Frame.java:83) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.ModelBuilder.cv_makeFramesAndBuilders(ModelBuilder.java:353) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.ModelBuilder.computeCrossValidation(ModelBuilder.java:262) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.ModelBuilder.trainModelNested(ModelBuilder.java:212) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.grid.GridSearch.startBuildModel(GridSearch.java:329) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.grid.GridSearch.buildModel(GridSearch.java:311) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.grid.GridSearch.gridSearch(GridSearch.java:215) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.grid.GridSearch.access$000(GridSearch.java:69) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.grid.GridSearch$1.compute2(GridSearch.java:136) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at water.H2O$H2OCountedCompleter.compute(H2O.java:1198) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 08-31 08:33:42.526 172.16.2.188:23555 8819 #47664-12 INFO: GET /3/Frames, parms: {} 08-31 08:33:42.606 172.16.2.188:23555 8819 #47664-12 WARN: Leaked frame: Frame data points to one or more deleted vecs. ####################### {code}

    JIRA | 3 months ago | Nidhi Mehta
    java.lang.NullPointerExceptionjava.lang.NullPointerException
  2. 0

    deserializing

    Oracle Community | 1 decade ago | 807554
    java.lang.NullPointerExceptionjava.lang.NullPointerException
  3. Speed up your debug routine!

    Automated exception search integrated into your IDE

  4. 0

    NullPointerException for logger in JDBCStore rev. 1.24

    Apache Bugzilla | 1 decade ago | tomekem
    java.lang.NullPointerExceptionjava.lang.NullPointerException

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.NullPointerExceptionjava.lang.NullPointerException

      No message provided

      at water.fvec.NewChunk.bufB()
    2. water.fvec
      Chunk.close
      1. water.fvec.NewChunk.bufB(NewChunk.java:1455)
      2. water.fvec.NewChunk.compress2(NewChunk.java:1209)
      3. water.fvec.NewChunk.compress(NewChunk.java:1013)
      4. water.fvec.NewChunk.new_close(NewChunk.java:876)
      5. water.fvec.Chunk.close(Chunk.java:561)
      5 frames
    3. water
      H2O$H2OCountedCompleter.compute1
      1. water.MRTask.compute2(MRTask.java:650)
      2. water.H2O$H2OCountedCompleter.compute1(H2O.java:1201)
      2 frames
    4. hex.tree
      ScoreBuildHistogram$Icer.compute1
      1. hex.tree.ScoreBuildHistogram$Icer.compute1(ScoreBuildHistogram$Icer.java)
      1 frame
    5. water
      H2O$H2OCountedCompleter.compute
      1. water.H2O$H2OCountedCompleter.compute(H2O.java:1197)
      1 frame
    6. jsr166y
      ForkJoinWorkerThread.run
      1. jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
      2. jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
      3. jsr166y.ForkJoinPool$WorkQueue.pollAndExecAll(ForkJoinPool.java:914)
      4. jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:979)
      5. jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
      6. jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
      6 frames