java.lang.AssertionError: null vec: $04ff1e000000ffffffff$nfs://home/nidhi/auto_sklearn_csv/dataset_16_mfeat-karhunen.arff.txt

JIRA | Nidhi Mehta | 3 months ago
  1. 0

    H2O Build git hash d651ad244a9510d286c6b6f0bc4a8d23b2f485ed Running a gbm grid search (with cross validation) on a couple of datasets and hit a few NPEs and java assertion error due to leaked frames and missing chunks. Could not repro the errors when ran them again. here are the stack traces- {code:java} [1] "/home/nidhi/auto_sklearn_csv/dataset_22_mfeat-zernike.arff.txt" Hyper-parameter: col_sample_rate_per_tree, 0.6 Hyper-parameter: learn_rate, 0.5 Hyper-parameter: max_depth, 1 Hyper-parameter: min_rows, 100 Hyper-parameter: min_split_improvement, 1e-04 Hyper-parameter: ntrees, 10000 Hyper-parameter: sample_rate, 0.8 [2016-09-05 07:06:27] failure_details: DistributedException from /172.17.2.208:28555 [2016-09-05 07:06:27] failure_stack_traces: DistributedException from /172.17.2.208:28555, caused by java.lang.NullPointerException at water.MRTask.getResult(MRTask.java:477) at water.MRTask.getResult(MRTask.java:485) at water.MRTask.doAll(MRTask.java:389) at water.MRTask.doAll(MRTask.java:383) at hex.tree.gbm.GBM$GBMDriver.buildNextKTrees(GBM.java:444) at hex.tree.SharedTree$Driver.scoreAndBuildTrees(SharedTree.java:344) at hex.tree.SharedTree$Driver.computeImpl(SharedTree.java:284) at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:168) at water.H2O$H2OCountedCompleter.compute(H2O.java:1198) at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) Caused by: java.lang.NullPointerException at water.fvec.NewChunk.chunkD(NewChunk.java:1395) at water.fvec.NewChunk.compress2(NewChunk.java:1114) at water.fvec.NewChunk.compress(NewChunk.java:1013) at water.fvec.NewChunk.new_close(NewChunk.java:876) at water.fvec.Chunk.close(Chunk.java:561) at water.MRTask.compute2(MRTask.java:650) at water.H2O$H2OCountedCompleter.compute1(H2O.java:1201) at hex.tree.gbm.GBM$GBMDriver$ComputePredAndRes$Icer.compute1(GBM$GBMDriver$ComputePredAndRes$Icer.java) at water.H2O$H2OCountedCompleter.compute(H2O.java:1197) ... 5 more {code} {code:java} ##when rerun runs fine [1] "/home/nidhi/auto_sklearn_csv/oh10.wc.arff.txt" |======================================================================| 100% |====== | 8% java.lang.AssertionError: Coldata -2.0 out of range glycogen:0.0-6.0 step=1.0 nbins=6 isInt=1 cnt=0.000000, [0.000000 - 1.000000], mean/var= 0.00/ 0.00, cnt=0.000000, [1.000000 - 2.000000], mean/var= 0.00/ 0.00, cnt=0.000000, [2.000000 - 3.000000], mean/var= 0.00/ 0.00, cnt=0.000000, [3.000000 - 4.000000], mean/var= 0.00/ 0.00, cnt=0.000000, [4.000000 - 5.000000], mean/var= 0.00/ 0.00, cnt=0.000000, [5.000000 - 6.000000], mean/var= 0.00/ 0.00, Error: java.lang.AssertionError: Coldata -2.0 out of range glycogen:0.0-6.0 step=1.0 nbins=6 isInt=1 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at hex.tree.DHistogram.bin(DHistogram.java:172) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at hex.tree.DHistogram.updateSharedHistosAndReset(DHistogram.java:617) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at hex.tree.ScoreBuildHistogram.accum_all(ScoreBuildHistogram.java:256) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at hex.tree.ScoreBuildHistogram.map(ScoreBuildHistogram.java:123) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at water.MRTask.compute2(MRTask.java:630) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at water.H2O$H2OCountedCompleter.compute1(H2O.java:1201) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at hex.tree.ScoreBuildHistogram$Icer.compute1(ScoreBuildHistogram$Icer.java) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at water.H2O$H2OCountedCompleter.compute(H2O.java:1197) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-1-57 ERRR: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-3-9 ERRR: Error: Missing chunk 9 for $04ff2e020080ffffffff$nfs://home/nidhi/auto_sklearn_csv/oh10.wc.arff.txt 09-01 11:39:15.863 172.16.2.189:43555 22281 FJ-3-21 ERRR: Error: Missing chunk 15 for $04ff2e020080ffffffff$nfs://home/nidhi/auto_sklearn_csv/oh10.wc.arff.txt ################ {code} {code:java} ##COULD NOT REPRO ON MASTER 18 fails check nohupR.out rollup error 08-31 03:59:44.789 172.16.2.188:23555 8819 FJ-2-171 INFO: {"_train":{"name":"dataset_16_mfeat-karhunen.arff.txt_model_8_cv_5_train","type":"Key"},"_valid":{"name":"dataset_16_mfeat-karhunen.arff.txt_model_8_cv_5_valid","type":"Key"},"_nfolds":0,"_keep_cross_validation_predictions":false,"_keep_cross_validation_fold_assignment":false,"_parallelize_cross_validation":true,"_auto_rebalance":true,"_seed":4563563636689,"_fold_assignment":"AUTO","_categorical_encoding":"AUTO","_distribution":"AUTO","_tweedie_power":1.5,"_quantile_alpha":0.5,"_huber_alpha":0.9,"_ignored_columns":[""],"_ignore_const_cols":true,"_weights_column":"__internal_cv_weights__","_offset_column":null,"_fold_column":null,"_is_cv_model":true,"_score_each_iteration":false,"_max_runtime_secs":0.0,"_stopping_rounds":2,"_stopping_metric":"AUTO","_stopping_tolerance":0.001,"_response_column":"class","_balance_classes":false,"_max_after_balance_size":5.0,"_class_sampling_factors":null,"_max_confusion_matrix_size":20,"_checkpoint":null,"_pretrained_autoencoder":null,"_ntrees":10000,"_max_depth":3,"_min_rows":1.0,"_nbins":20,"_nbins_cats":1024,"_min_split_improvement":1.0E-5,"_histogram_type":"AUTO","_r2_stopping":1.7976931348623157E308,"_nbins_top_level":1024,"_build_tree_one_node":false,"_score_tree_interval":20,"_initial_score_interval":4000,"_score_interval":4000,"_sample_rate":1.0,"_sample_rate_per_class":null,"_col_sample_rate_change_per_level":1.0,"_col_sample_rate_per_tree":1.0,"_learn_rate":0.01,"_learn_rate_annealing":1.0,"_col_sample_rate":1.0,"_max_abs_leafnode_pred":1.7976931348623157E308} 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: Grid search: model builder for parameters hex.tree.gbm.GBMModel$GBMParameters@127558f failed! Exception: java.lang.NullPointerExceptionjava.lang.NullPointerException 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.fvec.NewChunk.bufB(NewChunk.java:1455) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.fvec.NewChunk.compress2(NewChunk.java:1209) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.fvec.NewChunk.compress(NewChunk.java:1013) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.fvec.NewChunk.new_close(NewChunk.java:876) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.fvec.Chunk.close(Chunk.java:561) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.MRTask.compute2(MRTask.java:650) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.H2O$H2OCountedCompleter.compute1(H2O.java:1201) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at hex.tree.ScoreBuildHistogram$Icer.compute1(ScoreBuildHistogram$Icer.java) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at water.H2O$H2OCountedCompleter.compute(H2O.java:1197) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.ForkJoinPool$WorkQueue.pollAndExecAll(ForkJoinPool.java:914) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:979) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 WARN: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 INFO: Windowed averages (window size 2) of grid's best 3 logloss metrics: [0.20842527609394138, 0.18711026295815913, 0.16014838264852577] 08-31 03:59:44.858 172.16.2.188:23555 8819 FJ-1-43 INFO: Checking convergence with logloss metric: 0.20842527609394138 --> 0.16014838264852577 (still improving). 08-31 03:59:44.902 172.16.2.188:23555 8819 FJ-1-43 INFO: Creating 5 cross-validation splits with random number seed: 4563563636689 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: java.lang.AssertionError: null vec: $04ff1e000000ffffffff$nfs://home/nidhi/auto_sklearn_csv/dataset_16_mfeat-karhunen.arff.txt 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at water.fvec.Frame.<init>(Frame.java:113) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at water.fvec.Frame.<init>(Frame.java:83) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.ModelBuilder.cv_makeFramesAndBuilders(ModelBuilder.java:353) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.ModelBuilder.computeCrossValidation(ModelBuilder.java:262) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.ModelBuilder.trainModelNested(ModelBuilder.java:212) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.grid.GridSearch.startBuildModel(GridSearch.java:329) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.grid.GridSearch.buildModel(GridSearch.java:311) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.grid.GridSearch.gridSearch(GridSearch.java:215) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.grid.GridSearch.access$000(GridSearch.java:69) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at hex.grid.GridSearch$1.compute2(GridSearch.java:136) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at water.H2O$H2OCountedCompleter.compute(H2O.java:1198) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 08-31 03:59:44.910 172.16.2.188:23555 8819 FJ-1-43 ERRR: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 08-31 08:33:42.526 172.16.2.188:23555 8819 #47664-12 INFO: GET /3/Frames, parms: {} 08-31 08:33:42.606 172.16.2.188:23555 8819 #47664-12 WARN: Leaked frame: Frame data points to one or more deleted vecs. ####################### {code}

    JIRA | 3 months ago | Nidhi Mehta
    java.lang.AssertionError: null vec: $04ff1e000000ffffffff$nfs://home/nidhi/auto_sklearn_csv/dataset_16_mfeat-karhunen.arff.txt
  2. 0

    http://mr-0xb1:8080/job/h2o_master_DEV_pyunit_small/14549/artifact/h2o-py/tests/results/java_1_0.out.txt starts here 06-14 12:32:44.224 172.16.2.172:56791 15740 # Session INFO: ------------------------------------------------------------ 06-14 12:32:44.226 172.16.2.172:56791 15740 # Session INFO: 06-14 12:32:44.304 172.16.2.172:56791 15740 # Session INFO: STARTING TEST: pyunit_citi_bike_small.py 06-14 12:32:44.309 172.16.2.172:56791 15740 # Session INFO: 06-14 12:32:44.314 172.16.2.172:56791 15740 # Session INFO: ------------------------------------------------------------ .... 06-14 12:34:04.563 172.16.2.172:56791 15740 # Session INFO: Method: POST , URI: /99/Rapids, route: /99/Rapids, parms: {ast=(, (gput py51ed09ad-ed04-42cc-ae0e-49f1514c26bf (cbind %FALSE '$04ff01000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/31081_New_York_City__Hourly_2013.csv' '$04ff02000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/31081_New_York_City__Hourly_2013.csv' '$04ff03000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/31081_New_York_City__Hourly_2013.csv' '$04ff04000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/31081_New_York_City__Hourly_2013.csv' '$04ff18000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/31081_New_York_City__Hourly_2013.csv' '$04ff19000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/31081_New_York_City__Hourly_2013.csv' '$04ff1a000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/31081_New_York_City__Hourly_2013.csv' '$04ff1f000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/31081_New_York_City__Hourly_2013.csv' '$04ff22000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/31081_New_York_City__Hourly_2013.csv')) (colnames= %py51ed09ad-ed04-42cc-ae0e-49f1514c26bf (: #0 #8) (slist "Year Local" "Month Local" "Day Local" "Hour Local" "Dew Point (C)" "Humidity Fraction" "Rain (mm)" "Temperature (C)" "WC1")} 06-14 12:34:04.569 172.16.2.172:56791 15740 # Session INFO: Method: DELETE, URI: /3/DKV/py51ed09ad-ed04-42cc-ae0e-49f1514c26bf, route: /3/DKV/(?<key>.*), parms: {key=py51ed09ad-ed04-42cc-ae0e-49f1514c26bf} 06-14 12:34:04.573 172.16.2.172:56791 15740 # Session INFO: Method: POST , URI: /99/Rapids, route: /99/Rapids, parms: {ast=(, (gput py0cae986a-6c47-4208-90fa-7686a4b0d3e6 (cbind %FALSE 'pyd4365f80-c003-4de7-b98a-c3cd0470df3f' 'py86ae3907-f0fe-41cc-9717-d73851cbc226' 'py55bc4ed7-3814-4f2e-9cdc-177f4669dcf6' 'pyda6e981a-e1b9-4641-9614-25a61dec48ca' 'pyc9b01cbc-d120-41f5-beef-76d6597f1458' 'pyb72a7fa5-4939-4c48-ac55-1470b7c9db1a' 'py2283839f-2d05-4a84-bca8-7e2ab7cba326' 'py064e4f5e-a975-4f1a-b460-f4d990721915' 'pyfb56a793-5deb-4ec2-a64e-702e5385b338' 'pyb8e21cef-cdc9-4427-b6c9-b05b11d5a310')) (colnames= %py0cae986a-6c47-4208-90fa-7686a4b0d3e6 (: #0 #9) (slist "Year Local" "Month Local" "Day Local" "Hour Local" "Dew Point (C)" "Humidity Fraction" "Temperature (C)" "WC1" "msec" "Days")} 06-14 12:34:04.574 172.16.2.172:56791 15740 # Session ERRR: java.lang.AssertionError java.lang.AssertionError at water.fvec.Frame.<init>(Frame.java:102) at water.fvec.Frame.<init>(Frame.java:72) at water.fvec.Frame.<init>(Frame.java:70) at water.fvec.Frame.add(Frame.java:484) at water.rapids.ASTCbind.apply(ASTOp.java:2308) at water.rapids.AST.treeWalk(AST.java:49) at water.rapids.AST.treeWalk(AST.java:48) at water.rapids.ASTStatement.exec(AST.java:474) at water.rapids.AST.treeWalk(AST.java:112) at water.rapids.Exec.exec(Exec.java:68) at water.api.RapidsHandler.exec(RapidsHandler.java:37) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at water.api.Handler.handle(Handler.java:56) at water.api.RequestServer.handle(RequestServer.java:677) at water.api.RequestServer.serve(RequestServer.java:614) at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:438) at java.lang.Thread.run(Thread.java:745) 06-14 12:34:04.575 172.16.2.172:56791 15740 # Session WARN: Caught exception: java.lang.AssertionError; Stacktrace: [water.api.RapidsHandler.exec(RapidsHandler.java:132), sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source), sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), java.lang.reflect.Method.invoke(Method.java:606), water.api.Handler.handle(Handler.java:56), water.api.RequestServer.handle(RequestServer.java:677), water.api.RequestServer.serve(RequestServer.java:614), water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:438), java.lang.Thread.run(Thread.java:745)] 06-14 12:34:04.578 172.16.2.172:56791 15740 # Session INFO: Method: DELETE, URI: /3/DKV/DRFModel__a5596556bd859d81c1512f7527e24a95, route: /3/DKV/(?<key>.*), parms: {key=DRFModel__a5596556bd859d81c1512f7527e24a95} 06-14 12:34:04.583 172.16.2.172:56791 15740 # Session INFO: Method: POST , URI: /99/Rapids, route: /99/Rapids, parms: {ast=(, (gput pyabf1e289-7f0f-452e-a275-df5de30b0937 (cbind %FALSE '$04ff52000000fffffffffd6d436533f737a9554e05386860cd0d$' '$04ff53000000fffffffffd6d436533f737a9554e05386860cd0d$' '$04ff54000000fffffffffd6d436533f737a9554e05386860cd0d$' '$04ff55000000fffffffffd6d436533f737a9554e05386860cd0d$' '$04ff56000000fffffffffd6d436533f737a9554e05386860cd0d$' '$04ff57000000fffffffffd6d436533f737a9554e05386860cd0d$' '$04ff58000000fffffffffd6d436533f737a9554e05386860cd0d$' '$04ff59000000fffffffffd6d436533f737a9554e05386860cd0d$' '$04ff5a000000fffffffffd6d436533f737a9554e05386860cd0d$' '$04ff5b000000fffffffffd6d436533f737a9554e05386860cd0d$')) (colnames= %pyabf1e289-7f0f-452e-a275-df5de30b0937 (: #0 #9) (slist "Days" "start station name" "bikes" "Month" "DayOfWeek" "Humidity Fraction" "Rain (mm)" "Temperature (C)" "WC1" "Dew Point (C)")} 06-14 12:34:04.589 172.16.2.172:56791 15740 # Session INFO: Method: DELETE, URI: /3/DKV/pyabf1e289-7f0f-452e-a275-df5de30b0937, route: /3/DKV/(?<key>.*), parms: {key=pyabf1e289-7f0f-452e-a275-df5de30b0937} 06-14 12:34:04.592 172.16.2.172:56791 15740 # Session INFO: Method: DELETE, URI: /3/DKV/pye719600b-37f4-4237-936d-106cbfc024ec, route: /3/DKV/(?<key>.*), parms: {key=pye719600b-37f4-4237-936d-106cbfc024ec} 06-14 12:34:04.595 172.16.2.172:56791 15740 # Session INFO: Method: DELETE, URI: /3/DKV/pye719600b-37f4-4237-936d-106cbfc024ec, route: /3/DKV/(?<key>.*), parms: {key=pye719600b-37f4-4237-936d-106cbfc024ec} 06-14 12:34:04.598 172.16.2.172:56791 15740 # Session INFO: Method: POST , URI: /99/Rapids, route: /99/Rapids, parms: {ast=(, (gput py7110bc7b-ba69-4d6d-b124-c390ac0370fa (cbind %FALSE '$04ff01000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff02000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff03000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff04000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff05000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff06000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff07000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff08000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff09000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff0a000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff0b000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff0c000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff0d000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff0e000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' '$04ff0f000000ffffffff$nfs://home2/jenkins/slave_dir_from_mr-0xb1/workspace/h2o_master_DEV_pyunit_small/bigdata/laptop/citibike-nyc/2013-10.csv' 'py1014c88c-3e98-4405-ba47-c335b652bdf5')) (colnames= %py7110bc7b-ba69-4d6d-b124-c390ac0370fa (: #0 #15) (slist "tripduration" "starttime" "stoptime" "start station id" "start station name" "start station latitude" "start station longitude" "end station id" "end station name" "end station latitude" "end station longitude" "bikeid" "usertype" "birth year" "gender" "Days")} 06-14 12:34:04.607 172.16.2.172:56791 15740 # Session INFO: Method: DELETE, URI: /3/DKV/py7110bc7b-ba69-4d6d-b124-c390ac0370fa, route: /3/DKV/(?<key>.*), parms: {key=py7110bc7b-ba69-4d6d-b124-c390ac0370fa} 06-14 12:34:04.612 172.16.2.172:56791 15740 # Session INFO: Method: POST , URI: /99/Rapids, route: /99/Rapids, parms: {ast=(, (gput pyf77d2c7b-9ff0-46c4-a6f9-7f0f7b057a8b (cbind %FALSE '$04ff07000000fffffffffd6d436533f737a9554e05386860cd0d$' '$04ff08000000fffffffffd6d436533f737a9554e05386860cd0d$' '$04ff09000000fffffffffd6d436533f737a9554e05386860cd0d$' 'py8311c266-04e7-48c0-b094-f133b871e4f6' 'pya946a1b3-33ee-4c58-9dfe-8dd18d6c84f2')) (colnames= %pyf77d2c7b-9ff0-46c4-a6f9-7f0f7b057a8b (: #0 #4) (slist "Days" "start station name" "bikes" "Month" "DayOfWeek")} 06-14 12:34:04.612 172.16.2.172:56791 15740 # Session ERRR: java.lang.AssertionError java.lang.AssertionError at water.fvec.Frame.<init>(Frame.java:102) at water.fvec.Frame.<init>(Frame.java:72) at water.rapids.ASTCbind.apply(ASTOp.java:2301) at water.rapids.AST.treeWalk(AST.java:49) at water.rapids.AST.treeWalk(AST.java:48) at water.rapids.ASTStatement.exec(AST.java:474) at water.rapids.AST.treeWalk(AST.java:112) at water.rapids.Exec.exec(Exec.java:68) at water.api.RapidsHandler.exec(RapidsHandler.java:37) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at water.api.Handler.handle(Handler.java:56) at water.api.RequestServer.handle(RequestServer.java:677) at water.api.RequestServer.serve(RequestServer.java:614) at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:438) at java.lang.Thread.run(Thread.java:745) 06-14 12:34:04.613 172.16.2.172:56791 15740 # Session WARN: Caught exception: java.lang.AssertionError; Stacktrace: [water.api.RapidsHandler.exec(RapidsHandler.java:132), sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source), sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), java.lang.reflect.Method.invoke(Method.java:606), water.api.Handler.handle(Handler.java:56), water.api.RequestServer.handle(RequestServer.java:677), water.api.RequestServer.serve(RequestServer.java:614), water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:438), java.lang.Thread.run(Thread.java:745)] 06-14 12:34:04.616 172.16.2.172:56791 15740 # Session INFO: Method: DELETE, URI: /3/DKV/py33a32efb-4096-46f8-934d-9fb5df9f54df, route: /3/DKV/(?<key>.*), parms: {key=py33a32efb-4096-46f8-934d-9fb5df9f54df} 06-14 12:34:04.619 172.16.2.172:56791 15740 # Session INFO: Method: DELETE, URI: /3/DKV/DeepLearningModel__b767c13f6646ac7d5ee6953890214668, route: /3/DKV/(?<key>.*), parms: {key=DeepLearningModel__b767c13f6646ac7d5ee6953890214668}

    JIRA | 1 year ago | Kevin Normoyle
    java.lang.AssertionError
  3. Speed up your debug routine!

    Automated exception search integrated into your IDE

  4. 0

    parse iris, do some funs and apply This is an old test that was working in the Dec/Jan time frame 03-10 22:43:48.275 192.168.0.34:54333 419 # Session ERRR: java.lang.AssertionError java.lang.AssertionError at water.fvec.Frame.<init>(Frame.java:100) at water.fvec.Frame.<init>(Frame.java:72) at water.rapids.ASTApply.apply(ASTApply.java:63) at water.rapids.AST.treeWalk(AST.java:50) at water.rapids.Exec.exec(Exec.java:74) at water.api.RapidsHandler.exec(RapidsHandler.java:36) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at water.api.Handler.handle(Handler.java:57) at water.api.RequestServer.handle(RequestServer.java:629) at water.api.RequestServer.serve(RequestServer.java:587) at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:434) at java.lang.Thread.run(Thread.java:745) I put a long timeout on the Rapids request (25 secs) ...so it will look like it hangs that it scrapes the log for the Assertion error below, so you'll see that as the cause for the timeout (side note: the assertion error should cause the rapids json to complete with the stack trace, ideally..apparently doesn't) to run cd h2o-dev/py2/testdir_rapids python test_rapids_basic_with_funs_pick5.py you can use -uc to connect to existing localhost:54321 h2o.jar fails on this funs and apply 2015-03-10 22:38:28.525523 -- Start http://192.168.0.34:54333/1/Rapids.json # {'funs': '[(def anon {x} ((= !v (c {#1;#4567;(: #9 #90);(: #9 #45);#450});;(= !v2 %v );;(= !v2 (n %v %v));;(= !v2 (N %v %v));;(= !v2 (- %v %v));;(= !v2 (+ %v %v)));;;)]', 'ast': None}; 2015-03-10 22:38:28.530527 -- Start http://192.168.0.34:54333/1/Rapids.json # {'funs': None, 'ast': '(apply %r1 #2 %anon)'}; while it might be a little difficult to see how I'm using the list in "pick5" groups it looks like there's no random element..i.e. it should fail the same way each time..so I don't need to provide a seed here

    JIRA | 2 years ago | Kevin Normoyle
    java.lang.AssertionError

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.AssertionError

      null vec: $04ff1e000000ffffffff$nfs://home/nidhi/auto_sklearn_csv/dataset_16_mfeat-karhunen.arff.txt

      at water.fvec.Frame.<init>()
    2. water.fvec
      Frame.<init>
      1. water.fvec.Frame.<init>(Frame.java:113)
      2. water.fvec.Frame.<init>(Frame.java:83)
      2 frames
    3. hex
      ModelBuilder.trainModelNested
      1. hex.ModelBuilder.cv_makeFramesAndBuilders(ModelBuilder.java:353)
      2. hex.ModelBuilder.computeCrossValidation(ModelBuilder.java:262)
      3. hex.ModelBuilder.trainModelNested(ModelBuilder.java:212)
      3 frames
    4. hex.grid
      GridSearch$1.compute2
      1. hex.grid.GridSearch.startBuildModel(GridSearch.java:329)
      2. hex.grid.GridSearch.buildModel(GridSearch.java:311)
      3. hex.grid.GridSearch.gridSearch(GridSearch.java:215)
      4. hex.grid.GridSearch.access$000(GridSearch.java:69)
      5. hex.grid.GridSearch$1.compute2(GridSearch.java:136)
      5 frames
    5. water
      H2O$H2OCountedCompleter.compute
      1. water.H2O$H2OCountedCompleter.compute(H2O.java:1198)
      1 frame
    6. jsr166y
      ForkJoinWorkerThread.run
      1. jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
      2. jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
      3. jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
      4. jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
      5. jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
      5 frames