java.lang.AssertionError: col_data Infinity out of range FirstName :-Infinity-Infinity step=Infinity nbins=20 isInt=0 + cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, +

JIRA | Kevin Normoyle | 3 years ago
  1. 0

    What I've found, is that there are 3 flavors of every test that needs to be run full va va with autoframe full fvec A test ported to any of the three, can find unique problems in each I started porting more of the VA parse tests to fvec here's one that fails it creates a bunch of synthetic datasets for parsing and does a short DRF to make sure they parsed okay. The dataset will be in sandbox/syn_datasets when it fails, so you can probably run the test and get the failing dataset then play with it by itself in this case it's parsetmp_1_0_2_0.data the test passed in it's VA incarnation. the port is maintly parameter names and json response differences cd testdir_single_jvm python test_parse_many_cases.fvec.py test_parse_many_cases_fvec.Basic.test_A_many_parse1 (from nosetests) Failing for the past 26 builds (Since Unstable#751 ) Took 56 sec. Error Message check_sandbox_for_errors: Errors in sandbox stdout or stderr (or R stdout/stderr). Could have occurred at any prior time java.lang.AssertionError: col_data Infinity out of range FirstName :-Infinity-Infinity step=Infinity nbins=20 isInt=0 cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, 01:21:07.012 FJ-1-3 INFO WATER: at hex.gbm.DHistogram.incr(DHistogram.java:137) 01:21:07.013 FJ-1-3 INFO WATER: at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.accum_subset(SharedTreeModelBuilder.java:421) 01:21:07.014 FJ-1-3 INFO WATER: at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.map(SharedTreeModelBuilder.java:355) 01:21:07.018 FJ-1-3 INFO WATER: at water.MRTask2.compute2(MRTask2.java:378) 01:21:07.019 FJ-1-3 INFO WATER: at water.H2O$H2OCountedCompleter.compute(H2O.java:731) 01:21:07.030 FJ-1-3 INFO WATER: at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) 01:21:07.031 FJ-1-3 INFO WATER: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 01:21:07.033 FJ-1-3 INFO WATER: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) 01:21:07.034 FJ-1-3 INFO WATER: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 01:21:07.036 FJ-1-3 INFO WATER: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 01:21:07.014 FJ-0-1 ERRR WATER: + java.lang.AssertionError: col_data Infinity out of range FirstName :-Infinity-Infinity step=Infinity nbins=20 isInt=0 + cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + + at hex.gbm.DHistogram.incr(DHistogram.java:137) + at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.accum_subset(SharedTreeModelBuilder.java:421) + at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.map(SharedTreeModelBuilder.java:355) + at water.MRTask2.compute2(MRTask2.java:378) + at water.H2O$H2OCountedCompleter.compute(H2O.java:731) + at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) + at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) + at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) + at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) + at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) it had started doing a DRF with a particular dataset 2/DRF result: { "Request2": 0, "_distribution": null, "_mtry": 1, "_seed": 3757839644659944300, "classification": "true", "description": "Distributed RF", "destination_key": "DRF_a9ea1dcd36b98a92dde7b962621a4a93", "end_time": 0, "exception": null, "job_key": "$03010a89477532d4ffffffff$_971067754ca9e8c99d7d4f867b574499", "max_depth": 20, "min_rows": 1, "nbins": 20, "ntrees": 1, "response": {}, "response_info": { "h2o": "pytest-jenkins-17647", "node": "/10.137.71.117:54321", "redirect_url": "/2/DRFProgressPage.json?job_key=%2403010a89477532d4ffffffff%24_971067754ca9e8c99d7d4f867b574499&destination_key=DRF_a9ea1dcd36b98a92dde7b962621a4a93", "status": "redirect", "time": 11 }, "score_each_iteration": "false", "source": { "_key": "parsetmp_1_0_2_0.hex" }, "start_time": 1398586866983, "state": "RUNNING", "validation": null } redirect http://10.137.71.117:54321/2/DRFProgressPage.json?job_key=%2403010a89477532d4ffffffff%24_971067754ca9e8c99d7d4f867b574499&destination_key=DRF_a9ea1dcd36b98a92dde7b962621a4a93

    JIRA | 3 years ago | Kevin Normoyle
    java.lang.AssertionError: col_data Infinity out of range FirstName :-Infinity-Infinity step=Infinity nbins=20 isInt=0 + cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, +
  2. 0

    What I've found, is that there are 3 flavors of every test that needs to be run full va va with autoframe full fvec A test ported to any of the three, can find unique problems in each I started porting more of the VA parse tests to fvec here's one that fails it creates a bunch of synthetic datasets for parsing and does a short DRF to make sure they parsed okay. The dataset will be in sandbox/syn_datasets when it fails, so you can probably run the test and get the failing dataset then play with it by itself in this case it's parsetmp_1_0_2_0.data the test passed in it's VA incarnation. the port is maintly parameter names and json response differences cd testdir_single_jvm python test_parse_many_cases.fvec.py test_parse_many_cases_fvec.Basic.test_A_many_parse1 (from nosetests) Failing for the past 26 builds (Since Unstable#751 ) Took 56 sec. Error Message check_sandbox_for_errors: Errors in sandbox stdout or stderr (or R stdout/stderr). Could have occurred at any prior time java.lang.AssertionError: col_data Infinity out of range FirstName :-Infinity-Infinity step=Infinity nbins=20 isInt=0 cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, 01:21:07.012 FJ-1-3 INFO WATER: at hex.gbm.DHistogram.incr(DHistogram.java:137) 01:21:07.013 FJ-1-3 INFO WATER: at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.accum_subset(SharedTreeModelBuilder.java:421) 01:21:07.014 FJ-1-3 INFO WATER: at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.map(SharedTreeModelBuilder.java:355) 01:21:07.018 FJ-1-3 INFO WATER: at water.MRTask2.compute2(MRTask2.java:378) 01:21:07.019 FJ-1-3 INFO WATER: at water.H2O$H2OCountedCompleter.compute(H2O.java:731) 01:21:07.030 FJ-1-3 INFO WATER: at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) 01:21:07.031 FJ-1-3 INFO WATER: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 01:21:07.033 FJ-1-3 INFO WATER: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) 01:21:07.034 FJ-1-3 INFO WATER: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 01:21:07.036 FJ-1-3 INFO WATER: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 01:21:07.014 FJ-0-1 ERRR WATER: + java.lang.AssertionError: col_data Infinity out of range FirstName :-Infinity-Infinity step=Infinity nbins=20 isInt=0 + cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + + at hex.gbm.DHistogram.incr(DHistogram.java:137) + at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.accum_subset(SharedTreeModelBuilder.java:421) + at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.map(SharedTreeModelBuilder.java:355) + at water.MRTask2.compute2(MRTask2.java:378) + at water.H2O$H2OCountedCompleter.compute(H2O.java:731) + at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) + at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) + at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) + at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) + at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) it had started doing a DRF with a particular dataset 2/DRF result: { "Request2": 0, "_distribution": null, "_mtry": 1, "_seed": 3757839644659944300, "classification": "true", "description": "Distributed RF", "destination_key": "DRF_a9ea1dcd36b98a92dde7b962621a4a93", "end_time": 0, "exception": null, "job_key": "$03010a89477532d4ffffffff$_971067754ca9e8c99d7d4f867b574499", "max_depth": 20, "min_rows": 1, "nbins": 20, "ntrees": 1, "response": {}, "response_info": { "h2o": "pytest-jenkins-17647", "node": "/10.137.71.117:54321", "redirect_url": "/2/DRFProgressPage.json?job_key=%2403010a89477532d4ffffffff%24_971067754ca9e8c99d7d4f867b574499&destination_key=DRF_a9ea1dcd36b98a92dde7b962621a4a93", "status": "redirect", "time": 11 }, "score_each_iteration": "false", "source": { "_key": "parsetmp_1_0_2_0.hex" }, "start_time": 1398586866983, "state": "RUNNING", "validation": null } redirect http://10.137.71.117:54321/2/DRFProgressPage.json?job_key=%2403010a89477532d4ffffffff%24_971067754ca9e8c99d7d4f867b574499&destination_key=DRF_a9ea1dcd36b98a92dde7b962621a4a93

    JIRA | 3 years ago | Kevin Normoyle
    java.lang.AssertionError: col_data Infinity out of range FirstName :-Infinity-Infinity step=Infinity nbins=20 isInt=0 cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00,
  3. 0

    there's another jira probably somewhere. I don't know if it's possible for this to show up in other datasets. has importance=1 and just 1 tree. don't know if that matters it might be the NaN in the dataset here's the drf: /2/DRF.json?classification=1&importance=1&source=parsetmp_1_0_2_0.hex&ntrees=1&response=DateofBirth FJ-0-3 ERRR WATER: java.lang.AssertionError: col_data Infinity out of range FirstName:-Infinity-Infinity step=Infinity nbins=1024 isInt=0 cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, .... at hex.gbm.DHistogram.incr(DHistogram.java:137) at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.accum_subset(SharedTreeModelBuilder.java:518) at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.map(SharedTreeModelBuilder.java:452) at water.MRTask2.compute2(MRTask2.java:391) at water.H2O$H2OCountedCompleter.compute(H2O.java:711) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) it's pretty simple. hive separator in this case, but not sure if that matters I attached the file here: FirstName^AMiddleInitials^ALastName^ADateofBirth 0^A0.5^A1^A0 3^ANaN^A4^A1 6^A^A8^A0 0.6^A0.7^A0.8^A1 +0.6^A+0.7^A+0.8^A0 -0.6^A-0.7^A-0.8^A1 .6^A.7^A.8^A0 +.6^A+.7^A+.8^A1 -.6^A-.7^A-.8^A0 +0.6e0^A+0.7e0^A+0.8e0^A1 -0.6e0^A-0.7e0^A-0.8e0^A0 .6e0^A.7e0^A.8e0^A1 +.6e0^A+.7e0^A+.8e0^A0 -.6e0^A-.7e0^A-.8e0^A1 +0.6e00^A+0.7e00^A+0.8e00^A0 -0.6e00^A-0.7e00^A-0.8e00^A1 .6e00^A.7e00^A.8e00^A0 +.6e00^A+.7e00^A+.8e00^A1 -.6e00^A-.7e00^A-.8e00^A0 +0.6e-01^A+0.7e-01^A+0.8e-01^A1 -0.6e-01^A-0.7e-01^A-0.8e-01^A0 .6e-01^A.7e-01^A.8e-01^A1 +.6e-01^A+.7e-01^A+.8e-01^A0 -.6e-01^A-.7e-01^A-.8e-01^A1 +0.6e+01^A+0.7e+01^A+0.8e+01^A0 -0.6e+01^A-0.7e+01^A-0.8e+01^A1 .6e+01^A.7e+01^A.8e+01^A0 +.6e+01^A+.7e+01^A+.8e+01^A1 -.6e+01^A-.7e+01^A-.8e+01^A0 +0.6e102^A+0.7e102^A+0.8e102^A1 -0.6e102^A-0.7e102^A-0.8e102^A0 .6e102^A.7e102^A.8e102^A1 +.6e102^A+.7e102^A+.8e102^A0 -.6e102^A-.7e102^A-.8e102^A1 2014-06-30 15:44:44.245461 -- Start http://192.168.1.40:54321/2/PostFile.json?key=parsetmp_1_0_2_0.data #/home/kevin/h2o/py/testdir_single_jvm/sandbox/syn_datasets/parsetmp_1_0_2_0.data 2014-06-30 15:44:44.253077 -- Start http://192.168.1.40:54321/2/Parse2.json?single_quotes=0&source_key=parsetmp_1_0_2_0.data 2014-06-30 15:44:45.260299 -- Start http://192.168.1.40:54321/2/Progress2.json?job_key=$0301c0a8012832d4ffffffff$_a57367bdc85fd058b738e6d9faae422f&destination_key=parsetmp_1_0_2_0.hex 2014-06-30 15:44:45.274134 -- Start http://192.168.1.40:54321/2/Inspect2.json?src_key=parsetmp_1_0_2_0.hex 2014-06-30 15:44:45.285336 -- Start http://192.168.1.40:54321/2/SummaryPage2.json?source=parsetmp_1_0_2_0.hex&max_ncols=4 2014-06-30 15:44:45.378326 -- Start http://192.168.1.40:54321/2/Inspect2.json?src_key=parsetmp_1_0_2_0.hex 2014-06-30 15:44:45.382202 -- Start http://192.168.1.40:54321/2/DRF.json?classification=1&importance=1&source=parsetmp_1_0_2_0.hex&ntrees=1&response=DateofBirth 2014-06-30 15:44:50.501616 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff 2014-06-30 15:44:50.610065 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff 2014-06-30 15:44:50.713537 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff 2014-06-30 15:44:50.817189 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff 2014-06-30 15:44:50.920776 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff 2014-06-30 15:44:51.029056 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff

    JIRA | 2 years ago | Kevin Normoyle
    java.lang.AssertionError: col_data Infinity out of range FirstName:-Infinity-Infinity step=Infinity nbins=1024 isInt=0 cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, ....
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Input: - 64 machines, jordan release - airlines allyears1987-2007.csv from S3N via importfiles1 - drf2 50 trees, depth 60 Output: during generation of first tree DRF failed on : ERRR WATER: + java.lang.AssertionError: col_data 0.0 out of range CRSArrTime:1.0-2401.0 step=119.99999 nbins=20 isInt=1 + cnt=88, [13.000000 - 2400.000000], mean/var= 0.11/ 0.10, + cnt=0, [13.000000 - 2400.000000], mean/var= 0.00/ 0.00, + cnt=0, [13.000000 - 2400.000000], mean/var= 0.00/ 0.00, + cnt=0, [13.000000 - 2400.000000], mean/var= 0.00/ 0.00, + cnt=32, [13.000000 - 2400.000000], mean/var= 0.78/ 0.18, + cnt=59, [13.000000 - 2400.000000], mean/var= 0.00/ 0.00, + cnt=1165, [13.000000 - 2400.000000], mean/var= 0.27/ 0.20, + cnt=1031, [13.000000 - 2400.000000], mean/var= 0.24/ 0.18, + cnt=1022, [13.000000 - 2400.000000], mean/var= 0.14/ 0.12, + cnt=1039, [13.000000 - 2400.000000], mean/var= 0.21/ 0.16, + cnt=1257, [13.000000 - 2400.000000], mean/var= 0.18/ 0.15, + cnt=1270, [13.000000 - 2400.000000], mean/var= 0.17/ 0.14, + cnt=1161, [13.000000 - 2400.000000], mean/var= 0.20/ 0.16, + cnt=1145, [13.000000 - 2400.000000], mean/var= 0.08/ 0.08, + cnt=1075, [13.000000 - 2400.000000], mean/var= 0.13/ 0.11, + cnt=1298, [13.000000 - 2400.000000], mean/var= 0.14/ 0.12, + cnt=1391, [13.000000 - 2400.000000], mean/var= 0.18/ 0.15, + cnt=1158, [13.000000 - 2400.000000], mean/var= 0.19/ 0.16, + cnt=700, [13.000000 - 2400.000000], mean/var= 0.06/ 0.05, + cnt=514, [13.000000 - 2400.000000], mean/var= 0.12/ 0.11, + + at hex.gbm.DHistogram.incr(DHistogram.java:137) + at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.accum_subset(SharedTreeModelBuilder.java:432) + at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.map(SharedTreeModelBuilder.java:368) + at water.MRTask2.compute2(MRTask2.java:353) + at water.MRTask2.compute2(MRTask2.java:320) + at water.MRTask2.compute2(MRTask2.java:320) + at water.MRTask2.compute2(MRTask2.java:320) + at water.MRTask2.compute2(MRTask2.java:320) + at water.MRTask2.compute2(MRTask2.java:320) + at water.MRTask2.compute2(MRTask2.java:320) + at water.MRTask2.compute2(MRTask2.java:320) + at water.MRTask2.compute2(MRTask2.java:320) + at water.H2O$H2OCountedCompleter.compute(H2O.java:713) + at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) + at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) + at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) + at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) + at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

    JIRA | 3 years ago | Michal Malohlava
    java.lang.AssertionError: col_data 0.0 out of range CRSArrTime:1.0-2401.0 step=119.99999 nbins=20 isInt=1 cnt=88, [13.000000 - 2400.000000], mean/var= 0.11/ 0.10, cnt=0, [13.000000 - 2400.000000], mean/var= 0.00/ 0.00, cnt=0, [13.000000 - 2400.000000], mean/var= 0.00/ 0.00, cnt=0, [13.000000 - 2400.000000], mean/var= 0.00/ 0.00, cnt=32, [13.000000 - 2400.000000], mean/var= 0.78/ 0.18, cnt=59, [13.000000 - 2400.000000], mean/var= 0.00/ 0.00, cnt=1165, [13.000000 - 2400.000000], mean/var= 0.27/ 0.20, cnt=1031, [13.000000 - 2400.000000], mean/var= 0.24/ 0.18, cnt=1022, [13.000000 - 2400.000000], mean/var= 0.14/ 0.12, cnt=1039, [13.000000 - 2400.000000], mean/var= 0.21/ 0.16, cnt=1257, [13.000000 - 2400.000000], mean/var= 0.18/ 0.15, cnt=1270, [13.000000 - 2400.000000], mean/var= 0.17/ 0.14, cnt=1161, [13.000000 - 2400.000000], mean/var= 0.20/ 0.16, cnt=1145, [13.000000 - 2400.000000], mean/var= 0.08/ 0.08, cnt=1075, [13.000000 - 2400.000000], mean/var= 0.13/ 0.11, cnt=1298, [13.000000 - 2400.000000], mean/var= 0.14/ 0.12, cnt=1391, [13.000000 - 2400.000000], mean/var= 0.18/ 0.15, cnt=1158, [13.000000 - 2400.000000], mean/var= 0.19/ 0.16, cnt=700, [13.000000 - 2400.000000], mean/var= 0.06/ 0.05, cnt=514, [13.000000 - 2400.000000], mean/var= 0.12/ 0.11,
  6. 0

    there's another jira probably somewhere. I don't know if it's possible for this to show up in other datasets. has importance=1 and just 1 tree. don't know if that matters it might be the NaN in the dataset here's the drf: /2/DRF.json?classification=1&importance=1&source=parsetmp_1_0_2_0.hex&ntrees=1&response=DateofBirth FJ-0-3 ERRR WATER: java.lang.AssertionError: col_data Infinity out of range FirstName:-Infinity-Infinity step=Infinity nbins=1024 isInt=0 cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, .... at hex.gbm.DHistogram.incr(DHistogram.java:137) at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.accum_subset(SharedTreeModelBuilder.java:518) at hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.map(SharedTreeModelBuilder.java:452) at water.MRTask2.compute2(MRTask2.java:391) at water.H2O$H2OCountedCompleter.compute(H2O.java:711) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) it's pretty simple. hive separator in this case, but not sure if that matters I attached the file here: FirstName^AMiddleInitials^ALastName^ADateofBirth 0^A0.5^A1^A0 3^ANaN^A4^A1 6^A^A8^A0 0.6^A0.7^A0.8^A1 +0.6^A+0.7^A+0.8^A0 -0.6^A-0.7^A-0.8^A1 .6^A.7^A.8^A0 +.6^A+.7^A+.8^A1 -.6^A-.7^A-.8^A0 +0.6e0^A+0.7e0^A+0.8e0^A1 -0.6e0^A-0.7e0^A-0.8e0^A0 .6e0^A.7e0^A.8e0^A1 +.6e0^A+.7e0^A+.8e0^A0 -.6e0^A-.7e0^A-.8e0^A1 +0.6e00^A+0.7e00^A+0.8e00^A0 -0.6e00^A-0.7e00^A-0.8e00^A1 .6e00^A.7e00^A.8e00^A0 +.6e00^A+.7e00^A+.8e00^A1 -.6e00^A-.7e00^A-.8e00^A0 +0.6e-01^A+0.7e-01^A+0.8e-01^A1 -0.6e-01^A-0.7e-01^A-0.8e-01^A0 .6e-01^A.7e-01^A.8e-01^A1 +.6e-01^A+.7e-01^A+.8e-01^A0 -.6e-01^A-.7e-01^A-.8e-01^A1 +0.6e+01^A+0.7e+01^A+0.8e+01^A0 -0.6e+01^A-0.7e+01^A-0.8e+01^A1 .6e+01^A.7e+01^A.8e+01^A0 +.6e+01^A+.7e+01^A+.8e+01^A1 -.6e+01^A-.7e+01^A-.8e+01^A0 +0.6e102^A+0.7e102^A+0.8e102^A1 -0.6e102^A-0.7e102^A-0.8e102^A0 .6e102^A.7e102^A.8e102^A1 +.6e102^A+.7e102^A+.8e102^A0 -.6e102^A-.7e102^A-.8e102^A1 2014-06-30 15:44:44.245461 -- Start http://192.168.1.40:54321/2/PostFile.json?key=parsetmp_1_0_2_0.data #/home/kevin/h2o/py/testdir_single_jvm/sandbox/syn_datasets/parsetmp_1_0_2_0.data 2014-06-30 15:44:44.253077 -- Start http://192.168.1.40:54321/2/Parse2.json?single_quotes=0&source_key=parsetmp_1_0_2_0.data 2014-06-30 15:44:45.260299 -- Start http://192.168.1.40:54321/2/Progress2.json?job_key=$0301c0a8012832d4ffffffff$_a57367bdc85fd058b738e6d9faae422f&destination_key=parsetmp_1_0_2_0.hex 2014-06-30 15:44:45.274134 -- Start http://192.168.1.40:54321/2/Inspect2.json?src_key=parsetmp_1_0_2_0.hex 2014-06-30 15:44:45.285336 -- Start http://192.168.1.40:54321/2/SummaryPage2.json?source=parsetmp_1_0_2_0.hex&max_ncols=4 2014-06-30 15:44:45.378326 -- Start http://192.168.1.40:54321/2/Inspect2.json?src_key=parsetmp_1_0_2_0.hex 2014-06-30 15:44:45.382202 -- Start http://192.168.1.40:54321/2/DRF.json?classification=1&importance=1&source=parsetmp_1_0_2_0.hex&ntrees=1&response=DateofBirth 2014-06-30 15:44:50.501616 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff 2014-06-30 15:44:50.610065 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff 2014-06-30 15:44:50.713537 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff 2014-06-30 15:44:50.817189 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff 2014-06-30 15:44:50.920776 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff 2014-06-30 15:44:51.029056 -- Start http://192.168.1.40:54321/2/DRFProgressPage.json?job_key=$0301c0a8012832d4ffffffff$_a8eee7196ec0cc243d39cc05e41541fb&destination_key=DRF_9b148a6e6efc058cdab1b9f7c54142ff

    JIRA | 2 years ago | Kevin Normoyle
    java.lang.AssertionError: col_data Infinity out of range FirstName:-Infinity-Infinity step=Infinity nbins=1024 isInt=0 cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, ....

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.AssertionError

      col_data Infinity out of range FirstName :-Infinity-Infinity step=Infinity nbins=20 isInt=0 + cnt=29, [-6.000000 - 6.000000], mean/var= 0.52/ 0.26, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, + cnt=0, [-6.000000 - 6.000000], mean/var= 0.00/ 0.00, +

      at hex.gbm.DHistogram.incr()
    2. hex.gbm
      SharedTreeModelBuilder$ScoreBuildHistogram.map
      1. hex.gbm.DHistogram.incr(DHistogram.java:137)
      2. hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.accum_subset(SharedTreeModelBuilder.java:421)
      3. hex.gbm.SharedTreeModelBuilder$ScoreBuildHistogram.map(SharedTreeModelBuilder.java:355)
      3 frames
    3. water
      H2O$H2OCountedCompleter.compute
      1. water.MRTask2.compute2(MRTask2.java:378)
      2. water.H2O$H2OCountedCompleter.compute(H2O.java:731)
      2 frames
    4. jsr166y
      ForkJoinWorkerThread.run
      1. jsr166y.CountedCompleter.exec(CountedCompleter.java:429)
      2. jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
      3. jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
      4. jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
      5. jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
      5 frames