java.lang.RuntimeException: GLM2[dest=GLMModel__9c13d5eb7583e3413bb543d808244872, iteration=1, lambda = 0.0]: got NaNs and/or Infs in beta

JIRA | Kevin Normoyle | 2 years ago
  1. 0

    cd testdir_single_jvm python test_GLM2_params_rand2 -s 2261331184608071889 trial 12? 2/GLM2 parameters: {'family': 'poisson', 'max_predictors': None, 'cols': None, 'n_folds': 0, 'use_all_factor_levels': None, 'variable_importances': None, 'higher_accuracy': None, 'ignored_cols_by_name': None, 'response': 54, 'source': u'covtype.20k', 'has_intercept': None, 'lambda_search': None, 'destination_key': None, 'standardize': None, 'max_iter': 25, 'lambda_min_ratio': None, 'alpha': 0.1, 'non_negative': 1, 'beta_epsilon': None, 'nlambdas': None, 'tweedie_variance_power': None, 'ignored_cols': None, 'prior': None, 'link': 'log', 'strong_rules_enabled': None, 'lambda': 0} does a number of trials before it gets the exception Using random seed: 2261331184608071889 rjson error in poll_url: Got exception 'class java.lang.RuntimeException', with msg 'GLM2[dest=GLMModel__9c13d5eb7583e3413bb543d808244872, iteration=1, lambda = 0.0]: got NaNs and/or Infs in beta' java.lang.RuntimeException: GLM2[dest=GLMModel__9c13d5eb7583e3413bb543d808244872, iteration=1, lambda = 0.0]: got NaNs and/or Infs in beta at hex.glm.GLM2$Iteration.callback(GLM2.java:959) at hex.glm.GLM2$Iteration.callback(GLM2.java:871) at water.H2O$H2OCallback.onCompletion(H2O.java:671) at jsr166y.CountedCompleter.tryComplete(CountedCompleter.java:386) at water.MRTask2.compute2(MRTask2.java:426) at water.H2O$H2OCountedCompleter.compute(H2O.java:647) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

    JIRA | 2 years ago | Kevin Normoyle
    java.lang.RuntimeException: GLM2[dest=GLMModel__9c13d5eb7583e3413bb543d808244872, iteration=1, lambda = 0.0]: got NaNs and/or Infs in beta
  2. 0

    cd testdir_single_jvm python test_GLM2_params_rand2 -s 2261331184608071889 trial 12? 2/GLM2 parameters: {'family': 'poisson', 'max_predictors': None, 'cols': None, 'n_folds': 0, 'use_all_factor_levels': None, 'variable_importances': None, 'higher_accuracy': None, 'ignored_cols_by_name': None, 'response': 54, 'source': u'covtype.20k', 'has_intercept': None, 'lambda_search': None, 'destination_key': None, 'standardize': None, 'max_iter': 25, 'lambda_min_ratio': None, 'alpha': 0.1, 'non_negative': 1, 'beta_epsilon': None, 'nlambdas': None, 'tweedie_variance_power': None, 'ignored_cols': None, 'prior': None, 'link': 'log', 'strong_rules_enabled': None, 'lambda': 0} does a number of trials before it gets the exception Using random seed: 2261331184608071889 rjson error in poll_url: Got exception 'class java.lang.RuntimeException', with msg 'GLM2[dest=GLMModel__9c13d5eb7583e3413bb543d808244872, iteration=1, lambda = 0.0]: got NaNs and/or Infs in beta' java.lang.RuntimeException: GLM2[dest=GLMModel__9c13d5eb7583e3413bb543d808244872, iteration=1, lambda = 0.0]: got NaNs and/or Infs in beta at hex.glm.GLM2$Iteration.callback(GLM2.java:959) at hex.glm.GLM2$Iteration.callback(GLM2.java:871) at water.H2O$H2OCallback.onCompletion(H2O.java:671) at jsr166y.CountedCompleter.tryComplete(CountedCompleter.java:386) at water.MRTask2.compute2(MRTask2.java:426) at water.H2O$H2OCountedCompleter.compute(H2O.java:647) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

    JIRA | 2 years ago | Kevin Normoyle
    java.lang.RuntimeException: GLM2[dest=GLMModel__9c13d5eb7583e3413bb543d808244872, iteration=1, lambda = 0.0]: got NaNs and/or Infs in beta
  3. 0

    Getting this failure while building a GLM model on smalldata/airlines. Using h2o-dev/master, d1705680ef5417bcfdff34b84cdca2c7212c74dd. Log output: 02-05 14:23:01.959 172.16.2.96:54321 10761 # Session INFO: Method: POST , Path: /3/ModelBuilders/glm, route: /3/ModelBuilders/glm, parms: {tweedie_variance_power=NaN, standardize=true, alpha=[0.5], balance_classes=false, lambda_min_ratio=-1, tweedie_link_power=NaN, higher_accuracy=false, max_after_balance_size=5, link=family_default, use_all_factor_levels=false, lambda=[0.00001], response_column=IsDepDelayed, nlambdas=-1, do_classification=true, class_sampling_factors=, lambda_search=false, training_frame=allyears2k_headers.hex, n_folds=0, family=gaussian, prior1=0, solver=ADMM, destination_key=} 02-05 14:23:01.962 172.16.2.96:54321 10761 # Session INFO: Building H2O GLM model with these parameters: 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: { 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_train": { 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "name": "allyears2k_headers.hex", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "type": "Key" 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: }, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_dropNA20Cols": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_dropConsCols": true, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_score_each_iteration": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_max_confusion_matrix_size": 20, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_response_column": "IsDepDelayed", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_convert_to_enum": true, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_balance_classes": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_max_after_balance_size": 5.0, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_max_hit_ratio_k": 10, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_standardize": true, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_family": "gaussian", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_link": "family_default", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_solver": "ADMM", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_tweedie_variance_power": "NaN", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_tweedie_link_power": "NaN", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_alpha": [ 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: 0.5 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: ], 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_lambda": [ 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: 1.0E-5 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: ], 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_prior": -1.0, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_lambda_search": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_nlambdas": -1, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_lambda_min_ratio": -1.0, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_higher_accuracy": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_use_all_factor_levels": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_n_folds": 0, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_max_active_predictors": 10000 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: } ... barrier onExCompletion for hex.glm.GLM$1@79ea98b java.lang.RuntimeException: GLM2[dest=GLMModel__b779758b29d625985bd5144d73d34b36, iteration=1, lambda = 1.0E-5]: got NaNs and/or Infs in beta at hex.glm.GLM$GLMLambdaTask$Iteration.callback(GLM.java:530) at hex.glm.GLM$GLMLambdaTask$Iteration.callback(GLM.java:464) at water.H2O$H2OCallback.onCompletion(H2O.java:640) at jsr166y.CountedCompleter.tryComplete(CountedCompleter.java:386) at water.MRTask.compute2(MRTask.java:437) at water.H2O$H2OCountedCompleter.compute(H2O.java:582) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

    JIRA | 2 years ago | Prithvi Prabhu
    java.lang.RuntimeException: GLM2[dest=GLMModel__b779758b29d625985bd5144d73d34b36, iteration=1, lambda = 1.0E-5]: got NaNs and/or Infs in beta
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Getting this failure while building a GLM model on smalldata/airlines. Using h2o-dev/master, d1705680ef5417bcfdff34b84cdca2c7212c74dd. Log output: 02-05 14:23:01.959 172.16.2.96:54321 10761 # Session INFO: Method: POST , Path: /3/ModelBuilders/glm, route: /3/ModelBuilders/glm, parms: {tweedie_variance_power=NaN, standardize=true, alpha=[0.5], balance_classes=false, lambda_min_ratio=-1, tweedie_link_power=NaN, higher_accuracy=false, max_after_balance_size=5, link=family_default, use_all_factor_levels=false, lambda=[0.00001], response_column=IsDepDelayed, nlambdas=-1, do_classification=true, class_sampling_factors=, lambda_search=false, training_frame=allyears2k_headers.hex, n_folds=0, family=gaussian, prior1=0, solver=ADMM, destination_key=} 02-05 14:23:01.962 172.16.2.96:54321 10761 # Session INFO: Building H2O GLM model with these parameters: 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: { 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_train": { 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "name": "allyears2k_headers.hex", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "type": "Key" 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: }, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_dropNA20Cols": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_dropConsCols": true, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_score_each_iteration": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_max_confusion_matrix_size": 20, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_response_column": "IsDepDelayed", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_convert_to_enum": true, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_balance_classes": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_max_after_balance_size": 5.0, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_max_hit_ratio_k": 10, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_standardize": true, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_family": "gaussian", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_link": "family_default", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_solver": "ADMM", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_tweedie_variance_power": "NaN", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_tweedie_link_power": "NaN", 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_alpha": [ 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: 0.5 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: ], 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_lambda": [ 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: 1.0E-5 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: ], 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_prior": -1.0, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_lambda_search": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_nlambdas": -1, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_lambda_min_ratio": -1.0, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_higher_accuracy": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_use_all_factor_levels": false, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_n_folds": 0, 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: "_max_active_predictors": 10000 02-05 14:23:01.975 172.16.2.96:54321 10761 # Session INFO: } ... barrier onExCompletion for hex.glm.GLM$1@79ea98b java.lang.RuntimeException: GLM2[dest=GLMModel__b779758b29d625985bd5144d73d34b36, iteration=1, lambda = 1.0E-5]: got NaNs and/or Infs in beta at hex.glm.GLM$GLMLambdaTask$Iteration.callback(GLM.java:530) at hex.glm.GLM$GLMLambdaTask$Iteration.callback(GLM.java:464) at water.H2O$H2OCallback.onCompletion(H2O.java:640) at jsr166y.CountedCompleter.tryComplete(CountedCompleter.java:386) at water.MRTask.compute2(MRTask.java:437) at water.H2O$H2OCountedCompleter.compute(H2O.java:582) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

    JIRA | 2 years ago | Prithvi Prabhu
    java.lang.RuntimeException: GLM2[dest=GLMModel__b779758b29d625985bd5144d73d34b36, iteration=1, lambda = 1.0E-5]: got NaNs and/or Infs in beta
  6. 0

    for -" ~/h2o-dev/bigdata/laptop/citibike-nyc/2013-07.csv" on a two node cluster, when run - {code:java} buildModel 'glm', {"model_id":"glm-d216d362-8687-4a88-a897-26192ef32bde","training_frame":"X2013_07.hex","nfolds":0,"response_column":"tripduration","ignored_columns":["starttime","stoptime","start station id","start station longitude","end station id","end station name","end station latitude","end station longitude","bikeid","usertype","birth year","gender"],"ignore_const_cols":true,"family":"gaussian","tweedie_variance_power":0,"tweedie_link_power":1,"solver":"IRLSM","alpha":[],"lambda":[0],"lambda_search":false,"nlambdas":100,"standardize":true,"non_negative":false,"fold_assignment":"AUTO","score_each_iteration":false,"compute_p_values":true,"remove_colinear_columns":false,"max_iterations":-1,"link":"family_default","balance_classes":false,"max_confusion_matrix_size":20,"max_hit_ratio_k":10,"keep_cross_validation_predictions":false,"intercept":true,"objective_epsilon":0.000001,"beta_epsilon":0.0001,"gradient_epsilon":0.0001,"prior":-1,"lambda_min_ratio":-1,"max_active_predictors":-1,"class_sampling_factors":[],"max_after_balance_size":5} {code} {code:java} Got exception 'class java.lang.RuntimeException', with msg 'GLM[dest=glm-d216d362-8687-4a88-a897-26192ef32bde, iteration=1, lambda = 0.0]: got NaNs and/or Infs in beta' java.lang.RuntimeException: GLM[dest=glm-d216d362-8687-4a88-a897-26192ef32bde, iteration=1, lambda = 0.0]: got NaNs and/or Infs in beta at hex.glm.GLM$GLMSingleLambdaTsk$Iteration.callback(GLM.java:2038) at hex.glm.GLM$GLMSingleLambdaTsk$Iteration.callback(GLM.java:1921) at water.H2O$H2OCallback.onCompletion(H2O.java:1116) at jsr166y.CountedCompleter.__tryComplete(CountedCompleter.java:425) at water.RPC$2.compute2(RPC.java:621) at water.H2O$H2OCountedCompleter.compute(H2O.java:1057) at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) {code} but when run on 1 node runs fine, but get a bad model (p-values all one) {code:java} hdata = h2o.importFile("/Users/nidhimehta/h2o-dev/bigdata/laptop/citibike-nyc/2013-07.csv") hh =h2o.glm(x =c("start station name","start station latitude"),y = "tripduration",training_frame = hdata, lambda = 0,compute_p_values = T,remove_colinear_columns = F) --- names coefficients std_error z_value 325 start station name.Willoughby Ave & Hall St -60.672039 93997415.051949 -0.000001 326 start station name.Willoughby Ave & Walworth St 1287.155073 90829446.884767 0.000014 327 start station name.Willoughby St & Fleet St -117.654894 93985016.918904 -0.000001 328 start station name.Wythe Ave & Metropolitan Ave 659.314909 35797052.643935 0.000018 329 start station name.York St & Jay St 207.768930 71758409.736402 0.000003 330 start station latitude -1408.199133 2334864990.210110 -0.000001 p_value standardized_coefficients 325 1.000000 -60.672039 326 0.999989 1287.155073 327 1.000000 -117.654894 328 0.999985 659.314909 329 1.000000 207.768930 330 1.000000 -28.986619 {code} while R-glm na's out the last predictor and returns a decent model {code:java} x = read.csv("/Users/nidhimehta/h2o-dev/bigdata/laptop/citibike-nyc/2013-07.csv") gg = glm(formula = tripduration~start.station.name+start.station.latitude,data = x) summary(gg) ......... start.station.nameWilloughby Ave & Walworth St 1341.93595 311.89845 4.302 1.69e-05 *** start.station.nameWilloughby St & Fleet St -60.97084 276.97337 -0.220 0.825768 start.station.nameWythe Ave & Metropolitan Ave 680.90476 200.01770 3.404 0.000664 *** start.station.nameYork St & Jay St 251.04772 209.87106 1.196 0.231619 start.station.latitude NA NA NA NA --- {code} Would be good if H2O - tells the user - #got NaNs and/or Infs in beta : try running with regularization (and returns the NAed out last predictor model?)

    JIRA | 12 months ago | Nidhi Mehta
    java.lang.RuntimeException: GLM[dest=glm-d216d362-8687-4a88-a897-26192ef32bde, iteration=1, lambda = 0.0]: got NaNs and/or Infs in beta

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.RuntimeException

      GLM2[dest=GLMModel__9c13d5eb7583e3413bb543d808244872, iteration=1, lambda = 0.0]: got NaNs and/or Infs in beta

      at hex.glm.GLM2$Iteration.callback()
    2. hex.glm
      GLM2$Iteration.callback
      1. hex.glm.GLM2$Iteration.callback(GLM2.java:959)
      2. hex.glm.GLM2$Iteration.callback(GLM2.java:871)
      2 frames
    3. water
      H2O$H2OCallback.onCompletion
      1. water.H2O$H2OCallback.onCompletion(H2O.java:671)
      1 frame
    4. jsr166y
      CountedCompleter.tryComplete
      1. jsr166y.CountedCompleter.tryComplete(CountedCompleter.java:386)
      1 frame
    5. water
      H2O$H2OCountedCompleter.compute
      1. water.MRTask2.compute2(MRTask2.java:426)
      2. water.H2O$H2OCountedCompleter.compute(H2O.java:647)
      2 frames
    6. jsr166y
      ForkJoinWorkerThread.run
      1. jsr166y.CountedCompleter.exec(CountedCompleter.java:429)
      2. jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
      3. jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
      4. jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
      5. jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
      5 frames