java.lang.RuntimeException: water.DException$DistributedException: from /192.168.0.34:54323; by class water.KeySnapshot$GlobalUKeySetTask; class water.DException$DistributedException: from /192.168.0.34:54321; by class water.KeySnapshot$GlobalUKeySetTask; class java.lang.AssertionError: *** Attempting to block on task (class water.TaskGetKey) with equal or lower priority. Can lead to deadlock! 122 <= 122

JIRA | Kevin Normoyle | 2 years ago
  1. 0

    Not sure why this assertion is WARN? if it's an exception, it should be an ERROR? The assertion repros if the seed is used as shown. If not, I seem to get a bad json response, which is printed in red. I used a test that works on h2o that has minimal port to h2o-dev..i.e. it uses base h2o methods like import/parse/frames which I tested with other tests. the unique thing is this uses utf8 characters. I had sorted out all known issues with utf8 (i.e. what works) on h2o. I was suprised to see a bad json response from h2o-dev. I reduced the number of rows to get the domain values shorter for debug. this time, I got an assertion error from h2o-dev I'll leave the test as is (not sure it will complete but it should) it creates a random dataset. The seed can be forced with -s if needed to replicate ( I think not needed) actually I think I'll hardwire the seed in the test so you don't need the -s param shown. I also attached the file created in sandbox/syn_datasets to run cd h2o-dev/py2/testdir_multi_jvm (it does 2 jvms. don't know if that matters) python test_parse_rand_utf8.py -s 1364157389021990032 to run on existing cloud python test_parse_rand_utf8.py -s 1364157389021990032 -uc or python test_parse_rand_utf8.py -s 1364157389021990032 -uc localhost:54321 if you need to change the default 54321 port frames parameters: {'find_compatible_models': 0, 'len': 5, 'offset': 0} JSON call returned non-200 status with http://192.168.0.34:54323/3/Frames.json?find_compatible_models=0&len=5&offset=0 r.status_code: 500 r.headers: {'date': 'Sun, 16 Nov 2014 04:44:31 GMT', 'content-type': 'application/json'} r.text: {"schema_version":-1,"schema_name":"Schema","schema_type":"Iced","status_header":"500 Internal Server Error","errmsg":"RuntimeException: water.DException$DistributedException: from /192.168.0.34:54323; by class water.KeySnapshot$GlobalUKeySetTask; class water.DException$DistributedException: from /192.168.0.34:54321; by class water.KeySnapshot$GlobalUKeySetTask; class java.lang.AssertionError: *** Attempting to block on task (class water.TaskGetKey) with equal or lower priority. Can lead to deadlock! 122 <= 122","errurl":null,"stacktrace":["water.MRTask.getResult(MRTask.java:261)","water.MRTask.doAll(MRTask.java:291)","water.MRTask.doAllNodes(MRTask.java:283)","water.KeySnapshot.globalSnapshot(KeySnapshot.java:208)","water.KeySnapshot.globalSnapshot(KeySnapshot.java:195)","water.api.FramesHandler$Frames.fetchAll(FramesHandler.java:29)","water.api.FramesHandler.list(FramesHandler.java:116)","sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)","sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)","sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)","java.lang.reflect.Method.invoke(Method.java:606)","water.api.Handler.handle(Handler.java:55)","water.api.RequestServer.handle(RequestServer.java:476)","water.api.RequestServer.serve(RequestServer.java:454)","water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:432)","java.lang.Thread.run(Thread.java:745)"]} ERROR 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: java.lang.RuntimeException: water.DException$DistributedException: from /192.168.0.34:54323; by class water.KeySnapshot$GlobalUKeySetTask; class water.DException$DistributedException: from /192.168.0.34:54321; by class water.KeySnapshot$GlobalUKeySetTask; class java.lang.AssertionError: *** Attempting to block on task (class water.TaskGetKey) with equal or lower priority. Can lead to deadlock! 122 <= 122 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.MRTask.getResult(MRTask.java:261) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.MRTask.doAll(MRTask.java:291) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.MRTask.doAllNodes(MRTask.java:283) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.KeySnapshot.globalSnapshot(KeySnapshot.java:208) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.KeySnapshot.globalSnapshot(KeySnapshot.java:195) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.api.FramesHandler$Frames.fetchAll(FramesHandler.java:29) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.api.FramesHandler.list(FramesHandler.java:116) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at java.lang.reflect.Method.invoke(Method.java:606) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.api.Handler.handle(Handler.java:55) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.api.RequestServer.handle(RequestServer.java:476) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.api.RequestServer.serve(RequestServer.java:454) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:432) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at java.lang.Thread.run(Thread.java:745)

    JIRA | 2 years ago | Kevin Normoyle
    java.lang.RuntimeException: water.DException$DistributedException: from /192.168.0.34:54323; by class water.KeySnapshot$GlobalUKeySetTask; class water.DException$DistributedException: from /192.168.0.34:54321; by class water.KeySnapshot$GlobalUKeySetTask; class java.lang.AssertionError: *** Attempting to block on task (class water.TaskGetKey) with equal or lower priority. Can lead to deadlock! 122 <= 122
  2. 0

    Not sure why this assertion is WARN? if it's an exception, it should be an ERROR? The assertion repros if the seed is used as shown. If not, I seem to get a bad json response, which is printed in red. I used a test that works on h2o that has minimal port to h2o-dev..i.e. it uses base h2o methods like import/parse/frames which I tested with other tests. the unique thing is this uses utf8 characters. I had sorted out all known issues with utf8 (i.e. what works) on h2o. I was suprised to see a bad json response from h2o-dev. I reduced the number of rows to get the domain values shorter for debug. this time, I got an assertion error from h2o-dev I'll leave the test as is (not sure it will complete but it should) it creates a random dataset. The seed can be forced with -s if needed to replicate ( I think not needed) actually I think I'll hardwire the seed in the test so you don't need the -s param shown. I also attached the file created in sandbox/syn_datasets to run cd h2o-dev/py2/testdir_multi_jvm (it does 2 jvms. don't know if that matters) python test_parse_rand_utf8.py -s 1364157389021990032 to run on existing cloud python test_parse_rand_utf8.py -s 1364157389021990032 -uc or python test_parse_rand_utf8.py -s 1364157389021990032 -uc localhost:54321 if you need to change the default 54321 port frames parameters: {'find_compatible_models': 0, 'len': 5, 'offset': 0} JSON call returned non-200 status with http://192.168.0.34:54323/3/Frames.json?find_compatible_models=0&len=5&offset=0 r.status_code: 500 r.headers: {'date': 'Sun, 16 Nov 2014 04:44:31 GMT', 'content-type': 'application/json'} r.text: {"schema_version":-1,"schema_name":"Schema","schema_type":"Iced","status_header":"500 Internal Server Error","errmsg":"RuntimeException: water.DException$DistributedException: from /192.168.0.34:54323; by class water.KeySnapshot$GlobalUKeySetTask; class water.DException$DistributedException: from /192.168.0.34:54321; by class water.KeySnapshot$GlobalUKeySetTask; class java.lang.AssertionError: *** Attempting to block on task (class water.TaskGetKey) with equal or lower priority. Can lead to deadlock! 122 <= 122","errurl":null,"stacktrace":["water.MRTask.getResult(MRTask.java:261)","water.MRTask.doAll(MRTask.java:291)","water.MRTask.doAllNodes(MRTask.java:283)","water.KeySnapshot.globalSnapshot(KeySnapshot.java:208)","water.KeySnapshot.globalSnapshot(KeySnapshot.java:195)","water.api.FramesHandler$Frames.fetchAll(FramesHandler.java:29)","water.api.FramesHandler.list(FramesHandler.java:116)","sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)","sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)","sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)","java.lang.reflect.Method.invoke(Method.java:606)","water.api.Handler.handle(Handler.java:55)","water.api.RequestServer.handle(RequestServer.java:476)","water.api.RequestServer.serve(RequestServer.java:454)","water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:432)","java.lang.Thread.run(Thread.java:745)"]} ERROR 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: java.lang.RuntimeException: water.DException$DistributedException: from /192.168.0.34:54323; by class water.KeySnapshot$GlobalUKeySetTask; class water.DException$DistributedException: from /192.168.0.34:54321; by class water.KeySnapshot$GlobalUKeySetTask; class java.lang.AssertionError: *** Attempting to block on task (class water.TaskGetKey) with equal or lower priority. Can lead to deadlock! 122 <= 122 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.MRTask.getResult(MRTask.java:261) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.MRTask.doAll(MRTask.java:291) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.MRTask.doAllNodes(MRTask.java:283) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.KeySnapshot.globalSnapshot(KeySnapshot.java:208) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.KeySnapshot.globalSnapshot(KeySnapshot.java:195) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.api.FramesHandler$Frames.fetchAll(FramesHandler.java:29) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.api.FramesHandler.list(FramesHandler.java:116) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at java.lang.reflect.Method.invoke(Method.java:606) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.api.Handler.handle(Handler.java:55) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.api.RequestServer.handle(RequestServer.java:476) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.api.RequestServer.serve(RequestServer.java:454) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:432) 11-15 20:44:31.361 192.168.0.34:54323 8816 # Session WARN: at java.lang.Thread.run(Thread.java:745)

    JIRA | 2 years ago | Kevin Normoyle
    java.lang.RuntimeException: water.DException$DistributedException: from /192.168.0.34:54323; by class water.KeySnapshot$GlobalUKeySetTask; class water.DException$DistributedException: from /192.168.0.34:54321; by class water.KeySnapshot$GlobalUKeySetTask; class java.lang.AssertionError: *** Attempting to block on task (class water.TaskGetKey) with equal or lower priority. Can lead to deadlock! 122 <= 122
  3. 0

    Start client: java -jar build/h2o.jar -client Start node: java -jar build/h2o.jar Run runit: run.py --wipeall --usecloud nodeip:nodeport --test testdir_misc/runit_mergecat.R Stacktrace: onExCompletion for water.rapids.ASTMerge$MergeSet$MakeHash@6c490226 05-29 11:55:21.115 172.16.2.41:54321 17607 # Session ERRR: java.lang.RuntimeException: water.DException$DistributedException: from /172.16.2.41:54321; by class water.rapids.ASTMerge$MergeSet$MakeHash; class water.DException$DistributedException: from /172.16.2.41:54323; by class water.rapids.ASTMerge$MergeSet$MakeHash; class java.lang.NullPointerException: null water.DException$DistributedException: from /172.16.2.41:54323; by class water.rapids.ASTMerge$MergeSet$MakeHash; class java.lang.NullPointerException: null at water.rapids.ASTMerge$MergeSet$MakeHash.map(ASTMerge.java:192) at water.MRTask.compute2(MRTask.java:443) at water.H2O$H2OCountedCompleter.compute(H2O.java:666) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) java.lang.RuntimeException: water.DException$DistributedException: from /172.16.2.41:54321; by class water.rapids.ASTMerge$MergeSet$MakeHash; class water.DException$DistributedException: from /172.16.2.41:54323; by class water.rapids.ASTMerge$MergeSet$MakeHash; class java.lang.NullPointerException: null at water.MRTask.getResult(MRTask.java:267) at water.MRTask.doAll(MRTask.java:221) at water.MRTask.doAll(MRTask.java:216) at water.rapids.ASTMerge$MergeSet.setupLocal(ASTMerge.java:182) at water.MRTask.setupLocal0(MRTask.java:334) at water.MRTask.doAll(MRTask.java:301) at water.MRTask.doAllNodes(MRTask.java:290) at water.rapids.ASTMerge.apply(ASTMerge.java:106) at water.rapids.AST.treeWalk(AST.java:73) at water.rapids.AST.treeWalk(AST.java:83) at water.rapids.Exec.exec(Exec.java:68) at water.api.RapidsHandler.exec(RapidsHandler.java:37) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at water.api.Handler.handle(Handler.java:56) at water.api.RequestServer.handle(RequestServer.java:675) at water.api.RequestServer.serve(RequestServer.java:613) at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:438) at java.lang.Thread.run(Thread.java:745) Caused by: water.DException$DistributedException: from /172.16.2.41:54321; by class water.rapids.ASTMerge$MergeSet$MakeHash; class water.DException$DistributedException: from /172.16.2.41:54323; by class water.rapids.ASTMerge$MergeSet$MakeHash; class java.lang.NullPointerException: null at water.rapids.ASTMerge$MergeSet$MakeHash.map(ASTMerge.java:192) at water.MRTask.compute2(MRTask.java:443) at water.H2O$H2OCountedCompleter.compute(H2O.java:666) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 05-29 11:55:21.116 172.16.2.41:54321 17607 # Session WARN: Caught exception: water.DException$DistributedException: from /172.16.2.41:54321; by class water.rapids.ASTMerge$MergeSet$MakeHash; class water.DException$DistributedException: from /172.16.2.41:54323; by class water.rapids.ASTMerge$MergeSet$MakeHash; class java.lang.NullPointerException: null; Stacktrace: [water.api.RapidsHandler.exec(RapidsHandler.java:132), sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57), sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), java.lang.reflect.Method.invoke(Method.java:606), water.api.Handler.handle(Handler.java:56), water.api.RequestServer.handle(RequestServer.java:675), water.api.RequestServer.serve(RequestServer.java:613), water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:438), java.lang.Thread.run(Thread.java:745)

    JIRA | 2 years ago | Eric Eckstrand
    java.lang.RuntimeException: water.DException$DistributedException: from /172.16.2.41:54321; by class water.rapids.ASTMerge$MergeSet$MakeHash; class water.DException$DistributedException: from /172.16.2.41:54323; by class water.rapids.ASTMerge$MergeSet$MakeHash; class java.lang.NullPointerException: null water.DException$DistributedException: from /172.16.2.41:54323; by class water.rapids.ASTMerge$MergeSet$MakeHash; class java.lang.NullPointerException: null
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Steps to reproduce: {code} val data = sc.parallelize((1 to 100000).map(_=>java.util.UUID.randomUUID().toString+" "+java.util.UUID.randomUUID().toString)) val schema = StructType(Seq(StructField("c1", StringType, true))) val rowRDD = data.map(Row(_)) val stringColumnTable = sqlContext.applySchema(rowRDD, schema) val h2oFrame = toDataFrame(stringColumnTable) scala> h2oFrame.vecs.map(_.get_type_str) res2: Array[String] = Array(string) scala> org.apache.spark.examples.h2o.DemoUtils.splitFrame(h2oFrame, Array("1","2","3"), Array(0.6,0.2,0.2)) onExCompletion for hex.splitframe.ShuffleSplitFrame$1@5d9445ed onExCompletion for hex.splitframe.ShuffleSplitFrame$1@5d9445ed water.DException$DistributedException: from /192.168.1.4:54321; by class hex.splitframe.ShuffleSplitFrame$1; class java.lang.IllegalArgumentException: Operation not allowed on string vector. at water.fvec.CStrChunk.atd_impl(CStrChunk.java:39) at water.fvec.Chunk.atd(Chunk.java:252) at hex.splitframe.ShuffleSplitFrame$1.map(ShuffleSplitFrame.java:40) at water.MRTask.compute2(MRTask.java:430) at water.H2O$H2OCountedCompleter.compute(H2O.java:582) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) water.DException$DistributedException: from /192.168.1.4:54323; by class hex.splitframe.ShuffleSplitFrame$1; class java.lang.IllegalArgumentException: Operation not allowed on string vector. at water.fvec.CStrChunk.atd_impl(CStrChunk.java:39) at water.fvec.Chunk.atd(Chunk.java:252) at hex.splitframe.ShuffleSplitFrame$1.map(ShuffleSplitFrame.java:40) at water.MRTask.compute2(MRTask.java:430) at water.MRTask.compute2(MRTask.java:374) at water.MRTask.compute2(MRTask.java:374) at water.MRTask.compute2(MRTask.java:374) at water.H2O$H2OCountedCompleter.compute(H2O.java:582) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) java.lang.RuntimeException: water.DException$DistributedException: from /192.168.1.4:54327; by class hex.splitframe.ShuffleSplitFrame$1; class water.DException$DistributedException: from /192.168.1.4:54321; by class hex.splitframe.ShuffleSplitFrame$1; class java.lang.IllegalArgumentException: Operation not allowed on string vector. at water.MRTask.getResult(MRTask.java:265) at water.MRTask.doAll(MRTask.java:220) at water.MRTask.doAll(MRTask.java:217) at hex.splitframe.ShuffleSplitFrame.shuffleSplitFrame(ShuffleSplitFrame.java:29) at org.apache.spark.examples.h2o.DemoUtils$.splitFrame(DemoUtils.scala:110) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:35) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:40) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:42) at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:44) at $iwC$$iwC$$iwC$$iwC.<init>(<console>:46) at $iwC$$iwC$$iwC.<init>(<console>:48) at $iwC$$iwC.<init>(<console>:50) at $iwC.<init>(<console>:52) at <init>(<console>:54) at .<init>(<console>:58) at .<clinit>(<console>) at .<init>(<console>:7) at .<clinit>(<console>) at $print(<console>) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:852) at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1125) at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:674) at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:705) at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:669) at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:828) at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:873) at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:785) at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:628) at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:636) at org.apache.spark.repl.SparkILoop.loop(SparkILoop.scala:641) at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:968) at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916) at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916) at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:916) at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1011) at org.apache.spark.repl.Main$.main(Main.scala:31) at org.apache.spark.repl.Main.main(Main.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: water.DException$DistributedException: from /192.168.1.4:54327; by class hex.splitframe.ShuffleSplitFrame$1; class water.DException$DistributedException: from /192.168.1.4:54321; by class hex.splitframe.ShuffleSplitFrame$1; class java.lang.IllegalArgumentException: Operation not allowed on string vector. at water.fvec.CStrChunk.atd_impl(CStrChunk.java:39) at water.fvec.Chunk.atd(Chunk.java:252) at hex.splitframe.ShuffleSplitFrame$1.map(ShuffleSplitFrame.java:40) at water.MRTask.compute2(MRTask.java:430) at water.H2O$H2OCountedCompleter.compute(H2O.java:582) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) {code} Reproduced on this build: http://h2o-release.s3.amazonaws.com/sparkling-water/master/86/index.html

    JIRA | 2 years ago | Peter Rudenko
    java.lang.RuntimeException: water.DException$DistributedException: from /192.168.1.4:54327; by class hex.splitframe.ShuffleSplitFrame$1; class water.DException$DistributedException: from /192.168.1.4:54321; by class hex.splitframe.ShuffleSplitFrame$1; class java.lang.IllegalArgumentException: Operation not allowed on string vector.
  6. 0

    Steps to reproduce: {code} val data = sc.parallelize((1 to 100000).map(_=>java.util.UUID.randomUUID().toString+" "+java.util.UUID.randomUUID().toString)) val schema = StructType(Seq(StructField("c1", StringType, true))) val rowRDD = data.map(Row(_)) val stringColumnTable = sqlContext.applySchema(rowRDD, schema) val h2oFrame = toDataFrame(stringColumnTable) scala> h2oFrame.vecs.map(_.get_type_str) res2: Array[String] = Array(string) scala> org.apache.spark.examples.h2o.DemoUtils.splitFrame(h2oFrame, Array("1","2","3"), Array(0.6,0.2,0.2)) onExCompletion for hex.splitframe.ShuffleSplitFrame$1@5d9445ed onExCompletion for hex.splitframe.ShuffleSplitFrame$1@5d9445ed water.DException$DistributedException: from /192.168.1.4:54321; by class hex.splitframe.ShuffleSplitFrame$1; class java.lang.IllegalArgumentException: Operation not allowed on string vector. at water.fvec.CStrChunk.atd_impl(CStrChunk.java:39) at water.fvec.Chunk.atd(Chunk.java:252) at hex.splitframe.ShuffleSplitFrame$1.map(ShuffleSplitFrame.java:40) at water.MRTask.compute2(MRTask.java:430) at water.H2O$H2OCountedCompleter.compute(H2O.java:582) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) water.DException$DistributedException: from /192.168.1.4:54323; by class hex.splitframe.ShuffleSplitFrame$1; class java.lang.IllegalArgumentException: Operation not allowed on string vector. at water.fvec.CStrChunk.atd_impl(CStrChunk.java:39) at water.fvec.Chunk.atd(Chunk.java:252) at hex.splitframe.ShuffleSplitFrame$1.map(ShuffleSplitFrame.java:40) at water.MRTask.compute2(MRTask.java:430) at water.MRTask.compute2(MRTask.java:374) at water.MRTask.compute2(MRTask.java:374) at water.MRTask.compute2(MRTask.java:374) at water.H2O$H2OCountedCompleter.compute(H2O.java:582) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) java.lang.RuntimeException: water.DException$DistributedException: from /192.168.1.4:54327; by class hex.splitframe.ShuffleSplitFrame$1; class water.DException$DistributedException: from /192.168.1.4:54321; by class hex.splitframe.ShuffleSplitFrame$1; class java.lang.IllegalArgumentException: Operation not allowed on string vector. at water.MRTask.getResult(MRTask.java:265) at water.MRTask.doAll(MRTask.java:220) at water.MRTask.doAll(MRTask.java:217) at hex.splitframe.ShuffleSplitFrame.shuffleSplitFrame(ShuffleSplitFrame.java:29) at org.apache.spark.examples.h2o.DemoUtils$.splitFrame(DemoUtils.scala:110) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:35) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:40) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:42) at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:44) at $iwC$$iwC$$iwC$$iwC.<init>(<console>:46) at $iwC$$iwC$$iwC.<init>(<console>:48) at $iwC$$iwC.<init>(<console>:50) at $iwC.<init>(<console>:52) at <init>(<console>:54) at .<init>(<console>:58) at .<clinit>(<console>) at .<init>(<console>:7) at .<clinit>(<console>) at $print(<console>) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:852) at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1125) at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:674) at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:705) at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:669) at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:828) at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:873) at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:785) at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:628) at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:636) at org.apache.spark.repl.SparkILoop.loop(SparkILoop.scala:641) at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:968) at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916) at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916) at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:916) at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1011) at org.apache.spark.repl.Main$.main(Main.scala:31) at org.apache.spark.repl.Main.main(Main.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: water.DException$DistributedException: from /192.168.1.4:54327; by class hex.splitframe.ShuffleSplitFrame$1; class water.DException$DistributedException: from /192.168.1.4:54321; by class hex.splitframe.ShuffleSplitFrame$1; class java.lang.IllegalArgumentException: Operation not allowed on string vector. at water.fvec.CStrChunk.atd_impl(CStrChunk.java:39) at water.fvec.Chunk.atd(Chunk.java:252) at hex.splitframe.ShuffleSplitFrame$1.map(ShuffleSplitFrame.java:40) at water.MRTask.compute2(MRTask.java:430) at water.H2O$H2OCountedCompleter.compute(H2O.java:582) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) {code} Reproduced on this build: http://h2o-release.s3.amazonaws.com/sparkling-water/master/86/index.html

    JIRA | 2 years ago | Peter Rudenko
    java.lang.RuntimeException: water.DException$DistributedException: from /192.168.1.4:54327; by class hex.splitframe.ShuffleSplitFrame$1; class water.DException$DistributedException: from /192.168.1.4:54321; by class hex.splitframe.ShuffleSplitFrame$1; class java.lang.IllegalArgumentException: Operation not allowed on string vector.

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.RuntimeException

      water.DException$DistributedException: from /192.168.0.34:54323; by class water.KeySnapshot$GlobalUKeySetTask; class water.DException$DistributedException: from /192.168.0.34:54321; by class water.KeySnapshot$GlobalUKeySetTask; class java.lang.AssertionError: *** Attempting to block on task (class water.TaskGetKey) with equal or lower priority. Can lead to deadlock! 122 <= 122

      at water.MRTask.getResult()
    2. water
      KeySnapshot.globalSnapshot
      1. water.MRTask.getResult(MRTask.java:261)
      2. water.MRTask.doAll(MRTask.java:291)
      3. water.MRTask.doAllNodes(MRTask.java:283)
      4. water.KeySnapshot.globalSnapshot(KeySnapshot.java:208)
      5. water.KeySnapshot.globalSnapshot(KeySnapshot.java:195)
      5 frames
    3. water.api
      FramesHandler.list
      1. water.api.FramesHandler$Frames.fetchAll(FramesHandler.java:29)
      2. water.api.FramesHandler.list(FramesHandler.java:116)
      2 frames
    4. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:606)
      4 frames
    5. water.api
      RequestServer.serve
      1. water.api.Handler.handle(Handler.java:55)
      2. water.api.RequestServer.handle(RequestServer.java:476)
      3. water.api.RequestServer.serve(RequestServer.java:454)
      3 frames
    6. water
      NanoHTTPD$HTTPSession.run
      1. water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:432)
      1 frame
    7. Java RT
      Thread.run
      1. java.lang.Thread.run(Thread.java:745)
      1 frame