java.util.NoSuchElementException: key not found: StructType(StructField(user,IntegerType,false), StructField(product,IntegerType,false), StructField(rating,DoubleType,false))

Stack Overflow | mvg | 2 months ago
tip
Do you find the tips below useful? Click on the to mark them and say thanks to poroszd . Or join the community to write better ones.
  1. 0

    Integrating Spark MLLib algorithm to H2O ai using Sparkling water

    Stack Overflow | 2 months ago | mvg
    java.util.NoSuchElementException: key not found: StructType(StructField(user,IntegerType,false), StructField(product,IntegerType,false), StructField(rating,DoubleType,false))
  2. 0
    samebug tip
    You should use java.sql.Timestamp or Date to map BsonDateTime from mongodb.
  3. Speed up your debug routine!

    Automated exception search integrated into your IDE

  4. 0

    Execute: {noformat} df.select("created_date").first() Row(created_date=datetime.date(2006, 10, 3)) context.as_h2o_frame(df.select("created_date")) {noformat} THrows {noformat} Py4JJavaError: An error occurred while calling o70.asH2OFrame. : java.util.NoSuchElementException: key not found: DateType at scala.collection.MapLike$class.default(MapLike.scala:228) at scala.collection.AbstractMap.default(Map.scala:59) at scala.collection.MapLike$class.apply(MapLike.scala:141) at scala.collection.AbstractMap.apply(Map.scala:59) at org.apache.spark.h2o.utils.ReflectionUtils$.supportedTypeOf(ReflectionUtils.scala:110) at org.apache.spark.h2o.converters.SparkDataFrameConverter$$anonfun$4.apply(SparkDataFrameConverter.scala:70) at org.apache.spark.h2o.converters.SparkDataFrameConverter$$anonfun$4.apply(SparkDataFrameConverter.scala:70) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.Iterator$class.foreach(Iterator.scala:893) at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at org.apache.spark.h2o.converters.SparkDataFrameConverter$.toH2OFrame(SparkDataFrameConverter.scala:70) at org.apache.spark.h2o.H2OContext.asH2OFrame(H2OContext.scala:132) at org.apache.spark.h2o.H2OContext.asH2OFrame(H2OContext.scala:133) at org.apache.spark.h2o.JavaH2OContext.asH2OFrame(JavaH2OContext.java:100) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:237) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:280) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:214) at java.lang.Thread.run(Thread.java:745) {noformat}

    JIRA | 3 months ago | Michal Malohlava
    java.util.NoSuchElementException: key not found: DateType
  5. 0

    I am getting an error converting a sparkframe with date types to h2o: df.select("created_date").first() Row(created_date=datetime.date(2006, 10, 3)) context.as_h2o_frame(df.select("created_date")) --------------------------------------------------------------------------- Py4JJavaError Traceback (most recent call last) <ipython-input-9-b6ba696f5e41> in <module>() ----> 1 context.as_h2o_frame(df.select("created_date")) /opt/sparkling-water/2.0.3/py/build/dist/h2o_pysparkling_2.0-2.0.3-py2.7.egg/pysparkling/context.pyc in as_h2o_frame(self, dataframe, framename) 175 """ 176 if isinstance(dataframe, DataFrame): --> 177 return fc._as_h2o_frame_from_dataframe(self, dataframe, framename) 178 elif isinstance(dataframe, RDD): 179 # First check if the type T in RDD[T] is one of the python "primitive" types /opt/sparkling-water/2.0.3/py/build/dist/h2o_pysparkling_2.0-2.0.3-py2.7.egg/pysparkling/conversions.pyc in _as_h2o_frame_from_dataframe(h2oContext, dataframe, frame_name) 42 if dataframe.count() == 0: 43 raise ValueError('Cannot transform empty H2OFrame') ---> 44 j_h2o_frame = h2oContext._jhc.asH2OFrame(dataframe._jdf, frame_name) 45 j_h2o_frame_key = j_h2o_frame.key() 46 return H2OFrame.from_java_h2o_frame(j_h2o_frame,j_h2o_frame_key) /opt/spark/2.0.2/python/lib/py4j-src.zip/py4j/java_gateway.py in __call__(self, *args) 1131 answer = self.gateway_client.send_command(command) 1132 return_value = get_return_value( -> 1133 answer, self.gateway_client, self.target_id, self.name) 1134 1135 for temp_arg in temp_args: /opt/spark/2.0.2/python/pyspark/sql/utils.py in deco(*a, **kw) 61 def deco(*a, **kw): 62 try: ---> 63 return f(*a, **kw) 64 except py4j.protocol.Py4JJavaError as e: 65 s = e.java_exception.toString() /opt/spark/2.0.2/python/lib/py4j-src.zip/py4j/protocol.py in get_return_value(answer, gateway_client, target_id, name) 317 raise Py4JJavaError( 318 "An error occurred while calling {0}{1}{2}.\n". --> 319 format(target_id, ".", name), value) 320 else: 321 raise Py4JError( Py4JJavaError: An error occurred while calling o70.asH2OFrame. : java.util.NoSuchElementException: key not found: DateType at scala.collection.MapLike$class.default(MapLike.scala:228) at scala.collection.AbstractMap.default(Map.scala:59) at scala.collection.MapLike$class.apply(MapLike.scala:141) at scala.collection.AbstractMap.apply(Map.scala:59) at org.apache.spark.h2o.utils.ReflectionUtils$.supportedTypeOf(ReflectionUtils.scala:110) at org.apache.spark.h2o.converters.SparkDataFrameConverter$$anonfun$4.apply(SparkDataFrameConverter.scala:70) at org.apache.spark.h2o.converters.SparkDataFrameConverter$$anonfun$4.apply(SparkDataFrameConverter.scala:70) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.Iterator$class.foreach(Iterator.scala:893) at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at org.apache.spark.h2o.converters.SparkDataFrameConverter$.toH2OFrame(SparkDataFrameConverter.scala:70) at org.apache.spark.h2o.H2OContext.asH2OFrame(H2OContext.scala:132) at org.apache.spark.h2o.H2OContext.asH2OFrame(H2OContext.scala:133) at org.apache.spark.h2o.JavaH2OContext.asH2OFrame(JavaH2OContext.java:100) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:237) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:280) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:214) at java.lang.Thread.run(Thread.java:745) But the error is caused by the fact that we don’t have handling for Spark DateType here https://github.com/h2oai/sparkling-water/blob/master/core/src/main/scala/org/apache/spark/h2o/utils/SupportedTypes.scala#L77

    JIRA | 3 months ago | Avkash Chauhan
    java.util.NoSuchElementException: key not found: DateType
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.util.NoSuchElementException

    key not found: StructType(StructField(user,IntegerType,false), StructField(product,IntegerType,false), StructField(rating,DoubleType,false))

    at scala.collection.MapLike$class.default()
  2. Scala
    AbstractMap.apply
    1. scala.collection.MapLike$class.default(MapLike.scala:228)
    2. scala.collection.AbstractMap.default(Map.scala:59)
    3. scala.collection.MapLike$class.apply(MapLike.scala:141)
    4. scala.collection.AbstractMap.apply(Map.scala:59)
    4 frames
  3. org.apache.spark
    SparkDataFrameConverter$$anonfun$3.apply
    1. org.apache.spark.h2o.utils.ReflectionUtils$.vecTypeFor(ReflectionUtils.scala:132)
    2. org.apache.spark.h2o.converters.SparkDataFrameConverter$$anonfun$3.apply(SparkDataFrameConverter.scala:68)
    3. org.apache.spark.h2o.converters.SparkDataFrameConverter$$anonfun$3.apply(SparkDataFrameConverter.scala:68)
    3 frames
  4. Scala
    AbstractTraversable.map
    1. scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    2. scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    3. scala.collection.Iterator$class.foreach(Iterator.scala:893)
    4. scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
    5. scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
    6. scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    7. scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
    8. scala.collection.AbstractTraversable.map(Traversable.scala:104)
    8 frames
  5. org.apache.spark
    H2OContext.asH2OFrame
    1. org.apache.spark.h2o.converters.SparkDataFrameConverter$.toH2OFrame(SparkDataFrameConverter.scala:68)
    2. org.apache.spark.h2o.H2OContext.asH2OFrame(H2OContext.scala:132)
    3. org.apache.spark.h2o.H2OContext.asH2OFrame(H2OContext.scala:130)
    3 frames
  6. com.poc.sample
    RecommendataionAlgo.main
    1. com.poc.sample.RecommendataionAlgo$.main(RecommendataionAlgo.scala:54)
    2. com.poc.sample.RecommendataionAlgo.main(RecommendataionAlgo.scala)
    2 frames
  7. Java RT
    Method.invoke
    1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    4. java.lang.reflect.Method.invoke(Method.java:498)
    4 frames
  8. IDEA
    AppMain.main
    1. com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
    1 frame