java.lang.IllegalArgumentException: Don't know how to save StructField(blah,StructType(StructField(blah,StructType(StructField(ApplicationNaStructField(foo,BooleanType,true)),true)),true) to JDBC

GitHub | samelamin | 8 months ago
tip
Do you know that we can give you better hits? Get more relevant results from Samebug’s stack trace search.
  1. 0

    GitHub comment 231#231815897

    GitHub | 8 months ago | samelamin
    java.lang.IllegalArgumentException: Don't know how to save StructField(blah,StructType(StructField(blah,StructType(StructField(ApplicationNaStructField(foo,BooleanType,true)),true)),true) to JDBC
  2. 0

    Write Spark dataframe to Redshift:save StructField(user_agent,ArrayType(StringType,true),true)

    Stack Overflow | 10 months ago | jduff1075
    java.lang.IllegalArgumentException: Don't know how to save StructField(user_agent,ArrayType(StringType,true),true) to JDBC

    Root Cause Analysis

    1. java.lang.IllegalArgumentException

      Don't know how to save StructField(blah,StructType(StructField(blah,StructType(StructField(ApplicationNaStructField(foo,BooleanType,true)),true)),true) to JDBC

      at com.databricks.spark.redshift.JDBCWrapper$$anonfun$schemaString$1.apply()
    2. com.databricks.spark
      JDBCWrapper$$anonfun$schemaString$1.apply
      1. com.databricks.spark.redshift.JDBCWrapper$$anonfun$schemaString$1.apply(RedshiftJDBCWrapper.scala:262)
      2. com.databricks.spark.redshift.JDBCWrapper$$anonfun$schemaString$1.apply(RedshiftJDBCWrapper.scala:242)
      2 frames
    3. Scala
      ArrayOps$ofRef.foreach
      1. scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
      2. scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
      2 frames
    4. com.databricks.spark
      DefaultSource.createRelation
      1. com.databricks.spark.redshift.JDBCWrapper.schemaString(RedshiftJDBCWrapper.scala:242)
      2. com.databricks.spark.redshift.RedshiftWriter.createTableSql(RedshiftWriter.scala:75)
      3. com.databricks.spark.redshift.RedshiftWriter.com$databricks$spark$redshift$RedshiftWriter$$doRedshiftLoad(RedshiftWriter.scala:160)
      4. com.databricks.spark.redshift.RedshiftWriter$$anonfun$saveToRedshift$1.apply(RedshiftWriter.scala:378)
      5. com.databricks.spark.redshift.RedshiftWriter$$anonfun$saveToRedshift$1.apply(RedshiftWriter.scala:376)
      6. com.databricks.spark.redshift.RedshiftWriter.withStagingTable(RedshiftWriter.scala:121)
      7. com.databricks.spark.redshift.RedshiftWriter.saveToRedshift(RedshiftWriter.scala:376)
      8. com.databricks.spark.redshift.DefaultSource.createRelation(DefaultSource.scala:106)
      8 frames
    5. org.apache.spark
      ResolvedDataSource$.apply
      1. org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:222)
      1 frame