org.apache.spark.sql.AnalysisException

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • org.apache.spark.sql.AnalysisException: Duplicate column(s) : "Int8", "String" found, cannot save to parquet format; at org.apache.spark.sql.execution.datasources.parquet.ParquetRelation.checkConstraints(ParquetRelation.scala:190) at org.apache.spark.sql.execution.datasources.parquet.ParquetRelation.dataSchema(ParquetRelation.scala:199) at org.apache.spark.sql.sources.HadoopFsRelation.schema$lzycompute(interfaces.scala:561) at org.apache.spark.sql.sources.HadoopFsRelation.schema(interfaces.scala:560) at org.apache.spark.sql.execution.datasources.LogicalRelation.<init>(LogicalRelation.scala:37) at org.apache.spark.sql.SQLContext.baseRelationToDataFrame(SQLContext.scala:395) at org.apache.spark.sql.DataFrameReader.parquet(DataFrameReader.scala:267) at org.apache.spark.sql.SQLContext.parquetFile(SQLContext.scala:1052)
No Bugmate found.