Main$.createSparkSession() has thrown an IOException

java.io.IOException
Javadoc
Signals that an I/O exception of some sort has occurred. This class is the general class of exceptions produced by failed or interrupted I/O operations. @author unascribed @see java.io.InputStream @see java.io.OutputStream @since JDK1.0
at org.apache.hadoop.util.Shell.getQualifiedBinPath
(source)
at org.apache.hadoop.util.Shell.getWinUtilsPath
(source)
at org.apache.hadoop.util.Shell.<clinit>
at org.apache.hadoop.hive.conf.HiveConf$ConfVars.findHadoopBinary
at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>
at org.apache.hadoop.hive.conf.HiveConf.<clinit>
at java.lang.Class.forName0
(source)
at java.lang.Class.forName
(source)
at org.apache.spark.util.Utils$.classForName
at org.apache.spark.sql.SparkSession$.hiveClassesArePresent
at org.apache.spark.repl.Main$.createSparkSession
10 frames hidden

Typical Exception Messages

  1. Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

Origin

Class.forName() has thrown an IOException

java.io.IOException
7 frames hidden
java.lang.Class.forName
6Web pages

Similar

Specific cases

Solution

rp, 1 year ago

Download the winutils.exe for your Hadoop version: https://github.com/steveloughran/winutils .

Save it to HADOOP_HOME/bin

Check the source of this solution for more info

External results for this pattern (2)

  1. Unknown authorvia gitbooks.io3 months ago
    Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
    Show stack trace
  2. ziaul-haquevia GitHub1 year ago
    Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
    Show stack trace