org.apache.hadoop.hive.ql.metadata.HiveException

Failed to create spark client.

Samebug tips1

This is a bug in some versions of the Arduino IDE. Try updating to the version 1.6.12 or further.


rafaelrafael

Don't give up yet. Our experts can help. Paste your full stack trace to get a solution.

Solutions on the web1894

  • via Unknown by Unknown author,
  • Stack trace

    • org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create spark client. at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.open(SparkSessionImpl.java:57) at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionManagerImpl.getSession(SparkSessionManagerImpl.java:116) at org.apache.hadoop.hive.ql.exec.spark.SparkUtilities.getSparkSession(SparkUtilities.java:113) at org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:95) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1638) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1397) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1183) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:145) at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:70) at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:197) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656) at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:209) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Cannot run program "/home/adt/server/spark1.5/spark-1.5.1set mapreduce.input.fileinputformat.split.maxsize=750000000set hive.vectorized.execution.enabled=trueset hive.cbo.enable=trueset hive.optimize.reducededuplication.min.reducer=4set hive.optimize.reducededuplication=trueset hive.orc.splits.include.file.footer=falseset hive.merge.mapfiles=trueset hive.merge.sparkfiles=falseset hive.merge.smallfiles.avgsize=16000000set hive.merge.size.per.task=256000000set hive.merge.orcfile.stripe.level=trueset hive.auto.convert.join=trueset hive.auto.convert.join.noconditionaltask=trueset hive.auto.convert.join.noconditionaltask.size=894435328set hive.optimize.bucketmapjoin.sortedmerge=falseset hive.map.aggr.hash.percentmemory=0.5set hive.map.aggr=trueset hive.optimize.sort.dynamic.partition=falseset hive.stats.autogather=trueset hive.stats.fetch.column.stats=trueset hive.vectorized.execution.reduce.enabled=falseset hive.vectorized.groupby.checkinterval=4096set hive.vectorized.groupby.flush.percent=0.1set hive.compute.query.using.stats=trueset hive.limit.pushdown.memory.usage=0.4set hive.optimize.index.filter=trueset hive.exec.reducers.bytes.per.reducer=67108864set hive.smbjoin.cache.rows=10000set hive.exec.orc.default.stripe.size=67108864set hive.fetch.task.conversion=moreset hive.fetch.task.conversion.threshold=1073741824set hive.fetch.task.aggr=falseset mapreduce.input.fileinputformat.list-status.num-threads=5set spark.kryo.referenceTracking=false#set spark.kryo.classesToRegister=org.apache.hadoop.hive.ql.io.HiveKey,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch/bin/spark-submit": error=36, File name too long at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) at org.apache.hive.spark.client.SparkClientImpl.startDriver(SparkClientImpl.java:376) at org.apache.hive.spark.client.SparkClientImpl.<init>(SparkClientImpl.java:89) at org.apache.hive.spark.client.SparkClientFactory.createClient(SparkClientFactory.java:80) at org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.<init>(RemoteHiveSparkClient.java:88) at org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactory.createHiveSparkClient(HiveSparkClientFactory.java:58) at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.open(SparkSessionImpl.java:55) ... 22 more Caused by: java.io.IOException: error=36, File name too long at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:248) at java.lang.ProcessImpl.start(ProcessImpl.java:134) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ... 28 more

    Write tip

    You have a different solution? A short tip here would help you and many other users who saw this issue last week.

    Users with the same issue

    rprp
    Once,
    Unknown visitor
    Unknown visitorOnce,
    rexgreenzarexgreenza
    8 times, last one
    Unknown visitor
    Unknown visitorOnce,
    Unknown visitor
    Unknown visitorOnce,
    151 more bugmates