org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move source hdfs://mycluster/mapred/INPUT/.hive-staging_hive_2016-10-17_14-57-48_620_6609613978089243090-1/-ext-10000 to destination /mapred/INPUT

Stack Overflow | Ranjan Swain | 3 months ago
  1. 0

    Hive INSERT OVERWRITE query not writing to hdfs directory:Cannot get DistCp constructor

    Stack Overflow | 3 months ago | Ranjan Swain
    org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move source hdfs://mycluster/mapred/INPUT/.hive-staging_hive_2016-10-17_14-57-48_620_6609613978089243090-1/-ext-10000 to destination /mapred/INPUT
  2. 0
    samebug tip
    Check for bad records in the input data (like '(null)')
  3. 0
    samebug tip
    Download the winutils.exe for your Hadoop version: https://github.com/steveloughran/winutils . Save it to HADOOP_HOME/bin
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0
    samebug tip
    Bad input data (not properly separated)
  6. 0
    samebug tip
    If you are using maven you'd like to exclude your binaries from filtering process.

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.io.IOException

      Cannot get DistCp constructor: org.apache.hadoop.tools.DistCp.()

      at org.apache.hadoop.hive.shims.Hadoop23Shims.runDistCp()
    2. Hive Shims
      Hadoop23Shims.runDistCp
      1. org.apache.hadoop.hive.shims.Hadoop23Shims.runDistCp(Hadoop23Shims.java:1160)
      1 frame
    3. Hive Common
      FileUtils.copy
      1. org.apache.hadoop.hive.common.FileUtils.copy(FileUtils.java:553)
      1 frame
    4. Hive Query Language
      Driver.run
      1. org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2622)
      2. org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:105)
      3. org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:222)
      4. org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
      5. org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
      6. org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1653)
      7. org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1412)
      8. org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
      9. org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
      10. org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
      10 frames
    5. org.apache.hadoop
      CliDriver.main
      1. org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
      2. org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
      3. org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
      4. org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
      5. org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
      6. org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
      6 frames
    6. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:498)
      4 frames
    7. Hadoop
      RunJar.main
      1. org.apache.hadoop.util.RunJar.run(RunJar.java:221)
      2. org.apache.hadoop.util.RunJar.main(RunJar.java:136)
      2 frames