org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://host_hadoop:8020/messages

JIRA | Thomas Mager | 3 months ago
  1. 0

    I tried to do the Tutorial called EnronMail for the Hadoop MongoDB Connector. Therefore I included the MongoDB jars and the Connector jars in the Hadoop Classpath I created. The compiling of the job, which is part of the tutorial, worked errorless, so the jars are correct and are working fine. But I still get an error message, when I try to run the job. As a side remark: On MongoDB there is the database test with the collections enron_mail, restaurants and testData. My execution command is: hadoop jar EnronMail.jar EnronMail \ ${HADOOP_CLASSPATH} -libjars parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hadoop-0.20-mapreduce/lib/mongodb-driver-3.0.0.jar, parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hadoop-0.20-mapreduce/lib/mongo-hadoop-core-1.5.2.jar \ -Dmongo.input.uri=mongodb://User:Password@host_mongo:port/test.enron_mail \ -Dmongo.output.uri=mongodb://User:Password@host_mongo:port/test.enron_mail The error message is the following: 16/09/14 11:46:09 INFO util.MongoTool: Created a conf: 'Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-site.xml' on {class EnronMail} as job named 'EnronMail' 16/09/14 11:46:09 INFO util.MongoTool: Setting up and running MapReduce job in foreground, will wait for results. {Verbose? false} 16/09/14 11:46:10 INFO client.RMProxy: Connecting to ResourceManager at host_hadoop/10.58.121.65:8032 16/09/14 11:46:27 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/hdfs/.staging/job_1461168245937_0116 16/09/14 11:46:27 WARN security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://host_hadoop:8020/messages 16/09/14 11:46:27 ERROR util.MongoTool: Exception while executing job... org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://host_hadoop:8020/messages at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323) at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265) at com.mongodb.hadoop.BSONFileInputFormat.getSplits(BSONFileInputFormat.java:99) at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:305) at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:322) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1325) at com.mongodb.hadoop.util.MongoTool.runMapReduceJob(MongoTool.java:230) at com.mongodb.hadoop.util.MongoTool.run(MongoTool.java:100) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at EnronMail.main(EnronMail.java:41) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

    JIRA | 3 months ago | Thomas Mager
    org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://host_hadoop:8020/messages
  2. 0

    Apache Pig : Job in state DEFINE instead of RUNNING

    Stack Overflow | 10 months ago | Kaushik Lele
    org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: hdfs: //ip-172-31-53-48.ec2.internal:8020/user/e1681fe26eed362777aabca1682510/PigTestFile
  3. 0

    adam plugin calls ListDict instead of PluginExecutor

    GitHub | 3 years ago | heuermh
    org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/Users/xxx/working/adam/com.foo.MyPluginClass
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    hadoop bzip2 xml inputformat

    Stack Overflow | 3 years ago | Amit Patil
    org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://localhost/xmlinputformat/_xmlinputformat_sample.xml.bz2
  6. 0

    Input path does not exist: file:/D:/pigsample_1749383998_1377684507424

    Stack Overflow | 3 years ago | java_enthu
    org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/D:/pigsample_1288777582_1377684802262

  1. Nikolay Rybak 1 times, last 1 month ago
1 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. org.apache.hadoop.mapreduce.lib.input.InvalidInputException

    Input path does not exist: hdfs://host_hadoop:8020/messages

    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus()
  2. Hadoop
    FileInputFormat.listStatus
    1. org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
    2. org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
    2 frames
  3. com.mongodb.hadoop
    BSONFileInputFormat.getSplits
    1. com.mongodb.hadoop.BSONFileInputFormat.getSplits(BSONFileInputFormat.java:99)
    1 frame
  4. Hadoop
    Job$10.run
    1. org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:305)
    2. org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:322)
    3. org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
    4. org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
    5. org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
    5 frames
  5. Java RT
    Subject.doAs
    1. java.security.AccessController.doPrivileged(Native Method)
    2. javax.security.auth.Subject.doAs(Subject.java:415)
    2 frames
  6. Hadoop
    UserGroupInformation.doAs
    1. org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
    1 frame
  7. Hadoop
    Job.waitForCompletion
    1. org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
    2. org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1325)
    2 frames
  8. com.mongodb.hadoop
    MongoTool.run
    1. com.mongodb.hadoop.util.MongoTool.runMapReduceJob(MongoTool.java:230)
    2. com.mongodb.hadoop.util.MongoTool.run(MongoTool.java:100)
    2 frames
  9. Hadoop
    ToolRunner.run
    1. org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    2. org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    2 frames
  10. Unknown
    EnronMail.main
    1. EnronMail.main(EnronMail.java:41)
    1 frame
  11. Java RT
    Method.invoke
    1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    4. java.lang.reflect.Method.invoke(Method.java:606)
    4 frames
  12. Hadoop
    RunJar.main
    1. org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    2. org.apache.hadoop.util.RunJar.main(RunJar.java:136)
    2 frames