org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias example

JIRA | Christian Schneider | 2 years ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    "ERROR 6000, Output location validation failed" using PIG MongoDB-Hadoop Connector on EMR

    Stack Overflow | 2 years ago | d0x
    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias example
  2. 0

    h1. Description This PIG Script should load a BSON file and save it into another file. {code} REGISTER /home/hadoop/lib/mongo-java-driver-2.13.0.jar REGISTER /home/hadoop/lib/mongo-hadoop-core-1.3.2.jar REGISTER /home/hadoop/lib/mongo-hadoop-pig-1.3.2.jar SET mapreduce.fileoutputcommitter.marksuccessfuljobs false; SET mapreduce.map.speculative false; SET mapreduce.reduce.speculative false; data = LOAD 's3://xxx/xxx.bson' USING com.mongodb.hadoop.pig.BSONLoader(); // That works great STORE data INTO 'file:///tmp/test.pig' USING PigStorage('\t'); // That fails with ERROR 6000 STORE data INTO 'file://tmp/whatever.bson' USING com.mongodb.hadoop.pig.BSONStorage; // That fails with ERROR 6000 STORE data INTO 's3://xxx/whatever.bson' USING com.mongodb.hadoop.pig.BSONStorage; {code} The failure I get is this one: {code} grunt> STORE example INTO 'file://tmp/whatever.bson' USING com.mongodb.hadoop.pig.BSONStorage; 2015-03-27 11:57:42,078 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: LIMIT 2015-03-27 11:57:42,079 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, NewPartitionFilterOptimizer, PartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter], RULES_DISABLED=[FilterLogicExpressionSimplifier]} 2015-03-27 11:57:42,089 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 6000: <line 10, column 0> Output Location Validation Failed for: 'file://tmp/whatever.bson More info to follow: Output directory not set. 2015-03-27 11:57:42,089 [main] ERROR org.apache.pig.tools.grunt.Grunt - org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias example at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1637) at org.apache.pig.PigServer.registerQuery(PigServer.java:577) at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1091) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) at org.apache.pig.Main.run(Main.java:543) at org.apache.pig.Main.main(Main.java:156) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: org.apache.pig.impl.plan.VisitorException: ERROR 6000: <line 10, column 0> Output Location Validation Failed for: 'file://tmp/whatever.bson More info to follow: Output directory not set. at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:95) at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:66) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66) at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53) at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52) at org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:317) at org.apache.pig.PigServer.compilePp(PigServer.java:1382) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307) at org.apache.pig.PigServer.execute(PigServer.java:1299) at org.apache.pig.PigServer.access$400(PigServer.java:124) at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1632) ... 13 more Caused by: org.apache.hadoop.mapred.InvalidJobConfException: Output directory not set. at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:138) at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80) ... 26 more {code} h1. Steps to reproduce: 1. Spawn a cluster on AWS using this bootstrap script: {code} #!/bin/sh wget -P /home/hadoop/lib http://central.maven.org/maven2/org/mongodb/mongo-java-driver/2.13.0/mongo-java-driver-2.13.0.jar wget -P /home/hadoop/lib https://github.com/mongodb/mongo-hadoop/releases/download/r1.3.2/mongo-hadoop-core-1.3.2.jar wget -P /home/hadoop/lib https://github.com/mongodb/mongo-hadoop/releases/download/r1.3.2/mongo-hadoop-pig-1.3.2.jar wget -P /home/hadoop/lib https://github.com/mongodb/mongo-hadoop/releases/download/r1.3.2/mongo-hadoop-hive-1.3.2.jar cp /home/hadoop/lib/mongo* /home/hadoop/pig/lib cp /home/hadoop/lib/mongo* /home/hadoop/hive/lib cp /home/hadoop/lib/mongo* /home/hadoop/share/hadoop/common/lib/ {code} You can have just one server (no Tasktracker and Core instnaces are requiered). 2. Upload any bson dump to s3 3. Execute the PIG script above The failure occurs with {{pig -v}} and {{pig -x local -v}}

    JIRA | 2 years ago | Christian Schneider
    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias example
  3. 0

    h1. Description This PIG Script should load a BSON file and save it into another file. {code} REGISTER /home/hadoop/lib/mongo-java-driver-2.13.0.jar REGISTER /home/hadoop/lib/mongo-hadoop-core-1.3.2.jar REGISTER /home/hadoop/lib/mongo-hadoop-pig-1.3.2.jar SET mapreduce.fileoutputcommitter.marksuccessfuljobs false; SET mapreduce.map.speculative false; SET mapreduce.reduce.speculative false; data = LOAD 's3://xxx/xxx.bson' USING com.mongodb.hadoop.pig.BSONLoader(); // That works great STORE data INTO 'file:///tmp/test.pig' USING PigStorage('\t'); // That fails with ERROR 6000 STORE data INTO 'file://tmp/whatever.bson' USING com.mongodb.hadoop.pig.BSONStorage; // That fails with ERROR 6000 STORE data INTO 's3://xxx/whatever.bson' USING com.mongodb.hadoop.pig.BSONStorage; {code} The failure I get is this one: {code} grunt> STORE example INTO 'file://tmp/whatever.bson' USING com.mongodb.hadoop.pig.BSONStorage; 2015-03-27 11:57:42,078 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: LIMIT 2015-03-27 11:57:42,079 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, NewPartitionFilterOptimizer, PartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter], RULES_DISABLED=[FilterLogicExpressionSimplifier]} 2015-03-27 11:57:42,089 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 6000: <line 10, column 0> Output Location Validation Failed for: 'file://tmp/whatever.bson More info to follow: Output directory not set. 2015-03-27 11:57:42,089 [main] ERROR org.apache.pig.tools.grunt.Grunt - org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias example at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1637) at org.apache.pig.PigServer.registerQuery(PigServer.java:577) at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1091) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) at org.apache.pig.Main.run(Main.java:543) at org.apache.pig.Main.main(Main.java:156) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: org.apache.pig.impl.plan.VisitorException: ERROR 6000: <line 10, column 0> Output Location Validation Failed for: 'file://tmp/whatever.bson More info to follow: Output directory not set. at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:95) at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:66) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66) at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53) at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52) at org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:317) at org.apache.pig.PigServer.compilePp(PigServer.java:1382) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307) at org.apache.pig.PigServer.execute(PigServer.java:1299) at org.apache.pig.PigServer.access$400(PigServer.java:124) at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1632) ... 13 more Caused by: org.apache.hadoop.mapred.InvalidJobConfException: Output directory not set. at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:138) at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80) ... 26 more {code} h1. Steps to reproduce: 1. Spawn a cluster on AWS using this bootstrap script: {code} #!/bin/sh wget -P /home/hadoop/lib http://central.maven.org/maven2/org/mongodb/mongo-java-driver/2.13.0/mongo-java-driver-2.13.0.jar wget -P /home/hadoop/lib https://github.com/mongodb/mongo-hadoop/releases/download/r1.3.2/mongo-hadoop-core-1.3.2.jar wget -P /home/hadoop/lib https://github.com/mongodb/mongo-hadoop/releases/download/r1.3.2/mongo-hadoop-pig-1.3.2.jar wget -P /home/hadoop/lib https://github.com/mongodb/mongo-hadoop/releases/download/r1.3.2/mongo-hadoop-hive-1.3.2.jar cp /home/hadoop/lib/mongo* /home/hadoop/pig/lib cp /home/hadoop/lib/mongo* /home/hadoop/hive/lib cp /home/hadoop/lib/mongo* /home/hadoop/share/hadoop/common/lib/ {code} You can have just one server (no Tasktracker and Core instnaces are requiered). 2. Upload any bson dump to s3 3. Execute the PIG script above The failure occurs with {{pig -v}} and {{pig -x local -v}}

    JIRA | 2 years ago | Christian Schneider
    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias example
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    “ERROR 6000, Output location validation failed” using PIG MongoDB-Hadoop Connector on EMR

    Google Groups | 2 years ago | Christian Schneider
    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias example
  6. 0

    problem using BSON Store

    Google Groups | 2 years ago | batman
    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2042: Error in new logical plan. Try -Dpig.usenewlogicalplan=false.

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. org.apache.hadoop.mapred.InvalidJobConfException

      Output directory not set.

      at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs()
    2. Hadoop
      FileOutputFormat.checkOutputSpecs
      1. org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:138)
      1 frame
    3. org.apache.pig
      Main.main
      1. org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80)
      2. org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:66)
      3. org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)
      4. org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
      5. org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
      6. org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)
      7. org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
      8. org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)
      9. org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:317)
      10. org.apache.pig.PigServer.compilePp(PigServer.java:1382)
      11. org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
      12. org.apache.pig.PigServer.execute(PigServer.java:1299)
      13. org.apache.pig.PigServer.access$400(PigServer.java:124)
      14. org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1632)
      15. org.apache.pig.PigServer.registerQuery(PigServer.java:577)
      16. org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1091)
      17. org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
      18. org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
      19. org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
      20. org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
      21. org.apache.pig.Main.run(Main.java:543)
      22. org.apache.pig.Main.main(Main.java:156)
      22 frames
    4. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:606)
      4 frames
    5. Hadoop
      RunJar.main
      1. org.apache.hadoop.util.RunJar.main(RunJar.java:212)
      1 frame