java.io.IOException: com.mongodb.hadoop.splitter.SplitFailedException: Unable to calculate input splits: doing a splitVector across dbs isn't supported via mongos

JIRA | James Wahlin | 3 years ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    When running the mongo-hadoop example script {{examples/treasury_yield/run_job.sh}} against a {{mongos}} instance it fails with the following exception: {code} java.io.IOException: com.mongodb.hadoop.splitter.SplitFailedException: Unable to calculate input splits: doing a splitVector across dbs isn't supported via mongos at com.mongodb.hadoop.MongoInputFormat.getSplits(MongoInputFormat.java:53) at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:491) at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:508) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:392) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286) at com.mongodb.hadoop.util.MongoTool.run(MongoTool.java:102) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at com.mongodb.hadoop.examples.treasury.TreasuryYieldXMLConfig.main(TreasuryYieldXMLConfig.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: com.mongodb.hadoop.splitter.SplitFailedException: Unable to calculate input splits: doing a splitVector across dbs isn't supported via mongos at com.mongodb.hadoop.splitter.StandaloneMongoSplitter.calculateSplits(StandaloneMongoSplitter.java:79) at com.mongodb.hadoop.MongoInputFormat.getSplits(MongoInputFormat.java:51) ... 19 more {code} It appears that the mongo-hadoop connector is running splitVector against the {{admin}} database and mongos rejects because it is not the same database the split is being executed against. mongod allows this but mongos rejects. Instead we should run this command against the same database passed to the splitVector command. splitVector command execution is not restricted to admin.

    JIRA | 3 years ago | James Wahlin
    java.io.IOException: com.mongodb.hadoop.splitter.SplitFailedException: Unable to calculate input splits: doing a splitVector across dbs isn't supported via mongos
  2. 0

    When running the mongo-hadoop example script {{examples/treasury_yield/run_job.sh}} against a {{mongos}} instance it fails with the following exception: {code} java.io.IOException: com.mongodb.hadoop.splitter.SplitFailedException: Unable to calculate input splits: doing a splitVector across dbs isn't supported via mongos at com.mongodb.hadoop.MongoInputFormat.getSplits(MongoInputFormat.java:53) at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:491) at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:508) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:392) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286) at com.mongodb.hadoop.util.MongoTool.run(MongoTool.java:102) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at com.mongodb.hadoop.examples.treasury.TreasuryYieldXMLConfig.main(TreasuryYieldXMLConfig.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: com.mongodb.hadoop.splitter.SplitFailedException: Unable to calculate input splits: doing a splitVector across dbs isn't supported via mongos at com.mongodb.hadoop.splitter.StandaloneMongoSplitter.calculateSplits(StandaloneMongoSplitter.java:79) at com.mongodb.hadoop.MongoInputFormat.getSplits(MongoInputFormat.java:51) ... 19 more {code} It appears that the mongo-hadoop connector is running splitVector against the {{admin}} database and mongos rejects because it is not the same database the split is being executed against. mongod allows this but mongos rejects. Instead we should run this command against the same database passed to the splitVector command. splitVector command execution is not restricted to admin.

    JIRA | 3 years ago | James Wahlin
    java.io.IOException: com.mongodb.hadoop.splitter.SplitFailedException: Unable to calculate input splits: doing a splitVector across dbs isn't supported via mongos

    Root Cause Analysis

    1. com.mongodb.hadoop.splitter.SplitFailedException

      Unable to calculate input splits: doing a splitVector across dbs isn't supported via mongos

      at com.mongodb.hadoop.splitter.StandaloneMongoSplitter.calculateSplits()
    2. com.mongodb.hadoop
      MongoInputFormat.getSplits
      1. com.mongodb.hadoop.splitter.StandaloneMongoSplitter.calculateSplits(StandaloneMongoSplitter.java:79)
      2. com.mongodb.hadoop.MongoInputFormat.getSplits(MongoInputFormat.java:51)
      2 frames
    3. Hadoop
      Job$10.run
      1. org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:491)
      2. org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:508)
      3. org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:392)
      4. org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
      5. org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
      5 frames
    4. Java RT
      Subject.doAs
      1. java.security.AccessController.doPrivileged(Native Method)
      2. javax.security.auth.Subject.doAs(Subject.java:415)
      2 frames
    5. Hadoop
      UserGroupInformation.doAs
      1. org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
      1 frame
    6. Hadoop
      Job.waitForCompletion
      1. org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
      2. org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
      2 frames
    7. com.mongodb.hadoop
      MongoTool.run
      1. com.mongodb.hadoop.util.MongoTool.run(MongoTool.java:102)
      1 frame
    8. Hadoop
      ToolRunner.run
      1. org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      2. org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
      2 frames
    9. com.mongodb.hadoop
      TreasuryYieldXMLConfig.main
      1. com.mongodb.hadoop.examples.treasury.TreasuryYieldXMLConfig.main(TreasuryYieldXMLConfig.java:38)
      1 frame
    10. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:601)
      4 frames
    11. Hadoop
      RunJar.main
      1. org.apache.hadoop.util.RunJar.main(RunJar.java:212)
      1 frame