java.lang.IllegalArgumentException: Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path

Spring JIRA | Nachiket Bondale | 1 year ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    ***Version Spring XD Version : spring-xd-1.3.0.RELEASE, spring-xd-1.3.0.RELEASE-yarn OS & Version: Linux 2.6.32-431.29.2.el6.x86_64 Java Version: java version "1.7.0_65" ***Description The simple word count map reduce job using spring xd is failing with inline error message. ***Steps to recreate the problem 1. Created a jar for simple word count map reduce job. 2. Created jar using information given in ( http://docs.spring.io/spring-hadoop/docs/2.0.2.RELEASE/reference/html/hadoop.html#hadoop:tasklet ) 3. Once the final jar was ready, uploaded using "module upload --name test_mr_module --type job --file /home/user/jar/samplemrjob.jar" 4. After that created and deployed job using "job create --name test_mr_job --definition test_mr_module --deploy" 5. Finally launched using "job launch test_mr_job" which failed with inline error. ***Describe XD Deployment : Distributed Deployment Type : Distributed - YARN ( on AWS EC2 cloud ) Number of xd-admin’s and xd-container’s : 1 Admin and 3 Containers ***Describe Other Components Transport: Redis 3.0.1 ZooKeeper: Version 3.4.6.2.3.2.0-2950 Hadoop deployment Data Platform : Hortonworks HDP 2.3.2.0-2950 RDBMS: MySQL ***Error Message: ***************************************************** 05:29:52,673 INFO DeploymentsPathChildrenCache-0 container.DeploymentListener - Deploying job 'test_mr_job' 05:29:53,655 INFO DeploymentsPathChildrenCache-0 container.DeploymentListener - Deploying module [ModuleDescriptor@6e5af900 moduleName = 'test_mr_module', moduleLabel = 'test_mr_module', group = 'test_mr_job', sourceChannelName = [null], sinkChannelName = [null], index = 0, type = job, parameters = map[[empty]], children = list[[empty]]] 05:30:24,351 ERROR inbound.job:test_mr_job-redis:queue-inbound-channel-adapter1 step.AbstractStep - Encountered an error executing step teststep in job test_mr_job java.lang.IllegalArgumentException: Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:443) . . . at org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint.access$300(RedisQueueMessageDrivenEndpoint.java:54) at org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint$ListenerTask.run(RedisQueueMessageDrivenEndpoint.java:323) at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.URISyntaxException: Illegal character in path at index 11: /hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework at java.net.URI$Parser.fail(URI.java:2848) at java.net.URI$Parser.checkChars(URI.java:3021) at java.net.URI$Parser.parseHierarchical(URI.java:3105) at java.net.URI$Parser.parse(URI.java:3063) at java.net.URI.<init>(URI.java:588) at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:441) *****************************************************

    Spring JIRA | 1 year ago | Nachiket Bondale
    java.lang.IllegalArgumentException: Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path
  2. 0

    ***Version Spring XD Version : spring-xd-1.3.0.RELEASE, spring-xd-1.3.0.RELEASE-yarn OS & Version: Linux 2.6.32-431.29.2.el6.x86_64 Java Version: java version "1.7.0_65" ***Description The simple word count map reduce job using spring xd is failing with inline error message. ***Steps to recreate the problem 1. Created a jar for simple word count map reduce job. 2. Created jar using information given in ( http://docs.spring.io/spring-hadoop/docs/2.0.2.RELEASE/reference/html/hadoop.html#hadoop:tasklet ) 3. Once the final jar was ready, uploaded using "module upload --name test_mr_module --type job --file /home/user/jar/samplemrjob.jar" 4. After that created and deployed job using "job create --name test_mr_job --definition test_mr_module --deploy" 5. Finally launched using "job launch test_mr_job" which failed with inline error. ***Describe XD Deployment : Distributed Deployment Type : Distributed - YARN ( on AWS EC2 cloud ) Number of xd-admin’s and xd-container’s : 1 Admin and 3 Containers ***Describe Other Components Transport: Redis 3.0.1 ZooKeeper: Version 3.4.6.2.3.2.0-2950 Hadoop deployment Data Platform : Hortonworks HDP 2.3.2.0-2950 RDBMS: MySQL ***Error Message: ***************************************************** 05:29:52,673 INFO DeploymentsPathChildrenCache-0 container.DeploymentListener - Deploying job 'test_mr_job' 05:29:53,655 INFO DeploymentsPathChildrenCache-0 container.DeploymentListener - Deploying module [ModuleDescriptor@6e5af900 moduleName = 'test_mr_module', moduleLabel = 'test_mr_module', group = 'test_mr_job', sourceChannelName = [null], sinkChannelName = [null], index = 0, type = job, parameters = map[[empty]], children = list[[empty]]] 05:30:24,351 ERROR inbound.job:test_mr_job-redis:queue-inbound-channel-adapter1 step.AbstractStep - Encountered an error executing step teststep in job test_mr_job java.lang.IllegalArgumentException: Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:443) . . . at org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint.access$300(RedisQueueMessageDrivenEndpoint.java:54) at org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint$ListenerTask.run(RedisQueueMessageDrivenEndpoint.java:323) at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.URISyntaxException: Illegal character in path at index 11: /hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework at java.net.URI$Parser.fail(URI.java:2848) at java.net.URI$Parser.checkChars(URI.java:3021) at java.net.URI$Parser.parseHierarchical(URI.java:3105) at java.net.URI$Parser.parse(URI.java:3063) at java.net.URI.<init>(URI.java:588) at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:441) *****************************************************

    Spring JIRA | 1 year ago | Nachiket Bondale
    java.lang.IllegalArgumentException: Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path

    Root Cause Analysis

    1. java.lang.IllegalArgumentException

      Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path

      at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache()
    2. Hadoop
      JobSubmitter.addMRFrameworkToDistributedCache
      1. org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:443)
      1 frame