azkaban.jobtype.JobTypeManagerException: Failed to build job executor for job spark02

Google Groups | Unknown author | 2 months ago
  1. 0

    Spark job type support for azkaban 3.1 release

    Google Groups | 2 months ago | Unknown author
    azkaban.jobtype.JobTypeManagerException: Failed to build job executor for job spark02

    Root Cause Analysis

    1. azkaban.jobExecutor.utils.JobExecutionException

      Job type 'spark' is unrecognized. Could not construct job[{azkaban.job.attempt: 0, azkaban.job.attachment.file: /home/scienaptic/azkaban/azkaban-solo-server-3.1.0/executions/22/_job.22.spark02.attach, working.dir: /home/scienaptic/azkaban/azkaban-solo-server-3.1.0/executions/22, name: /usr/local/spark/latest/lib/spark-examples-1.6.1-hadoop2.6.0.jar, job.class: org.apache.spark.examples.SparkPi, jvm.args: -Dazkaban.flowid=spark02 -Dazkaban.execid=22 -Dazkaban.jobid=spark02, main.args: 100, type: spark, azkaban.job.outnodes: , azkaban.job.metadata.file: _job.22.spark02.meta, azkaban.memory.check: true, azkaban.job.innodes: , parent = {azkaban.flow.start.milliseconds: 316, azkaban.flow.start.year: 2016, azkaban.flow.start.timezone: America/Los_Angeles, azkaban.flow.projectid: 6, azkaban.flow.start.second: 58, azkaban.flow.submituser: spark, azkaban.flow.projectversion: 1, azkaban.flow.start.minute: 37, azkaban.flow.execid: 22, azkaban.flow.projectname: spark02, azkaban.flow.start.month: 09, azkaban.flow.start.hour: 00, azkaban.flow.uuid: b4595f0e-47f1-4ec1-b2e5-0d7886c6eb28, azkaban.flow.flowid: spark02, azkaban.flow.start.day: 28, azkaban.flow.projectlastchangedby: spark, azkaban.flow.projectlastchangeddate: 1474984995278, azkaban.flow.start.timestamp: 2016-09-28T00:37:58.316-07:00, }}] of type[spark].

      at azkaban.jobtype.JobTypeManager.buildJobExecutor()
    2. Azkaban
      JobTypeManager.buildJobExecutor
      1. azkaban.jobtype.JobTypeManager.buildJobExecutor(JobTypeManager.java:338)
      1 frame