org.apache.spark.scheduler.cluster.YarnScheduler: Lost executor 1 on cluster-1-w-0.c.justeat-datalake.internal: Container marked as failed: container_1470833273562_0003_01_000002 on host: cluster-1-w-0.c.justeat-datalake.internal. Exit status: 50. Diagnostics: Exception from container-launch. Container id: container_1470833273562_0003_01_000002 Exit code: 50 Stack trace: ExitCodeException exitCode=50:

GitHub | samelamin | 4 months ago
  1. 0

    Failing to save dataframe to bigquery

    GitHub | 4 months ago | samelamin
    org.apache.spark.scheduler.cluster.YarnScheduler: Lost executor 1 on cluster-1-w-0.c.justeat-datalake.internal: Container marked as failed: container_1470833273562_0003_01_000002 on host: cluster-1-w-0.c.justeat-datalake.internal. Exit status: 50. Diagnostics: Exception from container-launch. Container id: container_1470833273562_0003_01_000002 Exit code: 50 Stack trace: ExitCodeException exitCode=50:

    Root Cause Analysis

    1. org.apache.spark.scheduler.cluster.YarnScheduler

      Lost executor 1 on cluster-1-w-0.c.justeat-datalake.internal: Container marked as failed: container_1470833273562_0003_01_000002 on host: cluster-1-w-0.c.justeat-datalake.internal. Exit status: 50. Diagnostics: Exception from container-launch. Container id: container_1470833273562_0003_01_000002 Exit code: 50 Stack trace: ExitCodeException exitCode=50:

      at org.apache.hadoop.util.Shell.runCommand()
    2. Hadoop
      Shell$ShellCommandExecutor.execute
      1. org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
      2. org.apache.hadoop.util.Shell.run(Shell.java:456)
      3. org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
      3 frames
    3. hadoop-yarn-server-nodemanager
      ContainerLaunch.call
      1. org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
      2. org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
      3. org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
      3 frames
    4. Java RT
      Thread.run
      1. java.util.concurrent.FutureTask.run(FutureTask.java:266)
      2. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      3. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      4. java.lang.Thread.run(Thread.java:745)
      4 frames