Sahara Java Job

asked 2014-08-08 21:47:57 -0600

Nastooh gravatar image

updated 2014-08-08 21:48:38 -0600

On a 2 node cluster I am able, from command line, to run a simple wordcount java job, with the following xml and property files:

hadoop@Cluster121-NodeMasterGroup121-001:~$ cat wordcount/job.properties 
nameNode=hdfs://Cluster121-NodeMasterGroup121-001:8020
jobTracker=Cluster121-NodeMasterGroup121-001:8021
queueName=default

oozie.wf.application.path=${nameNode}/user/${user.name}/wordcount/workflow.xml


hadoop@Cluster121-NodeMasterGroup121-001:~$ cat wordcount/workflow.xml 
<workflow-app xmlns="uri:oozie:workflow:0.2" name="java-main-wf">
    <start to="java-node"/>
    <action name="java-node">
        <java>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <main-class>org.myorg.WordCount</main-class>
            <arg>/input</arg>
            <arg>/output22</arg>
        </java>
        <ok to="end"/>
        <error to="fail"/>
    </action>
    <kill name="fail">
        <message>Java failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

That is

hadoop@Cluster121-NodeMasterGroup121-001:~$ oozie job -oozie http://Cluster121-NodeMasterGroup121-001:11000/oozie -config wordcount/job.properties -run
job: 0000022-140808001255394-oozie-hado-W

However, running the same jar, as “java action”, in Sahara, results in java runtime error, indicating main entry was not found. Below are the xml, property and log file, as reported by oozie gui:
Job Definition:

<?xml version="1.0" ?>
<workflow-app name="job-wf" xmlns="uri:oozie:workflow:0.2">
  <start to="job-node"/>
  <action name="job-node">
    <java>
      <job-tracker>${jobTracker}</job-tracker>
      <name-node>${nameNode}</name-node>
      <configuration>
        <property>
          <name>mapred.job.queue.name</name>
          <value>default</value>
        </property>
      </configuration>
      <main-class>org.myorg.WordCount</main-class>
      <arg>/input</arg>
      <arg>/output222</arg>
    </java>
    <ok to="end"/>
    <error to="fail"/>
  </action>
  <kill name="fail">
    <message>Workflow failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
  </kill>
  <end name="end"/>
</workflow-app>

Job Configuration:

<configuration>
  <property>
    <name>user.name</name>
    <value>hadoop</value>
  </property>
  <property>
    <name>oozie.use.system.libpath</name>
    <value>true</value>
  </property>
  <property>
    <name>nameNode</name>
    <value>hdfs://Cluster121-NodeMasterGroup121-001:8020</value>
  </property>
  <property>
    <name>jobTracker</name>
    <value>Cluster121-NodeMasterGroup121-001:8021</value>
  </property>
  <property>
    <name>oozie.wf.application.path</name>
    <value>hdfs://Cluster121-NodeMasterGroup121-001:8020/user/hadoop/JavaJob01/6534c1f3-2723-4635-b9fc-553a18c19230/workflow.xml</value>
  </property>
</configuration>

Job Log:

2014-08-09 02:32:34,251  INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000023-140808001255394-oozie-hado-W] ACTION[0000023-140808001255394-oozie-hado-W@:start:] Start action [0000023-140808001255394-oozie-hado-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-08-09 02:32:34,251  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000023-140808001255394-oozie-hado-W] ACTION[0000023-140808001255394-oozie-hado-W@:start:] [***0000023-140808001255394-oozie-hado-W@:start:***]Action status=DONE
2014-08-09 02:32:34,251  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000023-140808001255394-oozie-hado-W] ACTION[0000023-140808001255394-oozie-hado-W@:start:] [***0000023-140808001255394-oozie-hado-W@:start:***]Action updated in DB!
2014-08-09 02:32:34,373  INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000023-140808001255394-oozie-hado-W] ACTION[0000023-140808001255394-oozie-hado-W@job-node] Start action [0000023-140808001255394-oozie-hado-W@job-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-08-09 02:32:34,498  WARN JavaActionExecutor:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000023-140808001255394-oozie-hado-W] ACTION[0000023-140808001255394-oozie-hado-W@job-node] credentials is null for the action
2014-08-09 02:32:35,009  INFO JavaActionExecutor:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000023-140808001255394-oozie-hado-W] ACTION[0000023-140808001255394-oozie-hado-W@job-node] checking action, external ID [job_201408080011_0029] status [RUNNING]
2014-08-09 02:32:35,012  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP ...
(more)
edit retag flag offensive close merge delete

Comments

we met the same issue using latest sahara, please update your resolution here if you find the solution. thanks!

9lives gravatar image9lives ( 2014-09-24 08:27:23 -0600 )edit