Revision history [back]

click to hide/show revision 1
initial version

savanna 0.3 excuting pig script encounter error

script: A = load '$INPUT' as (line); B = foreach A generate flatten(TOKENIZE(line)) as word; C = group B by word; D = foreach C generate group, COUNT(B); store D into '$OUTPUT';

there is no any exceptions ocurr in jobtacker,namenode,tasktracker or datanode logs. the only exceptions which i can find in /opt/oozie/logs/oozie.log is as follows:

2014-02-28 06:12:49,347 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time = 2014-02-28T06:11Z 2014-02-28 06:12:49,350 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService] 2014-02-28 06:13:09,664 INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService] 2014-02-28 06:13:09,687 INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService] 2014-02-28 06:13:49,351 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService] 2014-02-28 06:13:49,352 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time = 2014-02-28T06:12Z 2014-02-28 06:13:49,355 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time = 2014-02-28T06:12Z 2014-02-28 06:13:49,358 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService] 2014-02-28 06:14:09,689 INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService] 2014-02-28 06:14:09,712 INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService] 2014-02-28 06:14:49,358 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService] 2014-02-28 06:14:49,359 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time = 2014-02-28T06:13Z 2014-02-28 06:14:49,362 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time = 2014-02-28T06:13Z 2014-02-28 06:14:49,366 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService] 2014-02-28 06:15:05,358 WARN LiteWorkflowAppService:542 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] libpath [hdfs://v-ndhoo-v-nn-001:8020/user/hadoop/worldcount/4496286e-d5e0-42fe-8842-800bace75f54/lib] does not exist 2014-02-28 06:15:05,499 INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@:start:] Start action [0000004-140227105121167-oozie-hado-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2014-02-28 06:15:05,499 WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@:start:] [0000004-140227105121167-oozie-hado-W@:start:]Action status=DONE 2014-02-28 06:15:05,500 WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@:start:] [0000004-140227105121167-oozie-hado-W@:start:]Action updated in DB! 2014-02-28 06:15:05,602 INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] Start action [0000004-140227105121167-oozie-hado-W@job-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2014-02-28 06:15:06,029 WARN PigActionExecutor:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] credentials is null for the action 2014-02-28 06:15:06,475 WARN JobClient:746 - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 2014-02-28 06:15:06,575 WARN JobClient:870 - No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String). 2014-02-28 06:15:06,820 INFO PigActionExecutor:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] checking action, external ID [job_201402271128_0005] status [RUNNING] 2014-02-28 06:15:06,825 WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] [0000004-140227105121167-oozie-hado-W@job-node]Action status=RUNNING 2014-02-28 06:15:06,826 WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] [0000004-140227105121167-oozie-hado-W@job-node]Action updated in DB! 2014-02-28 06:15:09,712 INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService] 2014-02-28 06:15:09,728 INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService] 2014-02-28 06:15:19,450 INFO CallbackServlet:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] callback for action [0000004-140227105121167-oozie-hado-W@job-node] 2014-02-28 06:15:19,531 INFO PigActionExecutor:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] action completed, external ID [job_201402271128_0005]


2014-02-28 06:15:19,558 WARN PigActionExecutor:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]

2014-02-28 06:15:19,659 INFO ActionEndXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] ERROR is considered as FAILED for SLA 2014-02-28 06:15:19,762 INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@fail] Start action [0000004-140227105121167-oozie-hado-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2014-02-28 06:15:19,767 WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@fail] [0000004-140227105121167-oozie-hado-W@fail]Action status=DONE 2014-02-28 06:15:19,767 WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@fail] [0000004-140227105121167-oozie-hado-W@fail]Action updated in DB! 2014-02-28 06:15:19,855 WARN CoordActionUpdateXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100 2014-02-28 06:15:49,366 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService]

savanna 0.3 excuting pig script encounter error

script:
    A = load '$INPUT'  as (line);
    B = foreach A generate flatten(TOKENIZE(line)) as word; 
   C = group B by word;
   D = foreach C generate group, COUNT(B);
   store D into '$OUTPUT';

'$OUTPUT';

there is no any exceptions ocurr in jobtacker,namenode,tasktracker or datanode logs. the only exceptions which i can find in /opt/oozie/logs/oozie.log is as follows:

2014-02-28 06:12:49,347  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time =  2014-02-28T06:11Z
2014-02-28 06:12:49,350  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 06:13:09,664  INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:13:09,687  INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:13:49,351  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 06:13:49,352  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time =  2014-02-28T06:12Z
2014-02-28 06:13:49,355  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time =  2014-02-28T06:12Z
2014-02-28 06:13:49,358  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 06:14:09,689  INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:14:09,712  INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:14:49,358  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 06:14:49,359  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time =  2014-02-28T06:13Z
2014-02-28 06:14:49,362  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time =  2014-02-28T06:13Z
2014-02-28 06:14:49,366  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 06:15:05,358  WARN LiteWorkflowAppService:542 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] libpath [hdfs://v-ndhoo-v-nn-001:8020/user/hadoop/worldcount/4496286e-d5e0-42fe-8842-800bace75f54/lib] does not exist
2014-02-28 06:15:05,499  INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@:start:] Start action [0000004-140227105121167-oozie-hado-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-02-28 06:15:05,499  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@:start:] [0000004-140227105121167-oozie-hado-W@:start:]Action [***0000004-140227105121167-oozie-hado-W@:start:***]Action status=DONE
2014-02-28 06:15:05,500  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@:start:] [0000004-140227105121167-oozie-hado-W@:start:]Action [***0000004-140227105121167-oozie-hado-W@:start:***]Action updated in DB!
2014-02-28 06:15:05,602  INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] Start action [0000004-140227105121167-oozie-hado-W@job-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-02-28 06:15:06,029  WARN PigActionExecutor:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] credentials is null for the action
2014-02-28 06:15:06,475  WARN JobClient:746 - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
2014-02-28 06:15:06,575  WARN JobClient:870 - No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
2014-02-28 06:15:06,820  INFO PigActionExecutor:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] checking action, external ID [job_201402271128_0005] status [RUNNING]
2014-02-28 06:15:06,825  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] [0000004-140227105121167-oozie-hado-W@job-node]Action [***0000004-140227105121167-oozie-hado-W@job-node***]Action status=RUNNING
2014-02-28 06:15:06,826  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] [0000004-140227105121167-oozie-hado-W@job-node]Action [***0000004-140227105121167-oozie-hado-W@job-node***]Action updated in DB!
2014-02-28 06:15:09,712  INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:15:09,728  INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:15:19,450  INFO CallbackServlet:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] callback for action [0000004-140227105121167-oozie-hado-W@job-node]
2014-02-28 06:15:19,531  INFO PigActionExecutor:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] action completed, external ID [job_201402271128_0005]

[job_201402271128_0005]

2014-02-28 06:15:19,558  WARN PigActionExecutor:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]

[2] ------------------------------------------------------------------------------------------------------------------------- 2014-02-28 06:15:19,659 INFO ActionEndXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] ERROR is considered as FAILED for SLA 2014-02-28 06:15:19,762 INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@fail] Start action [0000004-140227105121167-oozie-hado-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2014-02-28 06:15:19,767 WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@fail] [0000004-140227105121167-oozie-hado-W@fail]Action [***0000004-140227105121167-oozie-hado-W@fail***]Action status=DONE 2014-02-28 06:15:19,767 WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@fail] [0000004-140227105121167-oozie-hado-W@fail]Action [***0000004-140227105121167-oozie-hado-W@fail***]Action updated in DB! 2014-02-28 06:15:19,855 WARN CoordActionUpdateXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100 2014-02-28 06:15:49,366 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService]

[org.apache.oozie.service.StatusTransitService]

savanna 0.3 excuting pig script encounter error

script:
    A = load '$INPUT'  as (line);
    B = foreach A generate flatten(TOKENIZE(line)) as word; 
   C = group B by word;
   D = foreach C generate group, COUNT(B);
   store D into '$OUTPUT';

there is no any exceptions ocurr in jobtacker,namenode,tasktracker or datanode logs. the only exceptions which i can find in /opt/oozie/logs/oozie.log is as follows:

2014-02-28 06:12:49,347  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time =  2014-02-28T06:11Z
2014-02-28 06:12:49,350  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 06:13:09,664  INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:13:09,687  INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:13:49,351  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 06:13:49,352  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time =  2014-02-28T06:12Z
2014-02-28 06:13:49,355  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time =  2014-02-28T06:12Z
2014-02-28 06:13:49,358  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 06:14:09,689  INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:14:09,712  INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:14:49,358  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 06:14:49,359  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time =  2014-02-28T06:13Z
2014-02-28 06:14:49,362  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time =  2014-02-28T06:13Z
2014-02-28 06:14:49,366  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 06:15:05,358  WARN LiteWorkflowAppService:542 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] libpath [hdfs://v-ndhoo-v-nn-001:8020/user/hadoop/worldcount/4496286e-d5e0-42fe-8842-800bace75f54/lib] does not exist
2014-02-28 06:15:05,499  INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@:start:] Start action [0000004-140227105121167-oozie-hado-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-02-28 06:15:05,499  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@:start:] [***0000004-140227105121167-oozie-hado-W@:start:***]Action status=DONE
2014-02-28 06:15:05,500  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@:start:] [***0000004-140227105121167-oozie-hado-W@:start:***]Action updated in DB!
2014-02-28 06:15:05,602  INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] Start action [0000004-140227105121167-oozie-hado-W@job-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-02-28 06:15:06,029  WARN PigActionExecutor:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] credentials is null for the action
2014-02-28 06:15:06,475  WARN JobClient:746 - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
2014-02-28 06:15:06,575  WARN JobClient:870 - No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
2014-02-28 06:15:06,820  INFO PigActionExecutor:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] checking action, external ID [job_201402271128_0005] status [RUNNING]
2014-02-28 06:15:06,825  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] [***0000004-140227105121167-oozie-hado-W@job-node***]Action status=RUNNING
2014-02-28 06:15:06,826  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] [***0000004-140227105121167-oozie-hado-W@job-node***]Action updated in DB!
2014-02-28 06:15:09,712  INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:15:09,728  INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 06:15:19,450  INFO CallbackServlet:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] callback for action [0000004-140227105121167-oozie-hado-W@job-node]
2014-02-28 06:15:19,531  INFO PigActionExecutor:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] action completed, external ID [job_201402271128_0005]

2014-02-28 06:15:19,558  WARN PigActionExecutor:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]
-------------------------------------------------------------------------------------------------------------------------

2014-02-28 06:15:19,659  INFO ActionEndXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@job-node] ERROR is considered as FAILED for SLA
2014-02-28 06:15:19,762  INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@fail] Start action [0000004-140227105121167-oozie-hado-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-02-28 06:15:19,767  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@fail] [***0000004-140227105121167-oozie-hado-W@fail***]Action status=DONE
2014-02-28 06:15:19,767  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[0000004-140227105121167-oozie-hado-W@fail] [***0000004-140227105121167-oozie-hado-W@fail***]Action updated in DB!
2014-02-28 06:15:19,855  WARN CoordActionUpdateXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000004-140227105121167-oozie-hado-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100
2014-02-28 06:15:49,366  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService]

open debug, this is more detail log output:

2014-02-28 07:48:44,688 DEBUG Client:848 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8021 from hadoop got value #147
2014-02-28 07:48:44,688 DEBUG RPC:233 - Call: getJobStatus 1
2014-02-28 07:48:44,690 DEBUG Client:813 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8021 from hadoop sending #148
2014-02-28 07:48:44,692 DEBUG Client:848 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8021 from hadoop got value #148
2014-02-28 07:48:44,692 DEBUG Counters:178 - Creating group oozie.launcher with nothing
2014-02-28 07:48:44,692 DEBUG Counters:178 - Creating group org.apache.hadoop.mapred.FileInputFormat$Counter with bundle
2014-02-28 07:48:44,692 DEBUG Counters:178 - Creating group org.apache.hadoop.mapred.JobInProgress$Counter with bundle
2014-02-28 07:48:44,692 DEBUG Counters:178 - Creating group org.apache.hadoop.mapred.FileOutputFormat$Counter with bundle
2014-02-28 07:48:44,693 DEBUG Counters:178 - Creating group FileSystemCounters with nothing
2014-02-28 07:48:44,693 DEBUG Counters:178 - Creating group org.apache.hadoop.mapred.Task$Counter with bundle
2014-02-28 07:48:44,694 DEBUG RPC:233 - Call: getJobCounters 4
2014-02-28 07:48:44,703 DEBUG HadoopAccessorService:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Checking if filesystem hdfs is supported
2014-02-28 07:48:44,703 DEBUG UserGroupInformation:1212 - PriviledgedAction as:hadoop via hadoop from:org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:420)
2014-02-28 07:48:44,704 DEBUG Client:244 - The ping interval is60000ms.
2014-02-28 07:48:44,704 DEBUG Client:287 - Use SIMPLE authentication for protocol ClientProtocol
2014-02-28 07:48:44,704 DEBUG Client:577 - Connecting to v-ndhoo-v-nn-001/10.12.1.4:8020
2014-02-28 07:48:44,705 DEBUG Client:813 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop sending #149
2014-02-28 07:48:44,705 DEBUG Client:786 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop: starting, having connections 2
2014-02-28 07:48:44,708 DEBUG Client:848 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop got value #149
2014-02-28 07:48:44,708 DEBUG RPC:233 - Call: getFileInfo 5
2014-02-28 07:48:44,708 DEBUG Client:813 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop sending #150
2014-02-28 07:48:44,710 DEBUG Client:848 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop got value #150
2014-02-28 07:48:44,710 DEBUG RPC:233 - Call: getBlockLocations 2
2014-02-28 07:48:44,710 DEBUG DFSClient:2285 - Connecting to /10.12.1.7:50010
2014-02-28 07:48:44,712 DEBUG FSInputChecker:1700 - DFSClient readChunk got seqno 0 offsetInBlock 0 lastPacketInBlock true packetLen 133
2014-02-28 07:48:44,713  WARN PigActionExecutor:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]
2014-02-28 07:48:44,726 DEBUG HadoopAccessorService:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Checking if filesystem hdfs is supported
2014-02-28 07:48:44,726 DEBUG UserGroupInformation:1212 - PriviledgedAction as:hadoop via hadoop from:org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:420)
2014-02-28 07:48:44,726 DEBUG Client:813 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop sending #151
2014-02-28 07:48:44,728 DEBUG Client:848 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop got value #151
2014-02-28 07:48:44,728 DEBUG RPC:233 - Call: getFileInfo 2
2014-02-28 07:48:44,739 DEBUG ActionCheckXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] ENDED ActionCheckXCommand for wf actionId=0000000-140228074649480-oozie-hado-W@job-node, jobId=0000000-140228074649480-oozie-hado-W
2014-02-28 07:48:44,739 DEBUG ActionCheckXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Queuing [1] commands with delay [0]ms
2014-02-28 07:48:44,740 DEBUG ActionCheckXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Released lock for [0000000-140228074649480-oozie-hado-W] in [action.check]
2014-02-28 07:48:44,740 DEBUG ActionEndXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Acquired lock for [0000000-140228074649480-oozie-hado-W] in [action.end]
2014-02-28 07:48:44,746 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Execute command [action.end] key [0000000-140228074649480-oozie-hado-W]
2014-02-28 07:48:44,747 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] STARTED ActionEndXCommand for action 0000000-140228074649480-oozie-hado-W@job-node
2014-02-28 07:48:44,750 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] End, name [job-node] type [pig] status[DONE] external status [FAILED/KILLED] signal value [null]
2014-02-28 07:48:44,756 DEBUG HadoopAccessorService:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Checking if filesystem hdfs is supported
2014-02-28 07:48:44,757 DEBUG UserGroupInformation:1212 - PriviledgedAction as:hadoop via hadoop from:org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:420)
2014-02-28 07:48:44,763 DEBUG HadoopAccessorService:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Checking if filesystem hdfs is supported
2014-02-28 07:48:44,763 DEBUG UserGroupInformation:1212 - PriviledgedAction as:hadoop via hadoop from:org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:420)
2014-02-28 07:48:44,763 DEBUG Client:813 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop sending #152
2014-02-28 07:48:44,764 DEBUG Client:848 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop got value #152
2014-02-28 07:48:44,765 DEBUG RPC:233 - Call: getFileInfo 2
2014-02-28 07:48:44,765 DEBUG Client:813 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop sending #153
2014-02-28 07:48:44,767 DEBUG Client:848 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop got value #153
2014-02-28 07:48:44,767 DEBUG RPC:233 - Call: delete 2
2014-02-28 07:48:44,775  INFO ActionEndXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] ERROR is considered as FAILED for SLA
2014-02-28 07:48:44,776 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Queuing commands for action=0000000-140228074649480-oozie-hado-W@job-node, status=ERROR, Set pending=true
2014-02-28 07:48:44,803 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] ENDED ActionEndXCommand for action 0000000-140228074649480-oozie-hado-W@job-node
2014-02-28 07:48:44,810 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Queuing [1] commands with delay [0]ms
2014-02-28 07:48:44,811 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Released lock for [0000000-140228074649480-oozie-hado-W] in [action.end]
2014-02-28 07:48:44,811 DEBUG SignalXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Acquired lock for [0000000-140228074649480-oozie-hado-W] in [signal]
2014-02-28 07:48:44,817 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Execute command [signal] key [0000000-140228074649480-oozie-hado-W]
2014-02-28 07:48:44,817 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] STARTED SignalCommand for jobid=0000000-140228074649480-oozie-hado-W, actionId=0000000-140228074649480-oozie-hado-W@job-node
2014-02-28 07:48:44,818 DEBUG LiteWorkflowInstance:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Signaling job execution path [/] signal value [ERROR]
2014-02-28 07:48:44,819 DEBUG LiteWorkflowInstance:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Exiting node [job-node] with transition[/#fail]
2014-02-28 07:48:44,819 DEBUG LiteWorkflowInstance:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Signaling job execution path [/] signal value [::synch::]
2014-02-28 07:48:44,819 DEBUG LiteWorkflowStoreService:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node]  Creating action for node [fail]
2014-02-28 07:48:44,821 DEBUG LiteWorkflowStoreService:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Setting action for cred: 'null', name: 'fail'
2014-02-28 07:48:44,822 DEBUG LiteWorkflowStoreService:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Setting action for userRetryMax: '0', userRetryInterval: '10', name: 'fail'
2014-02-28 07:48:44,829 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] SignalXCommand: Name: fail, Id: 0000000-140228074649480-oozie-hado-W@fail, Authcode:null
2014-02-28 07:48:44,846 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Updated the workflow status to 0000000-140228074649480-oozie-hado-W  status =RUNNING
2014-02-28 07:48:44,847 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] ENDED SignalCommand for jobid=0000000-140228074649480-oozie-hado-W, actionId=0000000-140228074649480-oozie-hado-W@job-node
2014-02-28 07:48:44,847 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Queuing [2] commands with delay [0]ms
2014-02-28 07:48:44,847 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Released lock for [0000000-140228074649480-oozie-hado-W] in [signal]
2014-02-28 07:48:44,848 DEBUG NotificationXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [action.notification] key [null]
2014-02-28 07:48:44,853 DEBUG ActionStartXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@job-node] Acquired lock for [0000000-140228074649480-oozie-hado-W] in [action.start]
2014-02-28 07:48:44,870 DEBUG ActionStartXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Execute command [action.start] key [0000000-140228074649480-oozie-hado-W]
2014-02-28 07:48:44,871 DEBUG ActionStartXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] STARTED ActionStartXCommand for wf actionId=0000000-140228074649480-oozie-hado-W@fail
2014-02-28 07:48:44,879  INFO ActionStartXCommand:539 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Start action [0000000-140228074649480-oozie-hado-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-02-28 07:48:44,879  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] [***0000000-140228074649480-oozie-hado-W@fail***]Action status=DONE
2014-02-28 07:48:44,880  WARN ActionStartXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] [***0000000-140228074649480-oozie-hado-W@fail***]Action updated in DB!
2014-02-28 07:48:44,896 DEBUG ActionStartXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] ENDED ActionStartXCommand for wf actionId=0000000-140228074649480-oozie-hado-W@fail, jobId=0000000-140228074649480-oozie-hado-W
2014-02-28 07:48:44,897 DEBUG ActionStartXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Queuing [1] commands with delay [0]ms
2014-02-28 07:48:44,897 DEBUG ActionStartXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Released lock for [0000000-140228074649480-oozie-hado-W] in [action.start]
2014-02-28 07:48:44,898 DEBUG ActionEndXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Acquired lock for [0000000-140228074649480-oozie-hado-W] in [action.end]
2014-02-28 07:48:44,908 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Execute command [action.end] key [0000000-140228074649480-oozie-hado-W]
2014-02-28 07:48:44,908 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] STARTED ActionEndXCommand for action 0000000-140228074649480-oozie-hado-W@fail
2014-02-28 07:48:44,914 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] End, name [fail] type [:KILL:] status[DONE] external status [OK] signal value [null]
2014-02-28 07:48:44,921 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Queuing commands for action=0000000-140228074649480-oozie-hado-W@fail, status=OK, Set pending=true
2014-02-28 07:48:44,949 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] ENDED ActionEndXCommand for action 0000000-140228074649480-oozie-hado-W@fail
2014-02-28 07:48:44,955 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Queuing [1] commands with delay [0]ms
2014-02-28 07:48:44,955 DEBUG ActionEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Released lock for [0000000-140228074649480-oozie-hado-W] in [action.end]
2014-02-28 07:48:44,955 DEBUG SignalXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Acquired lock for [0000000-140228074649480-oozie-hado-W] in [signal]
2014-02-28 07:48:44,970 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Execute command [signal] key [0000000-140228074649480-oozie-hado-W]
2014-02-28 07:48:44,971 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] STARTED SignalCommand for jobid=0000000-140228074649480-oozie-hado-W, actionId=0000000-140228074649480-oozie-hado-W@fail
2014-02-28 07:48:44,973 DEBUG LiteWorkflowInstance:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Signaling job execution path [/] signal value [OK]
2014-02-28 07:48:44,974 DEBUG LiteWorkflowInstance:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Completing job, kill node [fail]
2014-02-28 07:48:45,037 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Try to resolve KillNode message for jobid [0000000-140228074649480-oozie-hado-W], actionId [0000000-140228074649480-oozie-hado-W@fail], before resolve [Workflow failed, error message[${wf:errorMessage(wf:lastErrorNode())}]], after resolve [Workflow failed, error message[Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]]]
2014-02-28 07:48:45,073 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Updated the workflow status to 0000000-140228074649480-oozie-hado-W  status =KILLED
2014-02-28 07:48:45,125  WARN CoordActionUpdateXCommand:542 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100
2014-02-28 07:48:45,129 DEBUG WfEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[-] Execute command [wf_end] key [0000000-140228074649480-oozie-hado-W]
2014-02-28 07:48:45,130 DEBUG WfEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[-] STARTED WFEndXCommand 0000000-140228074649480-oozie-hado-W
2014-02-28 07:48:45,162 DEBUG HadoopAccessorService:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[-] Checking if filesystem hdfs is supported
2014-02-28 07:48:45,165 DEBUG UserGroupInformation:1212 - PriviledgedAction as:hadoop via hadoop from:org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:420)
2014-02-28 07:48:45,166 DEBUG WfEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[-] WF tmp dir :hdfs://v-ndhoo-v-nn-001:8020/user/hadoop/oozie-hado/0000000-140228074649480-oozie-hado-W
2014-02-28 07:48:45,166 DEBUG Client:813 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop sending #154
2014-02-28 07:48:45,167 DEBUG Client:848 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop got value #154
2014-02-28 07:48:45,172 DEBUG RPC:233 - Call: getFileInfo 6
2014-02-28 07:48:45,173 DEBUG Client:813 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop sending #155
2014-02-28 07:48:45,175 DEBUG Client:848 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop got value #155
2014-02-28 07:48:45,180 DEBUG RPC:233 - Call: delete 7
2014-02-28 07:48:45,181 DEBUG WfEndXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[-] ENDED WFEndXCommand 0000000-140228074649480-oozie-hado-W
2014-02-28 07:48:45,181 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] ENDED SignalCommand for jobid=0000000-140228074649480-oozie-hado-W, actionId=0000000-140228074649480-oozie-hado-W@fail
2014-02-28 07:48:45,181 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Queuing [2] commands with delay [0]ms
2014-02-28 07:48:45,181 DEBUG SignalXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Released lock for [0000000-140228074649480-oozie-hado-W] in [signal]
2014-02-28 07:48:45,182 DEBUG NotificationXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [action.notification] key [null]
2014-02-28 07:48:45,182 DEBUG NotificationXCommand:545 - USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000000-140228074649480-oozie-hado-W] ACTION[0000000-140228074649480-oozie-hado-W@fail] Execute command [job.notification] key [null]
2014-02-28 07:48:54,690 DEBUG Client:920 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8021 from hadoop: closed
2014-02-28 07:48:54,691 DEBUG Client:796 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8021 from hadoop: stopped, remaining connections 1
2014-02-28 07:48:55,174 DEBUG Client:920 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop: closed
2014-02-28 07:48:55,174 DEBUG Client:796 - IPC Client (47) connection to v-ndhoo-v-nn-001/10.12.1.4:8020 from hadoop: stopped, remaining connections 0
2014-02-28 07:49:03,501 DEBUG DagEngine:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Oozie DagEngine is using XCommands.
2014-02-28 07:49:03,503 DEBUG JobXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [job.info] key [0000000-140228074649480-oozie-hado-W]
2014-02-28 07:49:04,562 DEBUG ActionCheckerService$ActionCheckRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for potential checking
2014-02-28 07:49:04,606  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 07:49:04,606  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time =  2014-02-28T07:48Z
2014-02-28 07:49:04,609  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time =  2014-02-28T07:48Z
2014-02-28 07:49:04,613  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService]
2014-02-28 07:49:04,779  INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService]
2014-02-28 07:49:04,792  INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService]