我几乎没有像hadoop那样需要每天执行的命令,结果需要发送到我的邮件id。我如何使用oozie作业来完成这个任务?
kfgdxczn1#
下面的事情将被设置为做一个Map的行动和发送电子邮件之后。在oozie-site.xml中,添加以下设置并重新启动oozie。将值替换为特定于您的环境的相同值。
<property> <name>oozie.email.smtp.host</name> <value></value> </property> <property> <name>oozie.email.smtp.port</name> <value>25</value> </property> <property> <name>oozie.email.from.address</name> <value></value> </property> <property> <name>oozie.email.smtp.auth</name> <value>false</value> </property> <property> <name>oozie.email.smtp.username</name> <value></value> </property> <property> <name>oozie.email.smtp.password</name> <value></value> </property>
在workflow.xml中,使用环境设置添加以下说明。这包括mapreduce完成后的电子邮件触发操作。
<workflow-app name="WorkFlowJavaMapReduceAction" xmlns="uri:oozie:workflow:0.1"> <start to="mapReduceAction" /> <action name="mapReduceAction"> <map-reduce> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <prepare> <delete path="${outputDir}" /> </prepare> <configuration> <property> <name>mapred.mapper.new-api</name> <value>true</value> </property> <property> <name>mapred.reducer.new-api</name> <value>true</value> </property> <property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property> <property> <name>mapreduce.map.class</name> <value></value> </property> <property> <name>mapreduce.reduce.class</name> <value></value> </property> <property> <name>mapred.mapoutput.key.class</name> <value>org.apache.hadoop.io.Text</value> </property> <property> <name>mapred.mapoutput.value.class</name> <value>org.apache.hadoop.io.IntWritable</value> </property> <property> <name>mapred.output.key.class</name> <value>org.apache.hadoop.io.Text</value> </property> <property> <name>mapred.output.value.class</name> <value>org.apache.hadoop.io.IntWritable</value> </property> <property> <name>mapred.input.dir</name> <value>${inputDir}</value> </property> <property> <name>mapred.output.dir</name> <value>${outputDir}</value> </property> <property> <name>mapreduce.job.acl-view-job</name> <value>*</value> </property> <property> <name>oozie.launcher.mapreduce.job.acl-view-job</name> <value>*</value> </property> <property> <name>oozie.use.system.libpath</name> <value>false</value> </property> <property> <name>oozie.libpath</name> <value>${appPath}/lib</value> </property> </configuration> </map-reduce> <ok to="emailCommands" /> <error to="killJob" /> </action> <action name="emailCommands"> <fs> <mkdir path='${makeDirectoryAbsPath}' /> <move source='${dataInputDirectoryAbsPath}' target='${dataDestinationDirectoryRelativePath}' /> </fs> <ok to="sendEmailSuccess" /> <error to="sendEmailKill" /> </action> <action name="sendEmailSuccess"> <email xmlns="uri:oozie:email-action:0.1"> <to>${emailToAddress}</to> <subject>Status of workflow ${wf:id()}</subject> <body>The workflow ${wf:id()} completed successfully</body> </email> <ok to="end" /> <error to="end" /> </action> <action name="sendEmailKill"> <email xmlns="uri:oozie:email-action:0.1"> <to>${emailToAddress}</to> <subject>Status of workflow ${wf:id()}</subject> <body>The workflow ${wf:id()} had issues and was killed. The error message is: ${wf:errorMessage(wf:lastErrorNode())}</body> </email> <ok to="end" /> <error to="killJob" /> </action> <kill name="killJob"> <message>"Killed job due to error: ${wf:errorMessage(wf:lastErrorNode())}"</message> </kill> <end name="end" /> </workflow-app>
请参考这篇博文http://webthanga.blogspot.in/2016/01/oozie-map-reduce-action-with-send-mail.html
hmtdttj42#
在oozie coordinator的帮助下,您可以基于时间和数据触发器提交工作流。您可以在下面找到发送电子邮件的日常协调员工作的示例:
<?xml version="1.0" encoding="UTF-8"?> <coordinator-app name="cord1" frequency="${coord:days(1)}" start="${jobStart}" end="${jobEnd}" timezone="UTC" xmlns="uri:oozie:coordinator:0.4"> <action name="an-email"> <email xmlns="uri:oozie:email-action:0.1"> <to>bob@initech.com,the.other.bob@initech.com</to> <cc>will@initech.com</cc> <subject>Email notifications for ${wf:id()}</subject> <body>The wf ${wf:id()} successfully completed.</body> </email> <ok to="myotherjob"/> <error to="errorcleanup"/> </action> </coordinator-app>
您可以设置的其他频率如下:
${coord:minutes(int n)} ${coord:hours(int n)} ${coord:days(int n)} ${coord:months(int n)}
2条答案
按热度按时间kfgdxczn1#
下面的事情将被设置为做一个Map的行动和发送电子邮件之后。在oozie-site.xml中,添加以下设置并重新启动oozie。将值替换为特定于您的环境的相同值。
在workflow.xml中,使用环境设置添加以下说明。这包括mapreduce完成后的电子邮件触发操作。
请参考这篇博文http://webthanga.blogspot.in/2016/01/oozie-map-reduce-action-with-send-mail.html
hmtdttj42#
在oozie coordinator的帮助下,您可以基于时间和数据触发器提交工作流。您可以在下面找到发送电子邮件的日常协调员工作的示例:
您可以设置的其他频率如下: