首页 > 其他 > 详细

airflow脚本

时间:2020-01-14 15:47:45      阅读:118      评论:0      收藏:0      [点我收藏+]

airflow-1脚本:
*/2 * * * * /bin/sh /hongfeng/script/rsynce_airflow_log.sh >/dev/null 2>&1
*/4 * * * * /bin/sh /hongfeng/script/monitor_scheduler.sh >/dev/null 2>&1

1/ log的rsync_airflow_log.sh
rsync -azuq -e ssh root@10.52.51.8:/root/airflow/logs/ /root/airflow/logs/ --exclude ‘scheduler‘ --exclude ‘scheduler_failover‘ --exclude ‘dag_processor_manager‘
rsync -azuq -e ssh root@10.52.17.70:/root/airflow/logs/ /root/airflow/logs/ --exclude ‘scheduler‘ --exclude ‘scheduler_failover‘ --exclude ‘dag_processor_manager‘
rsync -azuq -e ssh root@10.52.139.191:/root/airflow/logs/ /root/airflow/logs/ --exclude ‘scheduler‘ --exclude ‘scheduler_failover‘ --exclude ‘dag_processor_manager‘

2/ monitor_scheduler.sh
#! /bin/sh

date=$(date +%Y-%m-%d-%H:%M)

file_path=/root/airflow/scheduler.log
check_time=150
file_old_stat="`stat ${file_path}|grep Size`"
sleep ${check_time}
file_new_stat="`stat ${file_path}|grep Size`"
if [[ `echo ${file_old_stat}` == `echo ${file_new_stat}` ]]; then
. /data/venv/bin/activate && supervisorctl restart airflow_scheduler && deactivate
echo "#${date},scheduler service restarted" >>/tmp/test/scheduler_hung.log
fi

3/ status_check.sh
ansible airflow -m shell -a ‘. /data/venv/bin/activate && supervisorctl status‘

4/ deploy_git_airflow.sh
ansible airflow -m shell -a ‘cd /root/airflow/dags && git pull‘
5/ 企业微信报警(G:\文档\开发\python)
*/2 * * * * (. /data/venv/bin/activate && python /root/airflow/airflow_health_monitor.py && deactivate) >> /root/airflow/airflow_health.log 2>&1 &

airflow脚本

原文:https://www.cnblogs.com/hongfeng2019/p/12191966.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!