调度工具Azkaban版本3.90.0安装

1. 下载、修改源码、编译

1. 下载和解压

[root@bigdata001 opt]#
[root@bigdata001 opt]# wget -O azkaban-3.90.0.tar.gz https://codeload.github.com/azkaban/azkaban/tar.gz/refs/tags/3.90.0
[root@bigdata001 opt]#
[root@bigdata001 opt]# tar -zxvf azkaban-3.90.0.tar.gz
[root@bigdata001 opt]#
[root@bigdata001 opt]# cd azkaban-3.90.0
[root@bigdata001 azkaban-3.90.0]#

2. 修改源码

添加SSL发送邮件：修改azkaban-common/src/main/java/azkaban/utils/EmailMessage.java中的sendEmail()方法，为props添加属性props.put("mail.smtp.socketFactory.class", "javax.net.ssl.SSLSocketFactory");

Web界面中History菜单的高级过滤条件，进行时间过滤时(Start Time和End Time都要介于所选的开始时间和结束时间之间)，会报异常：

HTTP ERROR 500

Problem accessing /history. Reason:

    Invalid format: "02/09/2022 1:48 PM" is malformed at "PM"

Caused by:

java.lang.IllegalArgumentException: Invalid format: "02/09/2022 1:48 PM" is malformed at "PM"
	at org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:866)
	at azkaban.webapp.servlet.HistoryServlet.handleHistoryPage(HistoryServlet.java:106)
	at azkaban.webapp.servlet.HistoryServlet.handleGet(HistoryServlet.java:60)
	at azkaban.webapp.servlet.LoginAbstractAzkabanServlet.doGet(LoginAbstractAzkabanServlet.java:122)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.Server.handle(Server.java:326)
	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
	at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

Powered by Jetty://

解决方法：修改/opt/azkaban-3.85.5/azkaban-web-server/src/main/java/azkaban/webapp/servlet/HistoryServlet.java的handleHistoryPage方法，在两处DateTimeFormat.forPattern(FILTER_BY_DATE_PATTERN)后添加.withLocale(java.util.Locale.ENGLISH)

3. 编译

[root@bigdata001 azkaban-3.90.0]# 
[root@bigdata001 azkaban-3.90.0]# ./gradlew build -x test
[root@bigdata001 azkaban-3.90.0]# 
[root@bigdata001 azkaban-3.90.0]# ./gradlew installDist
[root@bigdata001 azkaban-3.90.0]# 

2. 安装 2.1 在mysql中创建azkaban数据库和Azkaban相关表

将azkaban-3.90.0/azkaban-db/build/distributions/azkaban-db-0.1.0-SNAPSHOT.tar.gz中的create-all-sql-0.1.0-SNAPSHOT.sql，上传到mysql所在服务器的/opt目录下

[root@bigdata005 opt]# 
[root@bigdata005 opt]# mysql -u root -pRoot_123
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 183
Server version: 8.0.25 MySQL Community Server - GPL

Copyright (c) 2000, 2021, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 
mysql> CREATE DATABASE azkaban;
Query OK, 1 row affected (0.01 sec)

mysql> 
mysql> GRANT SELECT,INSERT,UPDATE,DELETE ON azkaban.* to 'root'@'%' WITH GRANT OPTION;
Query OK, 0 rows affected (0.01 sec)

mysql> 
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> 
mysql> use azkaban;
Database changed
mysql> 
mysql> source /opt/create-all-sql-0.1.0-SNAPSHOT.sql

mysql>

2.2 安装Executor Server

可以参考以下步骤部署多个Executor Server

[root@bigdata001 opt]# 
[root@bigdata001 opt]# mkdir azkaban
[root@bigdata001 opt]# 
[root@bigdata001 opt]# cp -r /opt/azkaban-3.90.0/azkaban-exec-server/build/install/azkaban-exec-server azkaban/
[root@bigdata001 opt]#
[root@bigdata001 opt]# cd azkaban
[root@bigdata001 azkaban]#
[root@bigdata001 azkaban]# ls
azkaban-exec-server
[root@bigdata001 azkaban]#
[root@bigdata001 azkaban]# cd azkaban-exec-server/
[root@bigdata001 azkaban-exec-server]#
[root@bigdata001 azkaban-exec-server]# ls
bin  conf  lib  plugins
[root@bigdata001 azkaban-exec-server]#

1. 修改conf/azkaban.properties

修改内容如下：

default.timezone.id=Asia/Shanghai
executor.global.properties=/opt/azkaban/azkaban-exec-server/conf/global.properties
azkaban.jobtype.plugin.dir=/opt/azkaban/azkaban-exec-server/plugins/jobtypes
mysql.host=bigdata005
mysql.user=root
mysql.password=Root_123

2. 修改plugins/jobtypes/commonprivate.properties

添加内容如下：

memCheck.enabled=false
azkaban.native.lib=false

3. 替换mysql的jar包

[root@bigdata001 azkaban-exec-server]# 
[root@bigdata001 azkaban-exec-server]# rm lib/mysql-connector-java-5.1.28.jar -f
[root@bigdata001 azkaban-exec-server]#
[root@bigdata001 azkaban-exec-server]# cd lib
[root@bigdata001 lib]#
[root@bigdata001 lib]# wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.25/mysql-connector-java-8.0.25.jar
[root@bigdata001 lib]#

4. 启动executor server

[root@bigdata001 bin]# 
[root@bigdata001 bin]# sh start-exec.sh 
[root@bigdata001 bin]# 
[root@bigdata001 bin]# curl -G "localhost:$(