Apache Doris 0.14.0部署HDFS Broker服务

1. 编译

官方提供Docker环境编译、Centos\Ubuntu环境编译、ARM64平台编译，我们这里采样Docker环境编译

下载Docker镜像

[root@bigdata001 ~]#
[root@bigdata001 ~]# docker pull apache/incubator-doris:build-env-1.2
build-env-1.2: Pulling from apache/incubator-doris
7a376117f479: Pull complete 
Digest: sha256:a0b8f7e3da126b2692600174cff762569fd38d722f54512a145d9b40a764747c
Status: Downloaded newer image for apache/incubator-doris:build-env-1.2
docker.io/apache/incubator-doris:build-env-1.2
[root@bigdata001 ~]# 

运行镜像

[root@bigdata001 ~]#
[root@bigdata001 ~]# docker run -it apache/incubator-doris:build-env-1.2
[root@efff319b8bb7 ~]#

下载解压Apache Doris源码

[root@efff319b8bb7 ~]#
[root@efff319b8bb7 ~]# wget --no-check-certificate https://dlcdn.apache.org/incubator/doris/0.14.0-incubating/apache-doris-0.14.0-incubating-src.tar.gz
[root@efff319b8bb7 ~]# 
[root@efff319b8bb7 ~]# tar -zxvf apache-doris-0.14.0-incubating-src.tar.gz
[root@efff319b8bb7 ~]#

编译源码

[root@97e3e5f3b1ac ~]# cd apache-doris-0.14.0-incubating-src
[root@97e3e5f3b1ac apache-doris-0.14.0-incubating-src]# 
[root@97e3e5f3b1ac apache-doris-0.14.0-incubating-src]# cd fs_brokers/
[root@97e3e5f3b1ac fs_brokers]# 
[root@97e3e5f3b1ac fs_brokers]# cd apache_hdfs_broker/
[root@97e3e5f3b1ac apache_hdfs_broker]# 
[root@97e3e5f3b1ac apache_hdfs_broker]# sh build.sh 
[root@97e3e5f3b1ac apache_hdfs_broker]#

编译后的结果文件在output下

[root@97e3e5f3b1ac apache_hdfs_broker]# cd output                       
[root@97e3e5f3b1ac output]# 
[root@97e3e5f3b1ac output]# zip -r apache_hdfs_broker.zip apache_hdfs_broker
[root@97e3e5f3b1ac output]#

将apache_hdfs_broker.zip导出到宿主机

[root@bigdata001 ~]# docker cp 97e3e5f3b1ac:/root/apache-doris-0.14.0-incubating-src/fs_brokers/apache_hdfs_broker/output/apache_hdfs_broker.zip /opt/apache-doris-0.14.0
[root@bigdata001 ~]#
[root@bigdata001 ~]# cd /opt/apache-doris-0.14.0
[root@bigdata001 apache-doris-0.14.0]# 
[root@bigdata001 apache-doris-0.14.0]# unzip apache_hdfs_broker.zip
[root@bigdata001 apache-doris-0.14.0]#

修改JAVA_HOME 修改apache_hdfs_broker/bin/start_broker.sh，修改内容如下：

export JAVA_HOME="/opt/jdk1.8.0_201"

然后将apache_hdfs_broker目录，复制到Doris集群的其它服务器的/opt/apache-doris-0.14.0目录下

2. HDFS Broker的部署

每一个FE和BE节点都需要部署HDFS Broker

在每台服务器启动如下命令，启动HDFS Broker

[root@bigdata001 bin]# pwd
/opt/apache-doris-0.14.0/apache_hdfs_broker/bin
[root@bigdata001 bin]# sh start_broker.sh --daemon
[root@bigdata001 bin]# 

使用Mysql客户端，连接Doris。执行以下命令将所有Broker服务添加到Doris

mysql> 
mysql> alter system add broker apache_hdfs_broker 
    -> "192.168.8.111:8000","192.168.8.112:8000","192.168.8.113:8000";
Query OK, 0 rows affected (0.19 sec)

mysql>

查看各个Broker的状态如下：

mysql> 
mysql> show proc "/brokers";
+--------------------+---------------+------+-------+---------------+----------------+--------+
| Name               | IP            | Port | Alive | LastStartTime | LastUpdateTime | ErrMsg |
+--------------------+---------------+------+-------+---------------+----------------+--------+
| apache_hdfs_broker | 192.168.8.113 | 8000 | true  | NULL          | NULL           |        |
| apache_hdfs_broker | 192.168.8.111 | 8000 | true  | NULL          | NULL           |        |
| apache_hdfs_broker | 192.168.8.112 | 8000 | true  | NULL          | NULL           |        |
+--------------------+---------------+------+-------+---------------+----------------+--------+
3 rows in set (0.03 sec)

mysql>