kafka集群搭建步骤

前提：1、准备3台linux虚机 2、已搭建zk环境

以下是实操命令步骤

一、下载kafka

[app@kafkanode01 ~]$ mkdir kafka [app@kafkanode01 ~]$ cd kafka [app@kafkanode01 kafka]$ mkdir kafka-logs [app@kafkanode01 kafka]$ wget http://mirrors.hust.edu.cn/apache/kafka/2.0.0/kafka_2.11-2.0.0.tgz

二、解压并配置kafka

[app@kafkanode01 kafka]$ tar -xzvf kafka_2.11-2.0.0.tgz [app@kafkanode01 kafka]$ cd kafka_2.11-2.0.0 [app@kafkanode01 kafka_2.11-2.0.0]$ cd config [app@kafkanode01 config]$ vi server.properties

node1修改为

broker.id=1 listeners=PLAINTEXT://10.10.30.238:9092 log.dirs=/home/app/kafka/kafka-logs zookeeper.connect=10.10.30.238:2181,10.10.30.239:2181,10.10.30.240:2181

node2修改为

broker.id=2 listeners=PLAINTEXT://10.10.30.239:9092 log.dirs=/home/app/kafka/kafka-logs zookeeper.connect=10.10.30.238:2181,10.10.30.239:2181,10.10.30.240:2181

node3修改为

broker.id=3 listeners=PLAINTEXT://10.10.30.240:9092 log.dirs=/home/app/kafka/kafka-logs zookeeper.connect=10.10.30.238:2181,10.10.30.239:2181,10.10.30.240:2181

以上ip需对应实际地址

三、启动kafka

cd /home/app/kafka/kafka_2.11-2.0.0/bin

./kafka-server-start.sh -daemon …/config/server.properties

tail -100f /home/app/kafka/kafka_2.11-2.0.0/logs/server.log

四、创建topic

cd /home/app/kafka/kafka_2.11-2.0.0/bin

[app@kafkanode01 bin]$ ./kafka-topics.sh --zookeeper localhost:2181 --list [app@kafkanode01 bin]$ ./kafka-topics.sh --zookeeper localhost:2181 --create --replication-factor 2 --partitions 3 --topic my-topic Created topic “my-topic”. [app@kafkanode01 bin]$ ./kafka-topics.sh --zookeeper localhost:2181 --list my-topic

五、测试生产和消费消息

[app@kafkanode03 bin]$ ./kafka-console-producer.sh --broker-list 10.10.30.240:9092 --topic my-topic

[app@kafkanode01 bin]$ ./kafka-console-consumer.sh --bootstrap-server 10.10.30.239:9092 --topic my-topic --from-beginning

kafka集群搭建完毕。