使用多副本(replica)实现高可用
#############################

使用命令行命令查看当前topic 的详细信息::

    [root@idc-cui-pr-kafka-3 kafka]# ./bin/kafka-topics.sh --zookeeper 10.140.2.93:2181,10.140.2.97:2181,10.140.2.92:2181 --describe --topic wen

    Topic:wen PartitionCount:1 ReplicationFactor:1 Configs:
        Topic: wen Partition: 0 Leader: 3 Replicas: 3 Isr: 3

说明::

    • PartitionCount:1   当前topic 的分区个数
    • ReplicationFactor:1 当前topic分区的副本数 

    本例显示的topic wen 副本数为1, 则当topic所在的一个节点(本例为broker 3) down 的状况下，此 topic 将不可用

为了实现topic的高可用，可以在线实时扩展副本::

    可以根据实际应用负载情况来扩展分区（可选）：
    [root@idc-cui-pr-kafka-3 kafka]# ./bin/kafka-topics.sh --alter --zookeeper 10.140.2.93:2181,10.140.2.97:2181,10.140.2.92:2181 --partitions 3 --topic wen 

    WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
    Adding partitions succeeded!

调整完分区信息如下::

    [root@idc-cui-pr-kafka-3 kafka]# bin/kafka-topics.sh --zookeeper 10.140.2.93:2181,10.140.2.97:2181,10.140.2.92:2181 --describe --topic wen
    Topic:wen PartitionCount:3 ReplicationFactor:1 Configs:
    Topic: wen Partition: 0 Leader: 3 Replicas: 3 Isr: 3
    Topic: wen Partition: 1 Leader: 1 Replicas: 1 Isr: 1
    Topic: wen Partition: 2 Leader: 2 Replicas: 2 Isr: 2

扩展副本::

    首先编辑一个a.json 文件，格式如下：
    {
        "version": 1, 
        "partitions": [
            {
                "topic": "wen", 
                "partition": 0, 
                "replicas": [
                    2, 
                    1, 
                    3
                ]
            },
            {
                "topic": "wen", 
                "partition": 1, 
                "replicas": [
                    3, 
                    2, 
                    1
                ]
            },
            {
                "topic": "wen", 
                "partition": 2, 
                "replicas": [
                    1, 
                    2, 
                    3
                ]
            }
        ]
    }

使用 kafka-reassign-partitions.sh 命令行动态调整副本::

    [root@idc-cui-pr-kafka-3 kafka]# bin/kafka-reassign-partitions.sh --zookeeper 10.140.2.93:2181,10.140.2.97:2181,10.140.2.92:2181 --reassignment-json-file a.json --execute

    Current partition replica assignment
    {"version":1,"partitions":[{"topic":"wen","partition":0,"replicas":[3],"log_dirs":["any"]},{"topic":"wen","partition":2,"replicas":[2],"log_dirs":["any"]},{"topic":"wen","partition":1,"replicas":[1],"log_dirs":["any"]}]}
    Save this to use as the --reassignment-json-file option during rollback
    Successfully started reassignment of partitions.


查看调整完成之后的topic 信息::

    [root@idc-cui-pr-kafka-3 kafka]# bin/kafka-topics.sh --zookeeper 10.140.2.93:2181,10.140.2.97:2181,10.140.2.92:2181 --describe --topic wen
    Topic:wen PartitionCount:3 ReplicationFactor:3 Configs:
    Topic: wen Partition: 0 Leader: 3 Replicas: 2,1,3 Isr: 3,2,1
    Topic: wen Partition: 1 Leader: 1 Replicas: 3,2,1 Isr: 1,3,2
    Topic: wen Partition: 2 Leader: 2 Replicas: 1,2,3 Isr: 2,1,3

说明::

    显示副本数为3： ReplicationFactor:3

    这样在三台节点kafka 集群, 有三副本的topic, 在down 掉两个 broker 节点的情况下, 仍然工作正常, 业务不受影响