Got it

How Do I Add and Delete Topic Configurations

Latest reply: Feb 28, 2022 05:09:45 278 2 0 0 0

Hello, everyone! 

I will share with you how to add and delete topic configurations when using Kafka. 

Symptom

Configure or modify a specific topic.

You can set different parameters depending on topics.

cleanup.policy
compression.type
delete.retention.ms
file.delete.delay.ms
flush.messages
flush.ms
index.interval.bytes
max.message.bytes
min.cleanable.dirty.ratio
min.insync.replicas
preallocate
retention.bytes
retention.ms
segment.bytes
segment.index.bytes
segment.jitter.ms
segment.ms
unclean.leader.election.enable

Method (for V100R002C50SPC20*)

  • Prerequisites

    The Kafka client has been installed.

  • Procedure

  1. Use PuTTY to log in to the node where the client is installed as the Kafka client installation user.

  2. Run the following command to switch to the Kafka client installation directory, for example, /opt/kafkaclient:

    cd /opt/kafkaclient

  3. Run the following command to configure environment variables:

    source bigdata_env

  4. Run the following command to perform user authentication (skip this step in normal mode):

    kinit component service user

  5. Run the following command to go to the Kafka client installation directory:

    cd Kafka/kafka/bin

  6. Run the following commands to configure and delete a topic:

    kafka-topics.sh --alter --topic <topic_name> --zookeeper <zookeeper_host:port>/kafka --config <name=value>

    kafka-topics.sh --alter --topic <topic_name> --zookeeper <zookeeper_host:port>/kafka --delete-config <name>

    For example:

    kafka-topics.sh --alter --topic test1 --zookeeper 192.168.X.X:24002/kafka  --config retention.ms=86400000

    kafka-topics.sh --alter --topic test1 --zookeeper 192.168.X.X:24002/kafka --delete-config retention.ms

  7. Run the following command to query topic information:

    kafka-topics.sh --describe -topic <topic_name> --zookeeper <zookeeper_host:port>/kafka

Method (for V100R002C60 or Later)

  • Prerequisites

  1. The system administrator has understood service requirements and prepared a Kafka administrator (belonging to the kafkaadmin group).

  2. The Kafka client has been installed.

  • Procedure

  1. Use PuTTY to log in to the node where the client is installed as the Kafka client installation user.

  2. Run the following command to switch to the Kafka client installation directory, for example, /opt/kafkaclient:

    cd /opt/kafkaclient

  3. Run the following command to configure environment variables:

    source bigdata_env

  4. Run the following command to perform user authentication (skip this step in normal mode):

    kinit component service user

  5. Run the following command to go to the Kafka client installation directory:

    cd Kafka/kafka/bin

  6. Run the following commands to configure and delete a topic:

    kafka-configs.sh --alter --zookeeper <zookeeper_host:port>/kafka --entity-type topics --entity-name <topic_name> --add-config <name=value>

    kafka-configs.sh --alter --zookeeper <zookeeper_host:port>/kafka --entity-type topics --entity-name <topic_name> --delete-config <name>

    For example:

    kafka-configs.sh --alter --zookeeper 192.168.X.X:24002/kafka --entity-type topics --entity-name test1 --add-config retention.ms=864000001

    kafka-configs.sh --alter --zookeeper 192.168.X.X:24002/kafka --entity-type topics --entity-name test1 --delete-config retention.ms

  7. Run the following command to query topic information:

    kafka-topics.sh --describe -topic <topic_name> --zookeeper <zookeeper_host:port>/kafka


We warmly welcome you to enjoy our community!



  • x
  • convention:

Saqibaz
Created Feb 28, 2022 04:59:22

Thanks for the sharing
View more
  • x
  • convention:

olive.zhao
Admin Created Feb 28, 2022 05:09:45

Good case!
View more
  • x
  • convention:

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.