Hello, everyone!
I will share with you how to deal with the problem of topic cannot be deleted when using Kafka.
Symptom
FusionInsight V100R002C50 cluster mainly installs ZooKeeper and Kafka components.
When running the following command on the Kafka client to delete topics, it is found that the topics cannot be deleted.
kafka-topics.sh --delete --topic test --zookeeper 192.168.X.X:24002/kafka
Possible Causes
The ZooKeeper IP address for connecting to the client is incorrect in the client command.
Kafka is abnormal and some Kafka nodes are stopped.
In Kafka, delete.topic.enable is set to true.
In Kafka, auto.create.topics.enable is set to true and the producer is not stopped.
Fault Locating
After the client command is run, the ZkTimeoutException exception is reported.
[2016-03-09 02:41,773] WARN Can not get the principle name from server 192.168.X.X (org.apache.zookeeper.ClientCnxn)Exception in thread "main" org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 30000at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)at kafka.admin.TopicCommand$.main(TopicCommand.scala:51)at kafka.admin.TopicCommand.main(TopicCommand.scala)
To rectify the fault, refer to Solution 1.
Run the following query command on the client:
kafka-topics.sh --list --zookeeper 192.168.X.X:24002/kafka
test - marked for deletion
On FusionInsight Manager, check the service status of Kafka Broker instances.
Run the cd /var/log/Bigdata/kafka/broker command to go to the log directory of node RunningAsController. Locate ineligible for deletion: test in the controller.log log.
2016-03-09 02:41,228 | INFO | [main] | [Controller 1]: List of topics to be deleted: | kafka.controller.KafkaController (Logging.scala:68)2016-03-09 02:41,229 | INFO | [main] | [Controller 1]: List of topics ineligible for deletion: test | kafka.controller.KafkaController (Logging.scala:68)
On FusionInsight Manager, view the delete topic enable status of Broker.
To rectify the fault, refer to Solution 2.
Run the following query command on the client:
kafka-topics.sh --describe -topic test --zookeeper 192.168.X.X:24002/kafka
Go to the log directory of node RunningAsController. Locate marked ineligible for deletion in the controller.log log.
2016-03-09 02:41,989 | INFO | [delete-topics-thread-3] | [delete-topics-thread-3], Handling deletion for topics test | kafka.controller.TopicDeletionManager$DeleteTopicsThread (Logging.scala:68)2016-03-09 02:41,990 | INFO | [delete-topics-thread-3] | [delete-topics-thread-3], Not retrying deletion of topic test at this time since it is marked ineligible for deletion | kafka.controller.TopicDeletionManager$DeleteTopicsThread (Logging.scala:68)
On FusionInsight Manager, query the Broker status.
If Operating Status of a Broker is Stopped or Health Status of a Broker is Bad, delete the topic and ensure that Brokers where partitions of the topic reside must be in the Good state.
To rectify the fault, refer to Solution 3.
Go to the log directory of node RunningAsController. Locate Deletion successfully in the controller.log. If New topics:[Set(test)] is displayed again, it indicates that the topic is created again.
2016-03-09 02:41,208 | INFO | [delete-topics-thread-3] | [delete-topics-thread-3], Deletion of topic test successfully completed | kafka.controller.TopicDeletionManager$DeleteTopicsThread (Logging.scala:68)
2016-03-09 02:41,501 | INFO | [ZkClient-EventThread-19-192.168.0.X:X,160.172.X.X:24002,160.172.X.X:24002/kafka] | [TopicChangeListener on Controller 3]: New topics: [Set(test)], deleted topics: [Set()], new partition replica assignment
On FusionInsight Manager, view the auto create topics enable status of Broker.
Confirm that the producer is stopped.
If the producer is not stopped, refer to Solution 4.
Solution
Perform the following operations when connection to ZooKeeper fails.
When the connection between the Kafka client and ZooKeeper times out, run the ping command to check whether the Kafka client can connect to ZooKeeper.
ping -c 4 192.168.X.X
If the ping fails, view the ZooKeeper service information on FusionInsight Manager.
If ZooKeeper is improperly configured, change the ZooKeeper IP address in the client command.
Perform the following operations when Kafka server configurations cannot be deleted.
Change the value of delete.topic.enable to true and restart Kafka.
After running the following query command on the client, it is found that Topic:test is not displayed in the command output:
kafka-topics.sh --list --zookeeper 192.168.X.X:24002/kafka
Go to the log directory of node RunningAsController. Locate Deletion of topic test successfully in the controller.log log.
2016-03-09 02:41,665 | INFO | [delete-topics-thread-3] | [Partition state machine on Controller 3]: Invoking state change to OfflinePartition for partitions [test,2],[test,15],[test,6],[test,16],[test,12],[test,7],[test,10],[test,13],[test,9],[test,19],[test,3],[test,5],[test,1],[test,0],[test,17],[test,8],[test,4],[test,11],[test,14],[test,18] | kafka.controller.PartitionStateMachine (Logging.scala:68)2016-03-09 02:41,668 | INFO | [delete-topics-thread-3] | [Partition state machine on Controller 3]: Invoking state change to NonExistentPartition for partitions [test,2],[test,15],[test,6],[test,16],[test,12],[test,7],[test,10],[test,13],[test,9],[test,19],[test,3],[test,5],[test,1],[test,0],[test,17],[test,8],[test,4],[test,11],[test,14],[test,18] | kafka.controller.PartitionStateMachine (Logging.scala:68)2016-03-09 02:41,977 | INFO | [delete-topics-thread-3] | [delete-topics-thread-3], Deletion of topic test successfully completed | kafka.controller.TopicDeletionManager$DeleteTopicsThread (Logging.scala:68)
Perform the following operations when some Kafka nodes are stopped.
Start the stopped Broker instances.
After running the following query command on the client, it is found that Topic:test is not displayed in the command output:
kafka-topics.sh --list --zookeeper 192.168.X.X:24002/kafka
Go to the log directory of node RunningAsController. Locate Deletion of topic test successfully in the controller.log log.
2016-03-09 02:41,463 | INFO | [delete-topics-thread-3] | [Partition state machine on Controller 3]: Invoking state change to NonExistentPartition for partitions [test,4],[test,1],[test,8],[test,2],[test,5],[test,9],[test,7],[test,6],[test,0],[test,3] | kafka.controller.PartitionStateMachine (Logging.scala:68)2016-03-09 02:41,726 | INFO | [delete-topics-thread-3] | [delete-topics-thread-3], Deletion of topic test successfully completed | kafka.controller.TopicDeletionManager$DeleteTopicsThread (Logging.scala:68)
Perform the following operations when Kafka configurations are automatically created and the producer is not stopped.
Stop the producer, change the value of auto.create.topics.enable to false, and restart Kafka.
Perform the delete operation again.
We warmly welcome you to enjoy our community!
