This post refers to the Kafka data production. More details to be found below.
1. Introduction to Kafka
Originally developed by Linkedin, Kafka is a distributed, partitioned, multi-copy, multi-subscriber and distributed log system based on ZooKeeper.
It can also be used as an MQ system. Kafka is mainly used in two types of applications:
1. Construct a real-time data stream channel between applications;
2. Construct real-time streaming applications for transmitting or processing data streams.
2. Basic realization of producers
1. Schematic
2. Specific implementation
2.1. Fire-and-forget mode
After a message is sent, the logic program does not need to check whether the message is sent successfully. This is the default writing mode. It depends on the high availability of the producer api itself (the retry is performed after the related parameters are configured). By default, the high throughput asynchronous transmission is used. In most cases, the data will be successful, but there will also be failures.
