Definition of Impala

Tomcat is a free-of-charge, lightweight, open source web application server developed by Apache Software Foundation, Sun and other companies, and individuals. Gaining support from Sun, Tomcat complies with the latest Servlet and JavaServer Page (JSP) specifications. It features advanced technology, stable performance, and good scalability, and occupies a small number of system resources during running. As a result, Tomcat applies to small- and medium-sized systems and scenarios with a few concurrent users. An HTTP server embedded in Tomcat enables Tomcat to work as a web server. It provides a configuration management tool and supports XML configuration files. Tomcat is different from the Apache web server, which is an HTTP server using C language.

Flume is a distributed, highly reliable, and HA massive log aggregation system. It supports custom data transmitters for data collection. Flume also processes data roughly and writes data to customizable receivers.

Impala provides rapid and interactive SQL queries for Hadoop data stored on HDFS and rapidly returns query results. Therefore, Impala is applicable to interactive queries and data analysis.

Kafka is a distributed, partitioned, and replicated message publishing and subscription system that provides features similar to the Java Message Service (JMS). Kafka features message persistence, high throughput, distribution, multi-client support, and real-time processing, and applies to online and offline message consumption. It is ideal for Internet service data collection scenarios, such as conventional data collection, active website tracing, aggregation of operation data in statistics systems (monitoring data), and log collection.

The number of files that HDFS NameNode can manage is restricted by the node's heap memory. A large number of small files (more small files indicate less data blocks) generated during the use of services can rapidly consume NameNode memory and slow NameNode running. A background small file merging feature (namely, SmallFS) is developed to solve this problem. SmallFS automatically detects small files in the system based on the file size threshold, merges them, and stores metadata to a local LevelDB to reduce the NameNode load. Moreover, it provides a new FileSystem interface for users to transparently access these small files.