使用 Homebrew 安装 flume

项目链接

• 主页

命令和别名

• flume-ng

项目历史

The Apache Flume site describes Flume as a reliable, available service with a simple streaming data-flow architecture and a flexible event model for online analytics. Its user guide centers on agents made of sources, channels, and sinks that are wired together in properties files.

Apache's release news records a long 1.x series as an Apache top-level project, with stable production-ready releases from the early 2010s through the 1.11.0 release in 2022.

采用历史

Flume became a common Hadoop-adjacent log collection component by integrating with HDFS, HBase, Hive, Kafka, Avro, Thrift, syslog, netcat, Elasticsearch, and other ingestion or storage systems across its 1.x releases.

The 2014 Apache Flume 1.4.0 release notes emphasize plugins.d support, embedding agents, Avro and HDFS improvements, and numerous contributors from Git/SVN logs, showing the project maturing as an extensible ingestion platform.

使用方式

Operators start agents with flume-ng, pass an agent name, config directory, and config file or URI, and define sources, channels, and sinks in properties-style configuration. Typical topologies include fan-in collection from many log producers, fan-out routing, RPC ingestion, and delivery to storage or analytics systems.

为什么软件包爱好者会关心

Flume is package-nerd significant as a classic Java distribution package: shell launchers, conf directories, plugin directories, many optional connectors, Hadoop ecosystem dependencies, and security-sensitive logging/configuration behavior all affect how downstream packages are assembled.

Its history also captures a transition in log ingestion packaging from Hadoop-era batch and HDFS pipelines toward Kafka and cloud-native collectors.

时间线

• 2013: Apache Flume 1.4.0 was released as the fourth Flume release as an Apache top-level project.
• 2015: Apache Flume 1.6.0 added Kafka source/sink support, a Kafka channel, Hive Streaming sink support, and end-to-end authentication.
• 2017: Apache Flume 1.8.0 added features including Taildir source and environment-variable support in configuration files.
• 2019: Apache Flume 1.9.0 added better SSL/TLS support, configuration filters, and Kafka client updates.
• 2022: Apache Flume 1.11.0 was released with Spring Boot deployment support and Kafka source/sink improvements.

Related projects

• Apache Hadoop, HDFS, HBase, Hive, Kafka, Avro, and Thrift are major ecosystem projects connected to Flume sources, channels, sinks, or release features.
• Apache BigTop is mentioned in Apache Flume release notes as a downstream distribution that shipped Flume binaries for a specific compatibility target.

来源

• https://flume.apache.org/
• https://flume.apache.org/releases/content/1.11.0/FlumeUserGuide.html
• https://flume.apache.org/source.html