使用 Homebrew 安装 orc-tools

项目链接

• 主页

命令和别名

• orc-tools

项目历史

The tools grew out of the need to inspect and manipulate ORC files without always launching Hive, Spark, or a full Hadoop job. The older Hive manual documented `hive --orcfiledump` for analyzing ORC files, while the Apache ORC project later documents a standalone ORC tools jar that packages utilities and Java dependencies, including Hadoop, into one artifact.

The ORC Java Tools documentation marks subcommand additions across ORC releases: `scan` since ORC 1.3, `convert` and `json-schema` since ORC 1.4, `key` since ORC 1.5, `count` and `version` since ORC 1.6, `sizes` since ORC 1.7.2, and `check` and `merge` since ORC 2.0.1.

采用历史

The tools inherit Apache ORC's adoption in Hadoop and analytical data lakes. ORC documentation says the format was created in January 2013 to speed up Apache Hive and improve Hadoop storage efficiency, and it names large Hadoop users such as Facebook and Yahoo in its adoption discussion.

Because ORC files are binary columnar files, command-line tools fill a package-maintenance gap: data engineers, release engineers, and distro packagers need a way to verify metadata, print data as JSON, convert CSV or JSON into ORC, count rows, inspect key metadata, and merge files during tests and migrations.

使用方式

Users invoke the tools as `java -jar orc-tools-X.Y.Z-uber.jar <sub-command> <args>` or through a packaged launcher such as Homebrew's `orc-tools`. Common workflows include `meta` to inspect file metadata, `data` to print records, `convert` to create ORC files from CSV or JSON, `count` to recursively count rows, `sizes` to inspect per-column disk usage, and `merge` to combine ORC files with compatible schemas.

The API documentation exposes the package as Java classes such as `Driver`, `FileDump`, `JsonFileDump`, `KeyTool`, `MergeFiles`, `PrintData`, `PrintVersion`, and `RowCount`, matching the command-line utility role rather than a general application framework.

为什么软件包爱好者会关心

For package nerds, `orc-tools` is the inspect-and-repair companion to a columnar storage format. It is the difference between ORC as an opaque data-lake file and ORC as something that can be validated, dumped, converted, counted, benchmarked, and scripted from a shell.

时间线

• 2013-01: Apache ORC documentation says ORC files were created as part of the Hive speed and Hadoop storage-efficiency initiative.
• 2013: The Hive manual documents ORC as introduced in Hive 0.11.0 and includes `hive --orcfiledump` for inspection.
• ORC 1.3 through 2.0.1: Apache ORC Java Tools documentation records the arrival of subcommands such as `scan`, `convert`, `count`, `sizes`, `check`, and `merge`.

Related projects

• Apache ORC Tools is part of the Apache ORC repository and follows the ORC format's ecosystem: Apache Hive, Apache Spark, Hadoop file systems, HDFS, and other data-lake engines that read or write ORC.

来源

• https://cwiki.apache.org/confluence/display/Hive/LanguageManual%2BORC
• https://formulae.brew.sh/formula/orc-tools
• https://orc.apache.org/api/orc-tools/org/apache/orc/tools/package-summary.html
• https://orc.apache.org/docs/
• https://orc.apache.org/docs/java-tools.html