alluxio mit Homebrew installieren

Projektlinks

• Homepage

Befehle und Aliase

• alluxio
• alluxio-common.sh
• alluxio-masters.sh
• alluxio-monitor.sh
• alluxio-mount.sh
• alluxio-start.sh
• alluxio-stop.sh
• alluxio-workers.sh
• launch-process

Projektgeschichte

The Alluxio README states that the project was formerly known as Tachyon and originated as a UC Berkeley AMPLab research project. It was the data layer of the Berkeley Data Analytics Stack, and the README points readers to Haoyuan Li's dissertation, Alluxio: A Virtual Distributed File System, for the research background.

The project later became Alluxio, an Apache-licensed open source system owned by the Alluxio Open Source Foundation and managed by an Alluxio Project Management Committee. Official docs position it as data orchestration technology for analytics and AI, with a memory-first tiered architecture, global namespace, and server-side API translation.

Adoptionsgeschichte

Official documentation says Alluxio has attracted more than 1000 contributors from over 300 institutions including Alibaba, Baidu, CMU, Google, IBM, Intel, NJU, Red Hat, Tencent, UC Berkeley, and Yahoo. It also says the project is deployed in production by hundreds of organizations, while the README describes petabyte-scale production deployments and a largest deployment exceeding thousands of nodes.

Alluxio adoption followed the data-platform pattern of being useful when compute and storage are decoupled. Its docs call out use with Apache Spark, Presto, Tensorflow, Apache HBase, Apache Hive, and Apache Flink above storage systems such as Amazon S3, Google Cloud Storage, OpenStack Swift, HDFS, IBM Cleversafe, EMC ECS, Ceph, NFS, Minio, and Alibaba OSS.

Wie es verwendet wird

Operators deploy an Alluxio master and workers, mount persistent under storage into the Alluxio namespace, and point compute jobs at Alluxio through HDFS-compatible, S3, FUSE/POSIX, REST, or Java APIs. The Spark guide, for example, shows Spark applications reading and writing `alluxio://` paths while Alluxio transparently fetches and caches data from under storage.

The command-line package includes administrative and cluster scripts such as `alluxio`, `alluxio-start.sh`, `alluxio-stop.sh`, `alluxio-masters.sh`, and `alluxio-workers.sh`. In package-manager terms this is not just a CLI utility; it is a local entry point into a distributed service stack.

Warum Paket-Nerds sich dafür interessieren

Alluxio is significant to package people because it packages a full distributed Java data system behind Unix-style scripts. A Homebrew install can be convenient for development and local testing, but production use usually involves clusters, Docker, Kubernetes, Hadoop/Spark classpaths, and configuration files.

The former Tachyon name, AMPLab origin, and BDAS connection tie Alluxio to the same research-to-production wave that shaped Spark-era big data infrastructure. Its package metadata has to communicate storage, cache, filesystem, and orchestration at once because all of those labels are true in different deployment modes.

Zeitleiste

• Tachyon era: Project originated at UC Berkeley AMPLab as the data layer of the Berkeley Data Analytics Stack.
• Alluxio rename: The project became Alluxio, described by the README as formerly known as Tachyon.
• Foundation/PMC era: README states the Alluxio Open Source Foundation owns the project and the PMC manages operation.
• Current: Official docs position Alluxio as data orchestration for analytics and AI across many compute engines and storage backends.

Related projects

• Apache Spark, Presto, Tensorflow, Apache HBase, Apache Hive, and Apache Flink are documented compute-side integrations.
• Amazon S3, Google Cloud Storage, OpenStack Swift, HDFS, IBM Cleversafe, EMC ECS, Ceph, NFS, Minio, and Alibaba OSS are documented storage-side examples.
• The Berkeley Data Analytics Stack and UC Berkeley AMPLab are central to Alluxio's origin story.

Quellen

• https://documentation.alluxio.io/os-en/api/posix-api.md
• https://documentation.alluxio.io/os-en/compute/spark.md
• https://documentation.alluxio.io/os-en/overview.md
• https://raw.githubusercontent.com/Alluxio/alluxio/master/README.md