jumanpp mit Homebrew, Nix installieren

Projektlinks

• Homepage

Befehle und Aliase

• jumanpp
• mkdarts_jumanpp

Projektgeschichte

Kyoto University's Juman++ page links the project as a Japanese morphological analyzer and cites the 2015 EMNLP paper on morphological analysis for unsegmented languages using recurrent neural network language models.

The GitHub README describes Juman++ as a new analyzer that uses an RNN language model, and states that version 2 improved accuracy and analysis speed by more than 250 times compared with the original Juman++.

The 2018 ACL Anthology system-demonstration paper describes the version 2 toolkit as a C++11/14 lattice-based morphological-analysis library with linear and recurrent neural net language models, plus tools for exposing model problems and partial annotation.

Adoptionsgeschichte

Juman++ gained adoption in Japanese NLP because it improved the accuracy and speed tradeoff of the JUMAN/Jumandic line while keeping a command-line analyzer usable in pipelines.

The Homebrew and Nix packages made the analyzer easy to install for developers outside Kyoto University's downloadable tarball workflow. The project also remained academically citeable through EMNLP 2015, ANLP 2018, EMNLP 2018, and a 2020 Journal of Natural Language Processing paper listed by the README.

Wie es verwendet wird

Users run `jumanpp` with UTF-8 Japanese text and receive one analyzed morpheme per line followed by `EOS`. The README documents options such as `--model`, `--beam`, `--specifics`, `--version`, and `--help`.

The package distribution includes a pretrained model, while the README warns that building directly from Git does not by itself provide a usable model for analysis.

Warum Paket-Nerds sich dafür interessieren

Juman++ is notable to package maintainers because the source tree alone is not the product: usable packages need the analyzer and model assets. That tension is visible in the README's warning that release packages are much larger than source snapshots because they include a pretrained model.

It is also a bridge between classic Japanese morphological analyzers and neural NLP tooling: it keeps the Unix filter style while using neural language-model scoring under the hood.

Zeitleiste

• 2015-09: The RNNLM-based Juman++ work appeared at EMNLP 2015.
• 2017-12-02: GitHub release v2.0.0-rc1 was published as the first preview.
• 2018-03-14: GitHub release v2.0.0-rc2 was published.
• 2018-11: The Juman++ toolkit paper appeared in the EMNLP 2018 system demonstrations.
• 2023-10-03: GitHub release v2.0.0-rc4 was published.

Related projects

• Juman++ is closely related to JUMAN, Jumandic, KNP, PyKNP, and the Kyoto University Text Corpus ecosystem.
• The README points to `jumanpp-jumandic` for training a Jumandic model and to tutorial code for building analyzers for other scriptio continua settings.

Quellen

• https://aclanthology.org/D15-1276/
• https://aclanthology.org/D18-2010/
• https://github.com/ku-nlp/jumanpp
• https://github.com/ku-nlp/jumanpp/releases
• https://nlp.ist.i.kyoto-u.ac.jp/EN/index.php?JUMAN%2B%2B