Install twarc with Homebrew

Project links

• Homepage

Commands and aliases

• twarc
• twarc2

Project history

The project was developed under DocNow with Mellon Foundation support, and its README defines its purpose narrowly: collection and archiving of Twitter data through the API. The project eventually exposed two command families: `twarc` for the older Twitter API v1.1 surface and `twarc2` for Twitter API v2 and Academic Access.

In 2021 DocNow announced twarc2 as a ground-up redesign for Twitter API v2 and the Academic Research track. The redesign responded to changes in Twitter's data model, rate limits, and full-archive access, while keeping the old v1.1-oriented command available for users who still depended on it.

The repository now notes that twarc is no longer actively supported after Twitter API quota changes made it unusable for its original research and archiving workflows.

Adoption history

twarc was adopted by researchers, archivists, journalists, and digital-humanities users who needed a repeatable way to collect Twitter data as JSON without writing their own API client. The official tutorial is aimed at researchers new to command-line social-media data collection and walks through search, JSON output, CSV conversion, and sharing tweet identifier datasets.

The twarc2 release notes describe a community around the tool, including code and design contributors and early feedback from researchers and practitioners. That community use case was shaped by the Twitter Academic Research track, which opened full-history search to approved researchers before later API changes reduced the tool's practical usefulness.

How it is used

Common package-nerd usage is installing the CLI, configuring Twitter/X developer credentials, and running commands such as `twarc2 search ... > tweets.jsonl` or using twarc as a Python library. The tool stores tweet data as line-oriented JSON and delegates rate-limit handling and high-fidelity v2 field expansion to the library.

twarc also became a base for small data-processing plugins. The README explicitly points users with use-case-specific needs toward external packages that hook into `twarc2` through `click-plugins`, keeping the core tool focused on acquisition and archival JSON.

Why package nerds care

twarc matters to package nerds because it turned a fragile web-API workflow into a packaged, scriptable Unix-style tool: install it, configure credentials, stream JSON, and compose the result with CSV converters, Python, pandas, jq, and archival pipelines.

Its history also records a hard lesson for API-dependent packages: a well-packaged tool can be technically sound and widely used, yet still be constrained or broken by upstream platform policy and quota changes.

Timeline

• 2010s: twarc becomes a DocNow tool for collecting and archiving Twitter JSON through the Twitter API.
• 2021-04-07: DocNow announces twarc2, redesigned for Twitter API v2 and Academic Research access.
• 2020s: twarc documents separate `twarc` and `twarc2` commands for Twitter API v1.1 and v2 workflows.
• 2020s: The repository notes that Twitter API quota changes left twarc no longer actively supported.

Related projects

• DocNow and the Documenting the Now project provided the institutional setting for twarc.
• twarc-csv and other twarc2 plugins extend collected JSON into analysis-friendly formats.
• Twitter/X API v1.1, Twitter/X API v2, and the Academic Research track shaped the tool's command design and adoption.

Sources

• DocNow project post, 'twarc2', 2021-04-07: https://medium.com/documenting-docnow/twarc2-779278e66ea0
• GitHub README: https://github.com/DocNow/twarc
• Official documentation tutorial: https://twarc-project.readthedocs.io/en/latest/tutorial/
• Official twarc2 documentation: https://twarc-project.readthedocs.io/en/latest/twarc2_en_us/