hyparquet

hyparam/hyparquet is an open-source project by hyparam: parquet file parser for javascript. Indexed here so it can be found — not resold.

It is free. Get it from the upstream repository: https://github.com/hyparam/hyparquet

From the project's own README (excerpt, reproduced for discovery under its MIT license):

hyparquet

Dependency free since 2023!

What is hyparquet?

Hyparquet is a JavaScript library for parsing Apache Parquet files in the browser. Apache Parquet is a popular columnar storage format that is widely used in data engineering, data science, and machine learning applications for storing large datasets. Hyparquet is designed to read parquet files efficiently over http, so that parquet files in cloud storage can be queried directly from the browser without needing a server.

> Powers HypAware, the improvement loop for all your agents. • Works in browsers and node.js • Pure JavaScript, no dependencies • Supports all parquet types, encodings, and compression codecs • Minimizes data fetching using HTTP range requests • Includes TypeScript definitions

Parquet Viewer

Try hyparquet online: Drag and drop your parquet file onto hyperparam.app to view it directly in your browser. This service is powered by hyparquet's in-browser capabilities.

Quick Start

Browser Example

In the browser use asyncBufferFromUrl to wrap a url for reading asynchronously over the network. It is recommended that you filter by row and column to limit fetch size:

Node.js Example

To read the contents of a local parquet file in a node.js environment use asyncBufferFromFile:

Note: hyparquet is published as an ES module, so dynamic import() may be required for old versions of node.

Parquet Writing

To create parquet files from javascript, check out the hyparquet-writer package.

Advanced Usage

Reading Metadata

You can read just the metadata, including schema and data statistics using the parquetMetadataAsync function. This is useful for getting the schema, number of rows, and column names without reading the entire file.

If the file (or at least its footer) is…