ipldstore

Background

In dClimate's pursuit of making climate data accessible and immutable using P2P and Web3 concepts alongside cloud optimized Zarr we came across the https://github.com/d70-t/ipldstore repository which seemed to do most of which we wanted. We forked this repository and made some additions of our own such as making the data leverage UnixFS as well as ensuring the metadata uses HAMTs for the zarr key/value pairs. (see: ipldstore/hamt_wrapper.py to see how this is done leveraging dClimate's py-hamt library)

Overview

The repo README details general usage but overall ipldstore is a python library which facilitates the storage/retrieval of Zarrs on IPFS using HAMTs.

Future Work

Exploring features from the original ipldstore library such as CAR Sharding which can hopefully make it upstream, the discussion here car fsspec reference file system output in the original repo as well, and ultimately reconcile our repo and merge it upstream :).

• Repo: https://github.com/dClimate/ipldstore
• Open Issues: https://github.com/dClimate/ipldstore/issues (We encourage contribution!) and the original repo issues which we would love to use