Experiments on the Read Speed of the Zarr package under different circumstances
These include:
- How much Compression affects read speed
- Effects of using Sharding on performance
- Effects of increasing
chunk_size - Whether joining data variables together improves read speed
These are done both in repeated (warm) or first-time (cold) access
- To Compress or Not to Compress — A Zarr Question
- Efficiently Querying Large Scientific Data Using Zarr’s partial decompress
Please open an issue if you'd like to see other experiments!
- Effects of Increasing
chunk_sizeon Cold Access Speed
- Same but Warm (Repeated) Access
- Cold Access Speed w/ 60% Compression Ratio
- Same but Warm Access
More results can be found in the experiment_results folder