What's the largest .zip file you've seen in actual usage?

[saulpw]

I'm thinking about making a demo involving unzip-http, which I think is really cool, but I'm wondering how useful it is in practice. Like do any of you actually have to interact with a terabyte .zip file at a url that has 100k .csv files in it?

[frosencrantz]

I don't have a really big one. Kaggle and HuggingFace might have some big data.

Here are a couple I've used VisiData to quickly look at:

• https://files.digital.nhs.uk/publicationimport/pub18xxx/pub18273/nhs-staf-2015-healthcare-gp-csv.zip
  • More data at https://digital.nhs.uk/services/organisation-data-service/export-data-files
• https://github.com/jart/cosmopolitan/releases/download/3.0.1/cosmos-3.0.1.zip
  • This was one I found recently. I think there might be an issue with pulling files out this file. It is mostly binaries, but you can quickly see what files it contains.

There might be a bug in the latest develop code related to pulling files or diving into files.

[frosencrantz]

I wonder if these penny stock data files would be of interest. These zip file contain csv files with no headers, one file per stock.

https://www.cs.cmu.edu/~pavlo/datasets/stocks/pennystocks.html

My comments do not really answer your question, since I don't actually use any of these files in practice.

[daviewales]

It's not terabytes, but I've sometimes used the feature to browse these Australian Census Zips:
https://www.abs.gov.au/census/find-census-data/datapacks