Allow Shapefile to read .zip files
#113
Conversation
We should subscribe to the FileIO interface so we can deal with streaming files as well... Co-authored-by: David Gleich <[email protected]>
|
Test failures seem a bit strange... |
|
Oh, this is awesome! Thanks for pushing it through! |
|
About those errors being a bit strange : |
|
Ah! Should I fix the tests then? |
|
Yes! lets look out for little breaking changes like that next time and bump the minor version ;) |
|
CI passes now. Should we merge and release? I've updated the minor version (so this would technically be a breaking release), even if nothing is actually going to break. |
|
What about .gz and every other zip format? .zip is pretty windows centric? |
|
It seems that ZipFile.jl only handles .zip files. We could switch to TranscodingStreams but I have no clue how (or if) it handles files... |
|
There are a lot of places that distribute shape files as zip files (e.g. this seems to be how the US government does it), so this provides benefit as is... i.e you can download the zip file and read it directly. (My original use case involved working with a few hundred such zip files, which I really didn't all want to unzip...) The TarFiles situation isn't as straightforward as there isn't (yet) a straightforward way to read a tar.gz file as a list of Julia IO objects as there is for zip files. See, e.g. JuliaIO/Tar.jl#95 So I wouldn't worry about .tar.gz files right now. |
|
Makes sense. If zip is 95% of the files and the only thing thats easy then thats fine. |
|
Do we need more reviews or should I merge + release? |
This is an implementation of #75 that "dispatches" on provided file name. Ideally, this should also work with some form of streaming type so that we can pass that to ZipFile and thus use cloud shapefiles directly, but this will do for now.
The PR implements an extension
ShapefileZipFilesExtwhich provides a method for a function_read_shp_from_zipfilethat is defined as a function with no methods inShapefile.jl. This also allows us to provide a nice error message if the extension is not loaded.CC: @dgleich