Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Critical: Fixes ZipFileHeader Struct and Zip64 Header creation - Implements switch to enable creation of large sized ZipSegments and ZIP_STORED containers #36

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

fservida
Copy link

@fservida fservida commented Aug 26, 2023

As per title, fixes the ZipFileHeader Struct, by changing file_size and compress_size to uint32. Else packing struct fails when file size is > 2GB.

Also introduces an optional switch to enable creation of zip based containers with arbitrary file sizes. This allows creation of containers fully compatible with archive management tools for decompression if needed, as well as ability to import the container as ZIP file in any forensic tool that might not support AFF4-L directly.
Default behaviour is left untouched.

Updated to PyYaml 5.4 to reflect latest packaged pyaff4 version on pip
…erwriting

Allow creation of containers based on ZIP_STORED
@fservida fservida changed the title Fixes ZipFileHeader Struct and implements switch to enable creation of large sized ZipSegments Critical: Fixes ZipFileHeader Struct and Zip64 Header creation - Implements switch to enable creation of large sized ZipSegments and ZIP_STORED containers Aug 27, 2023
@fservida
Copy link
Author

After playing around with first modifications, I noticed that files larger than 4GB (requiring zip64 headers) were corrupted.
Investigating highlighted that the code creates the local file header in two passes, first one not having the file size. In such a case, only the basic header is written followed by file data. If the file passes 4GB, the final header would require using a zip64 extension, which is appended to the local file header, but overwrites the first bytes of the file.

The hotfix actually reserves the space and writes the zip64 header all the time, independently of the size, and only marks the fields to be looked up in the zip64 header if needed - Should still be compliant with zip specification as the zip64 header is simply ignored if not needed.

…to 0xFFFFFFFF to force reading from extra field, extra field size is now written correctly (was always 0 before)
Updated PyYaml to avoid build issues on cython 3.0 yaml/pyyaml#601
Removed version pinning for pyyaml
Addedd pybindgen as required for successful build of fastchunking
require latest intervaltree to avoid issues with mutableset in python >=3.10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant