Write Code to pull down 6 tar files explode the tar files, checksum each file in the tar and produce a report for the curators

[carolyncole]

There are 6 works describe in #1918 that have tar files with different checksums.

The curators would like a report of the checksums of each file in the tar to prove that they are the same.

Acceptance Criteria:

• write code that can take an id, pull down the files locally, unzip and then checksum the files and use pkgdiff to write a report
  • https://datacommons.princeton.edu/describe/works/38
  • https://pdc-describe-prod.princeton.edu/describe/works/425
  • https://pdc-describe-prod.princeton.edu/describe/works/429
  • https://pdc-describe-prod.princeton.edu/describe/works/431
• Run the rake task to delete the dspace files and move the globus file to the original name after the curators have reviewed the report
• This code is throw away and will be utilized 6 times

[carolyncole]

Hey team! Please add your planning poker estimate with Zenhub @hectorcorrea @JaymeeH @leefaisonr @bess

[carolyncole]

I pulled the files and then ran pkgdif on each data_space globus pair. I noted no changes while running the pkgdiff, but to be certain I also searched the reports, which had 99 verdicts and 99 Unchanged reports:

The full report can be found https://drive.google.com/file/d/1bVuYBtQqTj80QZ8_mo511bYX0meJeXW7/view?usp=sharing

[matthewjchandler]

@carolyncole following Slack side conversations, I understand that the report confirms that duplicated tar files have matching contents.

I know there was a pointed discussion about 38, which has files with very similar names but dashes in some cases and underscores in others, and I saw that the decision was to just keep the files that were originally in DataSpace.

As for 425, 429, and 431, I understand that they have a lingering issue of README files that do not match, and so @sec122 and @neggink are working on manual review.

If there's anything else anyone needs to complete this ticket, let me know.