[ARPA] Add feature to export reporting data into a ZIP for gov partners auditing purposes

[mindyatwork]

Due to auditing requirements, gov partners need the ability to easily download all the Input templates that went into what they ultimately submit to Treasury.

💭 Proposal

• A button that downloads a Zip file of:
  • All the xlsm files that show up when you filter by Only exported to treasury for only the quarter the user is currently in.
  • An uploaded_by.csv that includes, per row:
    • filename
    • Email of the person who uploaded it
    • Agency
    • EC code
    • time stamp of upload
• Open design questions:
  • Where does this button sit? In the Uploads or Dashboard tab?
  • What is the text/color on the button?

See full 1-pager here: https://www.notion.so/usdr/Export-reporting-data-into-a-CSV-828402124113453d8109e145e191eb85

[mindyatwork]

Andrew is working on this one

[agn-dsgn]

Hi! I'm the designer picking this up. Could the owner of the one-pager (linked above) please add me to it?

Not sure who to ask @mhuang-usdr - thanks!

[mindyatwork]

So odd, I thought I gave you access already! Sent it back over. Thanks for asking :) — [image: USDR Logo] Mindy Huang Product Manager, Economic Stability E ***@***.*** Discover new USDR volunteer opportunities on our Twitter <https://twitter.com/USDResponse>, LinkedIn <https://www.linkedin.com/company/usdr>, and Medium <https://medium.com/u-s-digital-response> U.S. Digital Response <https://www.usdigitalresponse.org/> is a nonprofit that matches pro-bono technologists with government and organizations responding to crisis, to quickly deliver critical services and infrastructure that support the needs of the public. We’re non-partisan, fast, and free.

…

On Thu, Mar 9, 2023 at 1:29 PM agn-dsgn ***@***.***> wrote: Hi! I'm the designer picking this up. Could the owner of the one-pager <https://www.notion.so/usdr/Export-reporting-data-into-a-CSV-828402124113453d8109e145e191eb85> (linked above) please add me to it? Not sure who to ask @mhuang-usdr <https://github.com/mhuang-usdr> - thanks! — Reply to this email directly, view it on GitHub <#887 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVX62EKWFOESS4KOOUYJOKTW3JDULANCNFSM6AAAAAAUM45IW4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[agn-dsgn]

Discussed design options with @caitlinwinner and landed on this one (shown below). Looking for feedback from @mhuang-usdr and the relevant engineer (who be you??)

Design Spec r1 - outdated

Overview: The suggestion is to add a button within the table in Uploads that will allow users to download all the workbooks/uploads and an index .csv file

Button styling:

• add blue button to the right of "reset filters"
• use same styling as "download file" within each table row (+ same hover state)
• button text: "Download all as .zip"

Functionality:

• on click, execute download of a .zip containing the following materials: a .xlsm of every filtered workbook and an index .csv file
• if there is no filter applied to the table, I suggest to still show the button to make for a more consistent UI and allow users to download all uploads (and all their versions) if desired
• index file includes this info for each row/workbook: filename, email of the person who uploaded it, agency, EC code, time stamp of upload
• when there are not yet any uploads, disable this button. (Let me know if we don't have a disabled button state)

Additional changes:

• remove the question marks ? after each of the existing filters

Questions:

• Any issues with allowing the button to persist with or without the filter? Could the larger data sets be too large / slow / problematic? Could this affect .zip file preparation time and necessitate a progress indicator?
• What should we name the .index file? Ticket suggestion was uploaded_by.csv, and an alternate idea is index_MM-DD-YYYY.csv >> FINAL = index_[reporting period name]_YYYY-MM-DD.csv
• What should we name the .zip? Maybe something like workbooks_MM-DD-YYYY? >> FINAL = workbooks_[reporting period name]_YYYY-MM-DD.zip

[mindyatwork]

Any issues with allowing the button to persist with or without the filter? Could the larger data sets be too large / slow / problematic? Could this affect .zip file preparation time and necessitate a progress indicator?

Persisting without the filter seems the most straightforward UX, and downloading all files is a use case I'm sure will be requested once we give the ability to download all "Treasury files", so I'm a fan of keeping that. This does mean we'll likely need a progress indicator - even with the filter on folks will already have upwards of 50+ files in their zip!

We also currently have a 10mb download limit based on our tech stack - @TylerHendrickson this sounds like a use case that will make us move to S3 regardless.

What should we name the .index file? Ticket suggestion was uploaded_by.csv, and an alternate idea is index_MM-DD-YYYY.csv

Partners suggested index_[reporting period name]_YYYY-MM-DD.csv

What should we name the .zip? Maybe something like workbooks_MM-DD-YYYY?

Partners suggested workbooks_[reporting period name]_YYYY-MM-DD.zip

[agn-dsgn]

Great! The naming suggestions are perfect. Here are edited specs, with a couple final questions:

Design Spec - final

Overview: We will add a button within the table in Uploads that will allow users to download all the workbooks/uploads and an index .csv file

Link to Figma spec

Button:

• add button to the right of "reset filters"
• button text: "Download all as .zip"
• use same styling as "reset filters" button but with blue background, hex #1565c0
• hover and active states: same as for "Download file" button
• progress indicator: spinner + "Preparing..." similar to when downloading template
• SEE BELOW FOR RELATED QUESTIONS

Functionality:

• on click, execute download of a .zip containing the following materials: a .xlsm of every filtered workbook and an index .csv file
• if no filter applied, include every workbook
• index file includes this info for each row/workbook: filename, email of the person who uploaded it, agency, EC code, time stamp of upload

File naming:

• .zip file: workbooks_[reporting period name]_YYYY-MM-DD.zip
• .csv index file: index_[reporting period name]_YYYY-MM-DD.csv

Additional changes:

• remove the question marks ? after each of the existing filters

Questions:

• button disabled state TBD: disable button if there are no uploads yet. Do we have a disabled state?
• error handling: Do we have a generic message or pattern if this action fails?

[TylerHendrickson]

@TylerHendrickson Note to self: This should probably be blocked by the S3 transition.

[ClaireValdivia]

@as1729 curious about where this fits in with the s3 work, and wondering if there needs to be two issues - one to export ALL historic files as a zip, and another to be able to download files from a single closed period.

[as1729]

As @TylerHendrickson has commented earlier here. We should probably follow this order:

• Migrate EFS upload files into S3 #2455
• [STORY]: Admin export of all org's input files #2477
• This issue

The 2477 and this issue can be worked in parallel once we have officially moved over to S3.

[ClaireValdivia]

2/7 convo - confirmed that we want two separate issues - one to export ALL historic files as a zip, and another to be able to download files from a single closed period.

[ClaireValdivia]

Closing as this is now a duplicate of #3803