[FHIR Road Map]: Add Bulk upload interface to web.

[peterMuriuki]

Issue Context?

Support an interface that users can use to initialize bulk upload of resources to the web.
This can be a wizard that:

• Allows users to download a template for upload of the necessary resources.
• Upload the csv template
• Track upload progress(what resource uploads are pending), success and error rates(if upload was succesful, pointers on what errored out, completion percentage)

Issue Implemetation details?

R&D on building upon existing python tool that enables the upload.

Issue Acceptance criteria?

User can via fhir-web:

• Access the bulk upload module
• Download sample templates that they will use to populate the data.
• Upload the csv templates to upload the data.
• See a progress report of the upload process,
• See errors and warning messages with enough context on how to fix the same.
• Or see a summary of resources uploaded after a successful operation

FHIR resources to be used?

No response

Relevant Information

No response

[peterMuriuki]

graph TD

subgraph Dependencies
Binary
Group -->|depends on| Binary
UserRoles
UserGroups -->|depends on| UserRoles
Users -->|depends on| UserGroups
Locations
Organizations -->|depends on| Users
CareTeams -->|depends on| Users
CareTeams -->|depends on| Organizations
InventoryGroup -->|depends on| Locations
InventoryGroup -->|depends on| Group
UserCareTeamAssignment -->|depends on| CareTeams
UserCareTeamAssignment -->|depends on| Organizations
OrganizationLocationAssignment -->|depends on| Organizations
OrganizationLocationAssignment -->|depends on| Locations
end

Loading

[AnnieMungai]

@peterMuriuki QA feedback on bulk upload:

• Uploading all CSVs does not work. Could be the formatting. Updated on June 27th: This could be due to the max size of all the files (more than 8MB), or because individual files were failing.
  
• There is no button to upload service points.
• Uploads are taking awhile - there is no progress bar to show the user that there is a process going on. User may upload 2 similar files.
• Error/ success message is the same for all uploads " Commodity successfully uploaded". This is misleading as it says its a success, then fails. We need to be more clear on the errors.
• Organizations showed successful upload but I could not find any additional orgs on the orgs list.
  
• Locations upload - failed even though I seemed to have done everything correctly on the template (except missing geo-coordinatesand these should not be required.). See error below. I also tried breaking the locations to less 1000 and they still failed. Do we have a max number of items one can upload at a time?
  
• Users - successfully uploaded. Can upload and view them on the users list.
• Careteams - successully uploaded but cannot see the careteams on the UI. Same as the organizations.
• Careteam - users/practitioners - successully uploaded but cannot see the careteams on the UI.
• Products upload- failed.
• Inventory - failed- probably coz individual products and SPs did not load successful?
• Org-locations- Failed.

[peterMuriuki]

CareTeams csv

• [template] status field has value as single letter t

Organization field

• [template] status field has incorrect value 'TRUE' should be 'true'

Products

• [template] id is not a uuid
• [importer] Material number uses id field. either missing id field or material Number on the template.

Inventory

• [importer] bug when adding adding the usualId, po_number identifiers.Looks like it requires po_number filed to not be None which is the case in the template

Locations

• Q: Do we want to still migrate inactive service points?
• adminLevel 0 should be root fhir location
• madagascar should point to root fhir location id.
• row 589 Anjeke Ankilikira has no status seems to be duplicate
• service points have admin levels starting with 0
• [importer] location physcial type is not deduped to type field
• [importer] location parents are not added if parent name is not provided

service points
- some locations missing location type

Careteam to organization

• importer expects this to be in the careteams csv file

participant to CareTeam

• importer expects this to be in the careteams csv file