[June] [Pubs] Curation Workflow Tracking

[aditigopalan]

This ticket tracks curation workflow progression.

Note: It is possible for work to take place simultaneously in the three sections with overlapping periods, allowing curation workflows for different months to coincide.

1. Curation and Annotation

• Run Pubmed crawler to generate PublicationView manifest [205 publications generated, long sprint anticipated]
• Send Amber and Jineta a copy of the PublicationView manifest from latest crawl to review for MC2 Center Newsletter publication highlights
• Send Amber "News from CCKP" for MC2 Center Newsletter
• Annotate publications in PublicationView manifest [In progress]
• Generate ToolView and DatasetView manifests based on PublicationView manifest
• Run the automated curation workflow to upload publications, datasets and tools [This includes splitting manifests, processing and validating manifests, generating target synapse IDs for upload, schema updates, upload to synapse and (in progress) a validation check for uploads)
• Generate UNION tables
• QC of staging tables
• Performing automate portal sync to CCKP
• Validate data on the CCKP

Status check [Plan to report numbers for each category following pubmed crawl]:

• Publication upload [ ]
• Tool upload [ ]
• Data set upload [ ]

2. Data model

• Update valid values in the data model and build
• Generate templates from new model
• Release new model version [no changes, no release]
• Update DCA config [no changes, no update]

3. Contributor Engagement

• Emails to contributors (by grant) with info on newly added manifests, link to project, DCA, and instructions on review, annotation, validation, and submission, as applicable
• Jira Help Desk ticket tracking, review, triage, with data model updates, as needed (TBD)
• Annotation gap-filling of DatasetView and ToolView manifests

[aclayton555]

24-6 close out: aiming to upload May, April and June by mid 24-7/8 sprint. Will likely include a data model release during mid sprint

[aclayton555]

mid-sprint:

• Pending Annotations still growing? @jaybee84 Check which resources have this. Not that "Pending Annotations" was historically implemented with publications that were not yet open access. But these are now also pushed to portal as an incentive for adding annotations to these resources.
• resume emails to contributors with existing documentation. Maybe targeted engagement to certain contributor

[jaybee84]

Publications: pending annotation for tumor type + assay (as discussed in the meeting mostly due to restricted access, but some on open access)
Datasets : pending annotations for tissue