Spike: Success and Usage Metrics

[JohnNKing]

Backlog Task

This is a spike to determine what metrics we need to gather to evaluate the impact of ReportStream ETOR. This story could be considered complete when we've completed a draft plan created.

Who is the audience of this information?

• CDC leadership
• Current partners
• Potential future partners
• Our own team for identification and prioritization of future work

What is the purpose of the metrics?

• Show the value our work is having
• Attracting new partners
• Track improvements/changes over time

Timebox

Before CA go-live

Completion Criteria

• We've documented an initial set of aggregate ETOR usage metrics to provide to the CDC and/or partners.
• We've created backlog items to cover any work we may need to do to support the ETOR metrics previously discussed with the CDC, CA, and LA.

Tasks

• What tasks need to be done to fulfill the completion criteria?

Other Notes

Potential stats to consider:

• Turn-around time for NBS results
• Frequency of transcription errors
• Time to onboard a new hospital

Also see:

• https://aphlinformatics.atlassian.net/wiki/spaces/EPI/pages/2318139509/2024-03-19+Meeting+Notes
• https://aphlinformatics.atlassian.net/wiki/spaces/EPI/pages/2369781762/APHL+Detor+Evaluation+Metrics
• https://aphlinformatics.atlassian.net/wiki/spaces/ERI/pages/2332164122/4+11+24+SME+Working+Session
• https://aphlinformatics.atlassian.net/wiki/pages/viewpageattachments.action?pageId=2083651778&preview=%2F2083651778%2F2092400732%2FCDC%20ETOR%20via%20Intermediary%20Draft%20Evaluation%20Plan_10.06.23.docx
• https://aphlinformatics.atlassian.net/wiki/spaces/IR/pages/2060517419/Intermediary+Agnostic+Evaluation+Plan

[JohnNKing]

See https://www.newsteps.org/media/26/download?inline -- specifically quality indicators 2 and 5

[scleary1cs]

What did we agree to during the AL and LA kickoffs?

[scleary1cs]

What metrics are important to the labs, hospitals, Flexion, CDC?

[csweet92]

@JohnNKing @scleary1cs I'd be happy to be assigned to this

[csweet92]

Rebecca is leading this effort from the CDC side. Luther will be reaching out to see if there's anything she needs from our side and if she could share the progress.

[scleary1cs]

@kwright-bio can you please provide some JMC contacts that attended the NBS conference.

[scleary1cs]

How do we know when we've picked the right metrics?
How will we know when this story is complete?

[scleary1cs]

Do we have a sense of the folks who will work on this?
@kwright-bio

[csweet92]

IR-Intermediary Agnostic Evaluation Plan-111224-204744.pdf

[brick-green]

Leaving in product backlog till we determine how it should strucutred (epic or ticket)

[csweet92]

Latest update can be found here:
https://www.notion.so/flexion-cdc-ti/California-Dashboard-Requirements-13e5955365f580ca9be9f4bb1b8bab15?pvs=4#1625955365f580b5ac9cf914f8e0a5ac

[scleary1cs]

https://artkai.io/blog/north-star-framework