UC: Functional Unit Redefinition

[jawache]

This is written in the context of SCI but the same could be said for any denominator in the SCER spec.

The functional unit, per R, the denominator, can be redefined. For example, if it was per prompt, which would be an easy-to-understand and rationalize metric for LLMs, what is a prompt? Where does the User Journey start and end? Are we only including non-image-generated prompts? Is there a limit to the length of the prompt?

This is a user experience redefinition. For instance, if you are a travel organization and your booking software reports a score per booking, what is a booking? In year one, you could define it to include all the searches humans made to decide on the trip they wanted to book. In subsequent years, you might slowly redefine a booking to not include the initial searches.

Counter

• Use a standard definition for Functional Units for different domains. The standard clearly describes the whole user experience, describing the journey in such a way that there is no room for interpretation.
• The description of the user journey, which informs R might also inform the definition of the software boundary for this category of product.
• Provide many examples of reference user journeys to ensure there is a broad knowledge base to infer from.