Add ability to test saving from multiple results

[dstillman]

zotero/translators#3043 highlights how much we need the ability to add tests for multiple saving. I believe the reason we've never done it is that search results aren't stable, but I feel like we could come up with a test definition that would filter specific items from the results to save (based on either the key (URL, usually) or the title, I guess), so we could at least confirm that 1) the translator was finding results and 2) specific items could be saved.

@zoe-translates, maybe you could think about how we might define these?

[zoe-translates]

search results aren't stable, but I feel like we could come up with a test definition that would filter specific items from the results to save

This should be workable to some extent, unless the specific item no longer appears in the search-results page at the given URL. For example, it may have fallen in search ranks and is no longer at the firs page. I imagine this can be a problem with Google Scholar. I've heard many complaints about its tendency of trying to outsmart the user.

Currently in Z7's Scaffold, the 'multiple' test seems to emulate selecting the first choice and proceed with translating that. I think the result will show as failure if that single item fails to translate. This will make sure there's at least an arbitrary item that can be saved.

The main difficulty with finding "specific" items is that we have no control whether those items will show up at all. And whether they show up says very little about our particular translator's quality, or even utility.

But for those places where we in principle could be sure about the presence of some item, perhaps in the test case there may be an additional field indicating this, something like

{
	"type": "web",
	"url": "[...]",
	"items": "multiple",
	"targetRow": "[...]"
}

where "targetRow" is a string corresponding to one of the the text labels (conventionally called title in the translator code, often in a function called getSearchResults) among the multiple results, as appeared in the item selector UI. If the labels contain this one, the test passes. If this property is missing or value isn't a non-empty string, this additional test is not to be performed.

The reason I prefer a string value over an item object, is that to do this test but with items, in principle we have to translate every row in the multiple selection. With a string target, it suffices to inspect the object passed to Zotero.selectItems.

This will at least show that the translator is doing something more "expected".

I think perhaps we may design a Scaffold workflow like this:

1. Developer loads a page with multiple results in Browser tab in Scaffold, and clicks on "Create Web Test".
2. The item selector popup appears, but with the check-box column prominently labelled "Test Target?" or something. Probably with tooltip explaining this.
3. If an item is selected (preferably at most one is selectable), it becomes the "targetRow" in the generated test case.

But we should explain that the "Test Target?" box should only be ticked if the search query or URL is specifically crafted to ensure the stability of results. For example, an article-listing page of a journal issue should be fine, or a search query like https://zenodo.org/search?page=1&size=20&q=matplotlib+v3.7.1&type=software if the selected one is to be "matplotlib 3.7.1", but in general free-text searches in a database is not.

[zoe-translates]

An even more complicated design may look like this.

Instead of a string value at "targetRow", it takes the value of an object

{
        "selectionTitle": "...",
        "item": { ... }
}

where the "item" field is the translation result.

The workflow would be modified to be

3. If an item is selected (preferably at most one is selectable), its "title" or label text becomes the value of targetRow.selectionTitle in the generated test case; it is translated; and if successful, the targetRow field is saved with the translation result as the value of targetRow.selectionTitle.

When running the test from Scaffold, if the value of targetRow.selectionTitle shows up in the object passed to Zotero.selectItems(), the branch with the additional test is taken, and that specific item is translated, and its output checked against targetRow.item.

If the test case indicates "targetRow" but no matching row label or "selectionTitle" is found among the results, the test fails.

(Here the field name "selectionTitle" is based on the convention of naming this string title in the translator code, but the name title is a private variable to the function and maybe not very descriptive in the context of test cases. Perhaps an alternative field name would be "rowLabel" -- the text label of a row in the item selector).

[zoe-translates]

Oh, if we go along this, it's likely a better idea to use the URLs (properties keys) than the text labels or "title" (values) as the index to the specific search result. The URLs are unique, but titles may duplicate each other.

[AbeJellinek]

zotero/translators@f5d465b

We should at least support opt-in testing of specific returned items when the results should never change (book chapter multiples, sites where results can be sorted by publication date ascending, ...).