Add Trending Field to Solr

[benbdeitch]

Closes #7429

This PR adds support for trending scores to Solr, allowing us to better track which works are achieving a statistically notable increase in popularity. It adds several new fields, and comes with two scripts to be run-- one daily, the other hourly, to keep this information constantly up to date.

Currently, it's still in draft mode, as there is currently no code to automatically run the scripts.

Technical

This implementation uses Solr's ability to update documents in place, which requires the new trending fields to not be stored or indexed, and instead treated as a docValue. Essentially, they are left out of Solr's inverted index, and instead treated as a more usual document-to-value mapping.

This is both A) more performant than atomic updates, and B) avoids the issues that atomic updates can have with copyfield values.

The relevant cron commands are located in an added file, docker/cron.local

1. Delete your solr container and all related volumes.
2. Run docker compose up.
3. Going to your local solr instance, run a search for a work on Solr (e.g. key:"/works/OL54120W"), and check to ensure that the new fields are present.
4. Save a work to your 'want-to-read' list.
5. Set up a docker/cron.local file to run the cron jobs in, along with a new container. Change the times on the cron tasks to run more frequently; (* * * * *) will make them run every minute.
6. Make sure the container has access to both dbnet and webnet networks, and has depends on: db.
7. After a minute or so, run the search on Solr again, and see if the appropriate trending fields have updated. You can also check the logs of the cron-jobs container in Docker, to see if they're running correctly.

Screenshot

Stakeholders

@cdrini

[cdrini]

Niiiiice! Getting super close; next week after these changes, let's start adding these fields to prod solr I think!

[cdrini]

@benbdeitch notes this is for testing and should be deleted 😁

[cdrini]

Ran out of time will finish the rest

[cdrini]

Oh I forgot, also add a dummy override of the get_trending_scores method to

openlibrary/scripts/solr_builder/solr_builder/solr_builder.py

Line 56 in 694bd37

class LocalPostgresDataProvider(DataProvider):

with 0s