Optimize StrawberryFlavor (OCR, etc) Indexing/Updates and aggregation #256
Labels
enhancement
New feature or request
Events and Subscriber
JMESPath
a.*.wonderful.life[0]
JSON Postprocessors
Drupal Plugins that do stuff with JSON data
Property Keys Providers
Drupal plugins that harvest the possible keys that will be exposed to the rest of Drupal
Search API
F around and find out
Strawberry Flavor
Post Processing data extracted that goes into Solr
Typed Data and Search
Milestone
What?
For the sake of consistency we do a lot of update/tracking/parent and child re-tracking on any changes on ADOs and Flavor Data Sources. But on a large production server a few ADO updates that really don't affect at all Flavors end reindexing 10K++ (add an extra +) Flavor Documents and the number of Queries/Back and forth that are happening are/might end being too much with danger of timeouts and server slowdown.
We can Optimize and make better decisions on when these heavy Server tasks are needed using more logic/conditionals and maybe even aggregation tasks into single backend queue items to avoid so much processing.
Pieces we need to touch are:
https://github.com/esmero/strawberryfield/blob/1.1.0/src/Plugin/search_api/processor/StrawberryFlavorAggregate.php
https://github.com/esmero/strawberryfield/blob/1.1.0/src/EventSubscriber/StrawberryEventSaveFlavorSubscriber.php
https://github.com/esmero/strawberryfield/blob/1.1.0/src/Plugin/search_api/datasource/StrawberryfieldFlavorDatasource.php
And their interactions. There is a lot of Dinosaur-egg-Dinosaur situations in these parent-child interactions and sometimes the lack of context (is the operation an Update or a First Time Index?) makes all more complex
@alliomeria @aksm @karomabiles This is what was discussed today
Thanks
The text was updated successfully, but these errors were encountered: