Add test for arSolrExistsQuery #1854
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Set up docker-file which starts up AtoM on port 63001, solr with 3 instances on ports 8981-8983, and a simple php page on port 9001.
WIP Skeleton for CLI task and plugin added. CLI task creates new solr collection, but doesn't populate the index yet.
Converted elastic search information object model and all dependent classes to solr equivalent. WIP: arSolrPluginUtil still has a couple of Elastica references for search specific functions that need to be adapted/converted for solr. QubitSearch has not been converted, so no actual documents are added to Solr index yet. Changes will need to be made to either search itself or all of the update/populate methods inside models to add the documents to the solr index.
Added solr specific getInstance, enable, and disable methods. WIP: No documents are added to the solr index by the addDocument method in arSolrPlugin.
WIP: Data is being indexed using solr's REST API, but it is not searchable since copy fields have not been declared, and the schema is dynamic
fields to be created: - nested (fields: actorRelations, dates) - object (fields: alternativeIdentifiers, digitalObject, findingAid, i18n)
Fixed the multiValued parameter in the api request to convert binary to string accurately. Added code to skip adding fields to copy based on the include_in_all mapping value. Also added a function parameter for the stored api request parameter since the copy field does not need to be stored.
Adding a WIP solr CLI search tool to search solr. Deleted the experimental solr html search page since it is no longer needed.
Added a helper method for all the Solr HTTP requests and cleaned up the plugin code making the requests.
Updated solr CLI search task to use API requests instead of using SolrClient. Using edismax queries instead of regular solr queries to be able to apply boost to fields. Accepting command line field inputs is WIP.
Fixed CS-Fixer warnings
Added a class to handles Solr Queries and its parameters
arSolrSearchTask now accepts fields as a command line parameter. Also fixed minor bugs with the search task and arSolrQuery, and removed the 10 row default from the solr search handler.
Moved solr query related classes to its own folder, added solr query classes WIP: All solr query classes are just placeholder skeletons extending from arSolrAbstractQuery. Also, arSolrPluginQuery needs to adapt addAggs to set up all the appropriate aggregation parameters
Added Range Query support for Solr, and also moved methods for adding params to the Abstract Query class. Also cleaned up indentation. Note about TODO: Elastic Search uses a slightly different format than Solr for Range Queries so all calls to create a Range Query will have to be updated to use te Solr syntax instead of the ElasticSearch.
Added classes for ResultSet and QubitSolrSearchPager. ResultSet needs to be adapted to be able to ingest response correctly, and Qubit Types need to be filtered out to be able to display information correctly.
Added a getDocument function to arSolrResultSet that formats the documents from the result set into a nested associative array to match the structure expected in the templates.
Add a new arSolrResult class for use with some templates that interface directly with the result instead of interfacing with documents returned by the result set.
Updated arSolrMatchAllQuery to skip fields entirely and use lucene parser to fetch results. Also updated arSolrSearchTask to add support for MatchAllQuery.
Update the docker config to remove references to the old solr folder. Also remove the solr php package as it isn't needed (and available) anymore.
Fix search results skipping documents due to offset field in the incorrect location.
Updated arSolrBoolQuery to function with arSolrQuery, added support for should, and added rudimentary support for it in arSolrSearchTask.
Add lucene based existence query for arSolrExistsQuery
Update arSolrRangeQuery to be compatible with Elastica like Range Queries
The addSubProperties function calls in arSolrPlugin were setting stored to false which was causing several fields to not be stored while indexing. TODO Fixed this change, but multivalued fields were still broken with the generated query setting all fields as single valued, and indexed results still missing some info. Hard coded all fields as multivalued for now to get working search results, but this will need to updated.
Updated makeHttpRequest to use curl for better error handling. Also updated autocomplete fields to be multivalued since for scenarios when they have both referenceCode and titles, and performed some code cleanups in arSolrPlugin.
Update solr to only use multivalue fields where they are needed. TODO: arSolrPlugin includes a large list of fields that are being used to test field names for being multivalued. This should be updated to work via yml. arSolrMapping also has a change that sets all partial foreign type id's to be multivalued, which needs to be updated to apply that to only the required id fields.
Adds arSolrTermQuery for Term Queries using Solr. Also updated arSolrMatchQuery to be an extension of arSolrTermQuery, and updated arSolrQuery to provide more fuzzy results.
sbreker
commented
Aug 2, 2024
| { | ||
| return [ | ||
| 'New arSolrExistsQuery with blank field' => ['field' => ''], | ||
| 'New arSolrExistsQuery with null field' => ['field' => null], |
There was a problem hiding this comment.
@anvit and @melaniekung: thinking about this - should '' or null trigger a validation error in the constructor? What will happen if a query is executed against Solr with these values?
There was a problem hiding this comment.
hi steve, they both return parser errors
Adds PHPUnit tests for arSolrExistsQuery class. Refactor arSolrExistsQuery to return exceptions when query generation preconditions are not met before query statement is generated.
sbreker
force-pushed
the
dev/solr-plugin-wip-with-new-test
branch
from
185cc8d
to
491c4d5
Compare
anvit
force-pushed
the
dev/solr-plugin-wip
branch
2 times, most recently
from
d987262
to
28b10ea
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.