[General] Custom normalizers?

[colexbruhn]

What would you like to discuss?

Hey ParadeDB team!

I was wondering if it would be possible to implement custom normalizers for text fields.

I work with a lot of acronyms that have lots of little variations which necessitates a bit of preprocessing (just removing non alphanumeric characters and converting to upper case). I currently have a column for the raw acronyms and a separate column for the processed acronyms that I index over with parade.

My thinking is that a custom normalizer would simplify this process since I would only need to store a single non-normalized column for the acronyms and index over that with the custom normalizer. So in other words, the normalized version would still be present in the index, but it would just be abstracted away from the end-user which makes for a nice little QOL improvement.

Thanks for all your amazing work!

Full Name:

Cole Bruhn

Affiliation:

Lockheed Martin

[philippemnoel]

Hi @colexbruhn. Thank you for the suggestion and apologies for the delays in getting back to you.

Implementing this feature should be possible. Would you be open to doing a call to discuss your use case in more details? We'd like to properly understand what you're looking to do in order to build it correctly. You can email me at [email protected] to schedule a call.