Boolean search? AND, OR

[ghost]

Does Meilisearch currently support boolean search? Like: Truman || Bond or Truman OR Bond or (Truman OR Bond) AND Comedy. I cannot get this to work in the Sandbox with the 5000 movie example index.

[Kerollmops]

Hey @Fossil01,

It is not something that Meilisearch wants to support, the engine aims at being the best user-facing search engine. Therefore, we target user queries and not SQL-like syntax queries.

However, the engine supports filters and facets maybe you can tag your documents and use that instead. You can read more on our dedicated documentation page.

[ghost]

Ah shame. Just need my users to be able to search for more than 1 term, so in case of a movie list they should be able to do something like:

Bond -Spectre to show all Bond films without the Spectre one. Or Bond + Kill to only find Bond films where the title also contains the word Kill.

[menturion]

That's disappointing.
Thought that Meilisearch could compete with Elasticsearch in terms of basic feature set.

[Kerollmops]

Meilisearch is designed for user-end instant search.
Why does one exclude the other?

Sorry, I was thinking about an e-commerce application or a blog search bar but you are absolutely right we should not exclude those use cases.

[...] to allow users to narrow their search by means of Boolean operators.

I understand this use case. Which kind of platform do you manage? Are your users aware of the AND and OR syntax?

I added your feature idea to our roadmap prioritization table, we will see what we can do about it. As we are rewriting a big part of the search system we will be able to introduce a bunch of very interesting and important features in the future.

[menturion]

Which kind of platform do you manage?

Corporate intranet platform.

Are your users aware of the AND and OR syntax?

Yes, they are ... because it is important to find the relevant information.

[ghost]

My use case is exactly this. Front end user search where users can do AND/OR but also exclude words like on Google with the minus - character.

[Kerollmops]

Thank you very much for your answers.

Do you think that using the filtering syntax could do the job? Obviously, not for the word excluding feature, but for the AND and OR operations? By storing tags in a filterable field, for example?

[ghost]

Pretty sure that's not the same as true boolean search like Manticore (what i'm using now) or ES etc. offers.

[braco]

Also surprised this isn't supported. As searches get more complex, it can be very difficult to find the correct documents. Most users understand negation, it's been supported in search engines forever.

[machinedramon]

Hey @braco 👋

It seems that you are referring to negative keywords in the search query. Can you explain your use case, please? On which kind of platform are you using Meilisearch and why do you need this feature?

Hi! I would like to share my use case and explain why support for Boolean search is extremely important. I am developing a "Library" platform that contains tons of data on Books, Magazines, Journals, Newsletters, Videos, etc. Since we are in an academic environment, users are more accustomed to performing Boolean searches than regular searches, as they already know how to find the best results with their keywords. Currently, I have seen some information in the DOC about the use of filters, but I cannot apply them in any way. I am using Next 14.

[macraig]

@machinedramon thanks for sharing your use case. Can you expand on why you cannot apply filters in any way?

[machinedramon]

Hey @macraig, all my filterable attributes are correctly defined, and I can perform basic queries like this:

{
  "q": "text"
}

However, when I set up searches with filters, I only get exact match results. For example, if I want to search for the word "manual" within the title of a book, it doesn't return any results because it only works if I type the full title:

{
  "q": "",
  "attributesToRetrieve": ["title"],
  "filter": "title = \"MANUAL DAS ÁREAS DE PRESERVAÇÃO PERMANENTE\""
}

Response:

{
  "hits": [
    {
      "title": "MANUAL DAS ÁREAS DE PRESERVAÇÃO PERMANENTE"
    }
  ],
  "query": "",
  "processingTimeMs": 0,
  "limit": 20,
  "offset": 0,
  "estimatedTotalHits": 1
}

In other words, I can't return a term that simply contains the searched term; it only works if I type it in full, which prevents me from doing a partial search using filters. I've tried doing this:

"filter": "title IN [\"MANUAL DAS ÁREAS DE PRESERVAÇÃO PERMANENTE\"]"

But it returns the same result.

The ideal return would be something like this:

{
  "hits": [
    {
      "title": "MANUAL DAS ÁREAS DE PRESERVAÇÃO PERMANENTE"
    },
    {
      "title": "MANUAL PRÁTICO DE DIREITO ELEITORAL"
    },
    {
      "title": "MANUAL DO CANDIDATO E DA CANDIDATA A VEREADOR(A)"
    },
    {
      "title": "MANUAL SOBRE A NOVA LEI DE LICITAÇÕES E CONTRATOS ADMINISTRATIVOS"
    }
  ],
  ...
}

This way, I could use != to perform a negative search and find all titles that do not contain the searched term:

{
  "q": "",
  "attributesToRetrieve": ["title"],
  "filter": "title != \"manual\""
}

I need to perform a search like:

manual AND direito

and get as a result all the books that contain both 'manual' and 'direito' in the title, even if they contain other words as well.

manual NOT direito

I also want to be able to do searches like this, where I bring up everything that contains 'manual' in the title but does not contain the term 'direito'.

Any advice or suggestions on how to achieve this with Meilisearch would be greatly appreciated!

[machinedramon]

@macraig I found attributesToSearchOn, and this helped me to return all results that match the searched term:

{
     "q": "manual",
     "attributesToRetrieve": [
             "title"
         ],
    "attributesToSearchOn": ["title"]
}

But how to negate and retrieve results that doesn't match one more term? like:

{
     "q": "manual NOT direito",
     "attributesToRetrieve": [
             "title"
         ],
    "attributesToSearchOn": ["title"]
}

And how to return all results that don't match only the searched term? like:

{
     "q": "NOT manual",
     "attributesToRetrieve": [
             "title"
         ],
    "attributesToSearchOn": ["title"]
}

I need to combine filters for this? Is there any code example?

[macraig]

Hi @machinedramon !

In other words, I can't return a term that simply contains the searched term; it only works if I type it in full, which prevents me from doing a partial search using filters.

Is there a reason why you would need to apply the filter in this case? If you do a simple search where the query is "manual" you will get documents that have "manual" in the title.

If you absolutely need to use filters, I think you need the CONTAINS filter. We're planning to add this as an experimental feature in our next sprint.

But how to negate and retrieve results that doesn't match one more term? like:

This can be solved by using the negative keyword operator. You can search for manual -direito to get all documents containing "manual" excluding the ones that contain "direito".

[itsfolf]

+1 on this, I don't really understand how being a user-facing search engine would exclude features like this, regardless of the exact syntax or API that ends up being implemented. This type of expressions are provided by many search engines such as Twitter's and are very useful to filter down to specific content.
Currently, meilisearch already supports some form of OR, with matchingStrategy last, and AND, with phrase queries or matchingStrategy all, however, there is no way of combining these into more complex queries, such as apple OR orange tree or even apple OR orange tree OR fruit. It seems like #544 might cover some of these use cases, although I am not sure how chaining several CONTAINS filters to implement examples like the above would work out performance wise.

[macraig]

Hi @itsfolf , I might be missing something here, but if you're considering the CONTAINS filter as a solution, why not use the AND and OR filters?

[itsfolf]

Hi @itsfolf , I might be missing something here, but if you're considering the CONTAINS filter as a solution, why not use the AND and OR filters?

Yes chaining the CONTAINS filter with AND or OR would work for my use case (i.e. (body CONTAINS 'apple' OR body CONTAINS 'orange') AND body CONTAINS 'tree' for my first example), my only concern there becomes performance, which has been partly discussed in #544. There are a lot of documents and the body can be fairly long text, I'll see if I can do some testing with the prototype later this week, but perhaps it's also true that meilisearch is just not the best fit for my use.

[macraig]

@itsfolf I see. As for the performance concerns, it would be great if you could test the prototype and give us your feedback. We're also planning to test it further during our next sprint and release the feature as Experimental in v1.10 (which will be in pre-release in August).

[bedus-creation]

I have written a blog post about implementing OR condition with meilisearch https://dev.to/bedram-tamang/implementing-boolean-or-search-in-meilisearch-4hfn
we are using this solution until meilisearch implements these features.

[dureuill]

Hello @bedus-creation 👋

Thank you for the nice article; it's appreciated!

Note that the recently released Meilisearch v1.10 introduces federated search as an extension of multi-search.

Unlike regular multi-search, federated search unifies the result of all queries in a single list of results. This simplifies your code in two ways:

1. You don't need to do the merging yourself.
2. Documents are already deduplicated
3. The number of search queries can be reduced.

Taking the PHP OR engineer OR manager example from your blog post, it could be implemented thusly:

POST /multi-search
{
  // passing a "federation" object enables the federated search and merge all lists of results
  "federation": {
    "limit": 20,
    "offset": 0
  }
  "queries": [
    {
      "indexUid": "jobins_data",
      "q": "PHP" // return documents that definitely contain PHP, but may also contain engineer and/or manager
    },
    {
      "indexUid": "jobins_data",
      "q": "engineer" // return documents that definitely contain engineer, but may also contain PHP and/or manager
    },
    {
      "indexUid": "jobins_data",
      "q": "manager" // return documents that definitely contain manager, but may also contain PHP and/or engineer
    }
  ]
}

Of course, that's still one search request per keyword, but I don't think we can do much better even as a native feature.
Thanks again for your blog!