Search server

[art-w]

@sabine has a pending PR to add search to ocaml.org in packages documentation ocaml/ocaml.org#1096 using sherlodoc

This search uses the http rest api at https://doc.sherlocode.com which lies outside the ocaml infrastructure. We would like to deploy the sherlodoc executable closer to ocaml.org to avoid depending on an external service.

The main issue is the ~9go database file that needs to be accessible (otherwise we could just run the sherlodoc executable from the ocaml.org container) We aren't there yet, but we are planning to run the database creation as part of voodoo to keep it up to date. In the mean time, it would be nice if the search UI could be made accessible, for users benefit, to gather their feedback and improve upon in parallel...

Do you have some advice on how to proceed? :)

[mtelvers]

Why not wrap this into a Docker container and deploy it on docs.ci.ocaml.org? We could add the build/deployment to deploy.ci.ocaml.org.

[art-w]

Yes that will work thanks!

[tmcgilchrist]

@art-w please let me know if you need help with this. I am working on the docs-ci pipeline.

The main issue is the ~9go database file that needs to be accessible

Please share more details about how this database get generated. Is it alongside the docs ci pipeline outputs?

[art-w]

Thanks a lot @tmcgilchrist ! Currently I wrote a small dockerfile and looked into the existing Caddy config to redirect the rest /api calls to the sherlodoc container. I wanted to try and deploy that, against an existing database, but yes the next step was to generate the database from the docs ci pipeline (since its inputs are all the odocl files produced by it).

In the mean time, the ocaml.org team started playing with generating json search indexes from voodoo, such that the package search would be done straight from the browser in javascript. So it became less urgent to deploy sherlodoc -- I think we still plan to do it, to support "global opam documentation search" :)

Thanks again for proposing to help! Can I get back to you in a few days once I have a bit more time?

[sabine]

Indeed, Sherlodoc still needs to be deployed to enable a global search across opam package.

[avsm]

@art-w, I think this one's with you at the moment. Feel free to reassign if not.