Exporting doc information to use in models like SBERT

[mikesol]

I'm experimenting with a different indexing method of the docs using SBERT and FAISS. This is a popular approach for indexing/querying data these days, and while lots of companies offer paid versions of it, it's also fairly straightforward to do with open-source tools.

I'm wondering if it's possible to tweak doc search to dump the entire index in xml form. For example:

<entry>
  <package>purescript-lists</package>
  <module>Data.List</module>
  <id>singleton</id>
  <def>singleton :: forall a. a -> List a</def>
  <doc>Create a list with a single element.

Running time: `O(1)`</doc>
</entry>

It's easier to construct a data set for SBERT when data is formatted like this.

I see that a lot of the plumbing to do something like this is already there, but as I don't know the code well, it's tough to come up with a plan for a clean way to do it.

If you have pointers on how to hack at the repo to get there, I can give it a shot!