New release: WikiMed by Kiwix 2.2.9 with Full-Text search!

[Jaifroid]

Image: Full-Text search in the Linux app running under Ubuntu 22.0.4

The WikiMed app for Windows and Linux is now at version 2.2.9

The headline feature is that, for the Electron apps (for Linux and Windows), the openzim/javascript-libzim project has enabled us to include Full-Text search of the MDwiki archive (December 2022).

The Electron versions work with Windows 7/8/10/11 and Linux. The Linux packages have been tested on Ubuntu, Debian, Fedora and OpenSUSE (32bit and 64bit).

The packages contain the January 2023 WikiMed English ZIM (mdwiki_en_all_maxi_2023-01), together with the changes in the CHANGELOG. The Electron app uses Electron v18.3.15.

Release: https://github.com/kiwix/kiwix-js-windows/releases/tag/v2.2.9-WikiMed

Permalinks:

• https://kiwix.github.io/kiwix-js-windows/wikimed-uwp.html
• https://kiwix.github.io/kiwix-js-windows/wikimed-electron.html

Or open Command/PowerShell and type:

• winget install kiwix.wikimed.electron (in certification)

[Jaifroid]

@WikiDocJames This release should be of particular interest to you! It's our first attempt to integrate the full libzim library into the Electron apps. It's providing Full-Text search for archives that have an FT Index. It's still experimental, and may occasionally get stuck (restart app if it appears only to be showing title/prefix-style searches). It should work fine in the Electron apps for Windows and Linux (but not in the UWP/Microsoft Store app).

The Electron apps have the archive mdwiki_en_all_maxi_2022-12.zim, while the UWP/Mobile app has the smaller archive mdwiki_en_all-app_maxi_2022-12.zim without an FT Index.

As you've used the app before, there is a small bug that may force you to Reset the app (button in Configuration) after you first load, due to a pre-existing profile that is not fully erased currently. If you can't get to Configuration, just reboot the app, and then you should be able to get to the Reset button.

Any testing/feedback/comments/issues gratefully received!

[WikiDocJames]

I am trying to reset the app as you mention. I am looking under the wrench and not seeing that option. Currently I am seeing mdwiki_en_all-app_maxi_2022-11.zimaa being used.

[Jaifroid]

@WikiDocJames Apologies for this, that unfortunate bug is fixed in an emergency re-release of the app I made - 2.2.62. However, the Reset button is at the very bottom of the Config page, "Reset App", and that should also fix it

[WikiDocJames]

Perfect got it working. Very nice. J

…

On Fri, Dec 16, 2022 at 11:11 PM Jaifroid ***@***.***> wrote: @WikiDocJames <https://github.com/WikiDocJames> Apologies for this, that unfortunate bug is fixed in an emergency re-release of the app I made - 2.2.62 <https://github.com/kiwix/kiwix-js-windows/releases/tag/v2.2.62-WikiMed>. However, the Reset button is at the very bottom of the Config page, "Reset App", and that should also fix it [image: image] <https://user-images.githubusercontent.com/4304337/208228057-e6dc3a0f-c4fb-4652-b432-46aeb15873fe.png> — Reply to this email directly, view it on GitHub <#326 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFAWZSZFO5Y2XL6DE4THCO3WNVKQ7ANCNFSM6AAAAAAS66QRBU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- James Heilman MD, CCFP-EM, Wikipedian

[kelson42]

@Jaifroid I don't have checked, but I hope you don't mix (Xapian based) suggestions and (Xapian based as well) fulltext search feature. The first index is only based on titles and se second on the article contents. Or do you display suggestions (titles) based on the fulltext search results?

[Jaifroid]

@kelson42 It's a two-stage process in this implementation. First the backend returns any exact title-search matches (against the search, technically the search prefix, entered by the user). In parallel, a full-text search is being run, which is slower. The title-search results are displayed very fast, and usually it returns what the user is looking for almost instantly. If more results are needed (controlled by the params.maxSearchResults variable that users can set via the UI), then the slower full-text search results are added to the list as they arrive, below those returned from title search, filtering out any Directory Entries we already got from title search. An example is below, with a search for antibiotic resistance. I've drawn a red line to show where the title search results end and the full-text ones begin.

This is a simple but functional implementation which integrates full-text search into our existing search results UI. It doesn't have any fancy features like snippets, yet, as the bindings are not yet implemented in libzim and would probably require some thought on how best to display them. If you think it's necessary to distinguish title search and full-text search results, I could probably add some visual clue, but I'm not sure what purpose it would serve. Both types of search return a Directory Entry that we have to parse in the same way to produce the search results UI, and it make sense to show exact matches above fuzzy matches, as the list below clearly demonstrates. Most users will go straight to the first result. Users who are researching a topic in depth are more likely to look down the list.

If you'd like to discuss alternative search interfaces, perhaps we could open a dedicated issue.

[Jaifroid]

PS I forgot to specify that the title search results (above the red line) are those produced by our legacy backend, and have nothing to do with Xapian. The full-text results are those produced by the new libzim WASM. A number of our platforms and apps can't use the libzim WASM (e.g. UWP app, or Internet Explorer users, or older Firefox versions), so those users would only see the search results above the red line.

I don't think we get "suggestions" out of the libzim WASM implementation, we only get the full-text search results for now (depends on which bindings have been implemented -- see openzim/javascript-libzim#11).

[kelson42]

@Jaifroid Thank you for the explanation. I have no strong opinion on all of this. My feeling is that it should better be clear for the user that these results are based on a fulltext search whereas the other are based on the title only.

[Jaifroid]

@kelson42 I opened kiwix/kiwix-js#947 for that.

[Jaifroid]

Now updated to v2.2.9-WikiMed with the January 2023 WikiMed archive. Changelog:

Release 2.2.9-WikiMed

• UPDATE: Packaged archive updated to mdwiki_en_all[-app]_maxi_2023-01.zim
• ENHANCEMENT: Run full-text and title search in parallel for faster return of results
• BUGFIX: Fix libzim loading error in Electron app with large ZIM archives due to race condition
• FIX: Prevent Electron app from rebooting on every load to reactivate the Service Worker
• FIX: Add BOM to new JS files to conform to Microsoft Store requirements
• FIX: Exception caused by site.js script included in Wikimedia archives

https://github.com/kiwix/kiwix-js-windows/releases/tag/v2.2.9-WikiMed