Devanagari support

[sudanchapagain]

Opening this issue to track status of Devanagari support. I had emailed Mitchell before and he had responded that he will look into this.

Could Devanagari support be considered for Ghostty? Currently, the main available option for Devanagari compatibility in terminals is a font repository on GitHub (monotty/fonts). However, this project notes that their fonts aren't ideal to display Devanagari, as indicated by a recent update:

UPDATE: [May 2024]
Practice has shown that for correct display of the Devanagari script in the terminal, there is no need for special fonts - those proportional ones that exist are quite sufficient (e.g., Noto Sans Devanagari). The only thing that is needed is the support for tailored grapheme clustering on the terminal side. Here’s a description of the approach: https://gitlab.freedesktop.org/terminal-wg/specifications/-/issues/23. The fonts in our repository are not correct, do not use them. Maybe in the future we’ll make our fonts proportional.

Current status on Ghostty:

with both noto serif devanagari & noto sans devanagari installed on Linux (GNOME DE)

deva.webm

Appendix

sample tex

नेपाल एक सुन्दर देश हो। यहाँ हिमाल, पहाड, तराई र धेरै सांस्कृतिक विविधता छन्। नेपाली भाषाले धेरै किसिमका लोककथाहरू र परम्पराहरूलाई समेटेको छ। भारतं ज्ञानं धर्मं च प्रतिष्ठानम्। संस्कृतं पुरातनं भाषा अस्ति, यत्र वेदाः उपनिषदः च निर्मिताः। संस्कृतभाषायां अनेकाः ग्रन्थाः लिखिताः सन्ति।

[qwerasd205]

Those numbers in triangle brackets are a result of the shell. Place your sample text in a file and use cat to print it out in order to determine if Ghostty itself displays it correctly.

[mitchellh]

This looks pretty good for me. It looks better than iTerm, Kitty, and WezTerm at least.

CoreText:

Harfbuzz:

Is this not correct?

I'm not sure if Devanagari is RTL but note Ghostty only supports LTR and forces it.

[mitchellh]

And to reiterate, what @qwerasd205 said is correct. The weird numerical brackets you're seeing in your screenshot are your shell doing that, not Ghostty. You have to put it into a file and cat it to see what Ghostty would do.

[sudanchapagain]

Extremely sorry for my lazy issue. I should have researched more about the shell being responsible.

I'm not sure if Devanagari is RTL but note Ghostty only supports LTR and forces it.

Devanagari is LTR.

Is this not correct?

sadly no. It has spacing problem. I think this is what the people behind the font were talking about. Devanagari's glyphs on themselves take 1 single place (not terminal, generally) but when you add another glyph to it and if it is composable then they compose together to form new character or some form of merged glyph. That creates spacing problem. This Wikipedia article is better than my explanation https://en.wikipedia.org/wiki/Devanagari_conjuncts

in simple terms, this happens

the dis-continuos (dot dot dot thing) line is not meant to be written or rendered. it is just for representation.

With my no form of experience or knowledge, i think that is whats happening here. Sometimes the characters are way too close when they should not be and when there should be no space it has inserted spaces. To identify spaces, you should look into the top bar of words. In most cases (only very few exceptions) Devanagari words have a bar on top on themselves. Each continuous bar represents a single word.

ignore the grey lines in noto serif one. i forgot to erase them

edit: just checked Devanagari page too has few information on this. for example: the vowel diacritics

https://en.wikipedia.org/wiki/Devanagari

the first character is the first letter of the Devanagari script with no vowel and rest are same character but a new vowel being added.

in above ghostty's and harfbuzz rendering we can see how, diacritics that are present in top of the character or bottom of them are absolutely fine but those who modify the horizontal space, they start to create a problem. the second example's second word (the second example's first red line. the fat one) is the worst in this regard because it has three horizontal vowels in one small word and in ghostty's rendering or harfbuzz they are squeezed together.

Half characters too create problem. see first example's second red line. halant is responsible for half letters, in most cases adding halant will just mean a small diacritic will be added to the bottom of the letter but some characters like na (the first example's second red line) it transforms the letter itself. https://en.wikipedia.org/wiki/Virama

[mitchellh]

Thank you, can you help by pasting some of the problematic characters one at a time in a monospaced block like the following:

A
B
C

Since I can't easily type these, that would help immensely in my research of this problem.

[sudanchapagain]

Fixing just dependent vowels should fix a lot. Pasting everything is huge, 35*35 (Biconsonantal conjuncts) + like 30 or so Tri-, tetra- and penta-consonantal conjuncts + 10 or so irregular ligatures + 6*35 (horizontally space modifying vowel and consonant combinations). thats roughly 1500 characters (from my calculation). I will try to get all of it but will take a bit of time.

I think looking into just vowels for now, that too the ones who modify horizontal space and again that too for only one character and then testing it with other combinations will be better.

below is all dependent vowels that are connected with first consonant of the devanagari script. as in (a e i o u) connected with a.

dependent because vowels by themselves are written like normal characters and as such are not problematic. as in (a e i o u by themselves are written like bcd but when connected to letter they are written differently which is the primary issue)

का
कि
की
कु
कू
कृ
कॄ
के
कै
को
कौ
कं
कः

the ones who modify the horizontal space and to be tested

का
कि
की
को
कौ
कः

until i make the list, you want to look into consonant + consonant conjuncts this wikipedia page has it https://en.wikipedia.org/wiki/Devanagari_conjuncts

[sudanchapagain]

I just tested consonants too and its not just vowels, there is a problem with consonants too. the second consonant and third consonant too has problem. ख + ग when next to each other should appear as this खग (i hope GitHub is rendering it correctly). In ghostty it is rendered as this:

this means, the problematic characters are not 1500 or so but even more. I don't think this is worth it to fix now and i don't want to waste your time , but if you do want to, i will take time out to test all character combinations but it is a lot and i think it is a significant time investment for you.

[mitchellh]

Thanks @sudanchapagain. No, I think it's enough to know what's already not working. That gives us enough to work through on why we aren't shaping these and what's causing it. Once we figure that out, I think its highly likely the rest of the language "just works"