Problem with Vietnamese preset in font builder

[brianhuster]

Affected Version

Master

Describe the Bug

Crosspoint font builder has a range for Vietnamese language ([(0x01A0, 0x01B0), (0x1EA0, 0x1EF9)]), however it only covers letters specific to Vietnamese. This means many accented letters also needed in Vietnamese like

Đ đ → U+0110 U+0111
Ă ă → U+0102 U+0103
 â → U+00C2 U+00E2
Ê ê → U+00CA U+00EA
Ô ô → U+00D4 U+00F4

are not included in the font if users only choose "Vietnamese" range, and they will still see "tofu" when reading books with these characters. To cover all Vietnamese characters, users need to choose "Latin Extended" or "reading" box, but then "Vietnamese" box is just redundant, as both of those presets already cover all Vietnamese characters

Steps to Reproduce

See the code https://github.com/crosspoint-reader/crosspoint-reader/blob/62e84a16f86fe8d205e53770c0eaa320c658cdca/lib/EpdFont/scripts/fontconvert_sdcard.py#L39

Expected Behavior

We have 2 choices:

• Just remove Vietnamese preset, as Latin Extended and "reading" already covers all Vietnamese characters
• Extend Vietnamese preset to cover all Latin characters needed to write Vietnamese. But I don't recommend this because there are Vietnamese books that use accented letters not found in Vietnamese like ñ, ç when writing foreign names.

Relevant Log Output/Screenshots

[Uri-Tauber]

I dont really have a say in the matter, as I'm not a Vietnamese speaker.
I believe @jetaudio is a Vietnamese speaker, so I'm deferring to him the decision.

Decide between you what to do, and I will approve the PR you will make.

[brianhuster]

I believe @uxjulia is Vietnamese too, given the surname Nguyen

[uxjulia]

I don't think this issue requires that any of us actually be Vietnamese but sure, I'll chime in. I think it makes sense to have the Vietnamese preset include all of the necessary unicode interval ranges so users don't need to know anything about what the other presets include and have to do manual mixing and matching. If they only choose to select "Vietnamese" then they have everything they need. If we remove it altogether, the less technical users might not understand and think Vietnamese is not supported.

If I'm not mistaken, the "Reading" interval is always included as a base. So for this change, I think the Vietnamese range only needs to add the latin extended range and any other missing intervals.

Edit: Also, I'm not sure if the website's font builder is looking at this file anymore. I thought we had separated it, I need to double check where the actual change belongs.

Edit 2: can confirm the change belongs in the crosspoint-tools repo

[brianhuster]

If I'm not mistaken, the "Reading" interval is always included as a base

AFAIK the one always included as a base is "Base", that is much smaller than "reading" as it only has ASCII letters and some punctuations. Both "reading" and another preset called "default" already include all Vietnamese characters, that's why I suggested removing the Vietnamese preset, though I won't oppose just extending the Vietnamese preset to include all Vietnamese characters

I'm not sure if the website's font builder is looking at this file anymore. I thought we had separated it, I need to double check where the actual change belongs.

Yes, because of a comment from @Uri-Tauber, I thought the font-building script from crosspoint-tools repo is regularly synced with the one from crosspoint-reader, but I realize they have diverged

[Uri-Tauber]

Yes, because of a comment from @Uri-Tauber, I thought the font-building script from crosspoint-tools repo is regularly synced with the one from crosspoint-reader, but I realize they have diverged

Thats should not be the case.

[uxjulia]

If I'm not mistaken, the "Reading" interval is always included as a base

AFAIK the one always included as a base is "Base", that is much smaller than "reading" as it only has ASCII letters and some punctuations. Both "reading" and another preset called "default" already include all Vietnamese characters, that's why I suggested removing the Vietnamese preset, though I won't oppose just extending the Vietnamese preset to include all Vietnamese characters

I'm not sure if the website's font builder is looking at this file anymore. I thought we had separated it, I need to double check where the actual change belongs.

Yes, because of a comment from @Uri-Tauber, I thought the font-building script from crosspoint-tools repo is regularly synced with the one from crosspoint-reader, but I realize they have diverged

Yeah they've diverged since the sd card font generator in crosspoint doesn't need all the extra intervals that the font builder on the website provides. We decoupled the dependency so website updates can be made independent of the firmware code.

And you're right, "reading" isn't bundled in by default, only base is.