fix(text/unstable): handle non-Latin-script text in slugify

[lionel-rowe]

Fixes #5830

Changed tests:

• "déjà vu" is now déjà-vu, not deja-vu (and same for other diacritic tests). This was the most common handling of Latin-text diacritics in the sites I checked in text/slugify gives empty results for non-Latin alphabets #5830 (comment).
• "$33,000" is now 33-000, not 33000. The Unicode category of , is "Punctuation (Other)", which is very broad and IMO should generally replaced with "-" rather than "". I don't think it's probably desirable to special-case individual characters beyond Unicode category as it just ends up with a load of hard-coded and largely arbitrary chars (with the exception of straight quote marks, which are sort-of in the "wrong" category as quote marks typically fall in "Punctuation (initial)" or "Punctuation (final)"). Also, while there isn't much to choose between 33-000 vs 33000 (33000 is more natural, but 33-000 more readable due to retaining the thousands-place delimitation) , is used as a decimal point in many languages, and la-valeur-de-pi-est-3-14 is clearly preferable to la-valeur-de-pi-est-314.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.31%. Comparing base (8abff45) to head (2cb9e9b).
Report is 8 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5880   +/-   ##
=======================================
  Coverage   96.30%   96.31%           
=======================================
  Files         483      483           
  Lines       39059    39060    +1     
  Branches     5713     5713           
=======================================
+ Hits        37615    37619    +4     
+ Misses       1399     1396    -3     
  Partials       45       45           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[timreichen]

See #5830 (comment)
I am against this change, because this use case is better handled with @std/text/to-kebab-case

import { toKebabCase } from "@std/text/to-kebab-case";
console.log(toKebabCase("déjà vu")); // "déjà-vu"
console.log(toKebabCase("$33,000")); // "33-000"

slugify() is supposed to match all characters to [a-zA-Z0-9-] and remove all others.

[iuioiua]

@lionel-rowe, I think we should first come to an agreement on the solution in #5830 before opening a PR. Are you alright to close this for now?