Many natural language text fields (such as "statusReason") have no language or direction support

[aphillips]

A number of natural language text fields appear in the document without support for language or base direction. For discussion of why this is important and some guidelines, see String-Meta. One example is:

6.2 Disputes
https://w3c.github.io/vc-data-model/#disputes

"statusReason": "Credential contains disputed statements",

The property statusReason can contain natural language text (not merely an identifier or value string, as in most other examples). Correct rendering in some languages depends on additional metadata.

This is I18N comment 580

[aphillips]

(please add the i18n-comment label)

[brentzundel]

Decision from F2F Barcelona:
The support for this depends on the underlying syntax standards.

• @msporny will submit a PR that adds language explaining how JSON-LD 1.1 addresses this.
• @chaals will submit a PR that recommends the approach an issuer who uses JSON should take to address this issue, along with an example.

[burnburn]

Hey @msporny @chaals I thought both of you said that Manu had submitted the second PR, but I can't find it. Can you please link it here?

[msporny]

Hey @msporny @chaals I thought both of you said that Manu had submitted the second PR, but I can't find it. Can you please link it here?

Not yet, working on it now.

[msporny]

I added an i18n considerations section in 1cadfb1. It provides some guidance wrt. how to handle i18n correctly based on work done in the W3C Publishing WG and W3C JSON-LD 1.1 WG.

Closing issue, please re-open if you disagree with the solution.

[chaals]

The current approach assumes that processors can handle HTML - it makes sense to test this in CR as implementation information so I am not re-opening.

An alternative would be to make strings into arrays or objects, taking some of the @language approach from JSON-LD 1.1 (which could be readily extended to handle base direction with something like @textDirection ...)

For cases where a field is essentially bilingual, this might be a better approach. Instead of requiring HTML support it requires specific ability to handle the structure. On the other hand, this would enable plain JSON to actually deal with language information, in a way that is compatible with JSON-LD (by requiring processors to implement whatever subset of the JSON-LD mechanism is chosen)...

[aphillips]

I don't have permission to reopen.

There are a number of problems with the text in the I18N Considerations section, which I will attempt to address with comments on 1cadfb1.

The other problem here is that none of the examples adopt recommended usage. I scanned the document closely looking to see if the spec defined any natural language fields itself (as opposed to just using them as examples) and I didn't see any specific fields that are normatively defined. Since the Web is made of strings, the use of enumerated (usually-English language) text for values is a common practice that is not "non-internationalized" in and of itself. These values do not require language or direction markup, since they are not intended for display to users.

However, many examples in the document do have natural language text in them (the cited statusReason is one such) and these really should be good examples if possible.

I also am not sure if some of these fields are not defined either directly or imported as examples from ancillary standards.

[burnburn]

Thanks to great and timely feedback from Addison and Richard, with the merge of PR #454 and PR #469 this issue is no longer a blocker for CR entry. It is, however, a blocker for CR Exit at this time since more improvement of the examples and I18N Considerations was requested.

[msporny]

Ok, so @chaals, @burnburn, and I had a chance to figure out how to get to a solution that would meet all of the following criteria:

• Work for JSON-LD implementations.
• Work for JSON implementations.
• Be likely to be used by JSON developers that don't care about i18n.
• Have a positive impact on i18n support and would be "The Right Thing To Do(tm)" long term.

We believe the following would most likely work:

• Use JSON-LD 1.1's scoped contexts feature to alias all of the language keywords, specifically: @value, @language, and presumably @dir (upcoming JSON-LD 1.1 feature).
• Suggest that both JSON-LD and JSON processors process objects that contain {value: "foo", "language": "ar", dir: "rtl"} in the same way (as an object that expresses language information about a literal.

This makes i18n support a first class feature in both JSON-LD and JSON with a good portability story to other syntaxes, and suggests a pattern that JSON developers are likely to use.

We're working on the details and a set of PRs to implement this now.

[BigBlueHat]

* presumably `@dir` (upcoming JSON-LD 1.1 feature).

Sadly @dir didn't make the cut... w3c/json-ld-syntax#11

However, we have been working with the I18N folks to find the current "best practices" for text direction in JSON and JSON-LD. The latest result being a "workaround" using RLM/LRM markers in the value plus the language:

{
  "value": "‏HTML و CSS: تصميم و إنشاء مواقع الويب",
  "language": "ar"
}

Other than that approach, the other one frequently recommended is the one you have in the text already--using HTML (or a subset) for conveying sufficient language/direction information. This approach has the advantage of supporting multi-language strings--which (to my knowledge) no other approach supports.

We're all still working through the pragmatics of implementing these things, so inputs and explorations are definitely helpful.

Cheers!
🎩

[msporny]

We're all still working through the pragmatics of implementing these things, so inputs and explorations are definitely helpful.

Update, there is now active discussion on how to address this here: w3c/rdf-dir-literal#3

I'm putting together a PR based on where the discussion is at present:

w3c/rdf-dir-literal#3 (comment)

[msporny]

PR is in... please review so we can close this issue: #641

[msporny]

The PR has been merged, marking 7 day close.

[burnburn]

Closing.