Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JsonReader with strictness set to LENIENT, #2769

Open
d-william opened this issue Oct 20, 2024 · 2 comments · May be fixed by #2779 or #2783
Open

JsonReader with strictness set to LENIENT, #2769

d-william opened this issue Oct 20, 2024 · 2 comments · May be fixed by #2779 or #2783
Labels

Comments

@d-william
Copy link
Contributor

Gson version

2.11.0

Java / Android version

any version

Used tools

none

Description

With a JsonReader with strictness = LENIENT, all values after a null are read as a single string begining by "null".

Expected behavior

Values are correctly read.

Reproduction steps

public static void test(Stream<String> stream) throws IOException {
    Iterator<String> iterator = stream.iterator();
    StringWriter str = new StringWriter();

    try (JsonWriter writer = new JsonWriter(str)) {
        writer.setStrictness(Strictness.LENIENT);
        while (iterator.hasNext()) {
            TypeAdapters.STRING.write(writer, iterator.next());
        }
        writer.flush();
    }

    JsonReader reader = new JsonReader(new StringReader(str.toString()));
    reader.setStrictness(Strictness.LENIENT);

    System.out.println(TypeAdapters.STRING.read(reader));
    System.out.println(TypeAdapters.STRING.read(reader));
  }

With stream = Stream.of("value1", "value2"); -> OK

value1
value2

With stream = Stream.of("value1", null);, -> OK

value1
null

(the printed null is really a null value, not "null" String -> OK)

With stream = Stream.of(null, "value1"); -> KO

null"value1"
java.lang.IllegalStateException: Expected a string but was END_DOCUMENT at line 1 column 13 path $

With stream = Stream.of(null, "value1", "value2"); -> KO (and so on)

null"value1""value2"
Exception in thread "main" java.lang.IllegalStateException: Expected a string but was END_DOCUMENT at line 1 column 21 path $

With stream = Stream.of("value1", null, "value2"); -> KO

value1
null"value2"

With stream = Stream.of("value1", null, "value2", "value3"); -> KO (and so on)

value1
null"value2""value3"

Same behavior happens with other json primitive type but not with json object or json array.
Save behavior happens in previous gson version with #setLenient(true);.

@d-william d-william added the bug label Oct 20, 2024
@Marcono1234
Copy link
Collaborator

Marcono1234 commented Oct 20, 2024

Not sure if this something which can be solved for JsonReader without causing backward compatibility issues.

Maybe the underlying issue here is how JsonWriter writes multiple top-level values: It just concatenates them without any separating whitespace. Not sure if that can be changed in a backward compatible way though.

Is your main use case to write multiple top-level values?
In that case a workaround (or maybe even proper solution) could be to create new JsonWriter for each top-level value (it wouldn't even have to be lenient then). However, this might cause some overhead due to all the needed new objects.

StringWriter str = new StringWriter();

while (iterator.hasNext()) {
    String value = iterator.next();
 
    JsonWriter writer = new JsonWriter(str);
    writer.setStrictness(Strictness.STRICT);
    TypeAdapters.STRING.write(writer, value);
    writer.flush();
    str.append('\n');
}

@d-william
Copy link
Contributor Author

d-william commented Oct 21, 2024

Actualy, fixing the thing on JsonReader side only is impossible i think, some cases will remain impossible to solve.
For example, a similare issue happens while writing two longs : write(1) then write(2) will produce nextLong() = 12 ...

I like your workaround, however, my case is to write and read multiple top-level values. So it is hard to implement the reading part because internaly JsonReader read into a buffer more bytes than necessary. So during the second new JsonReader(reader);, it will fail to read the next value because the inner reader has been consumed too much.

A solution in my implementation can be to write before each value an int indicating the number of bytes to read until the next value (instead of '\n'). And in the reading part, to give to JsonReader a Reader that artificially limit the number of bytes to read (by returning -1 when n bytes have been read).

Edit : Actualy using a single JsonReader with Strictness.LENIENT corretly handle multiple top level values separate by '\n'

IlCommittatore pushed a commit to IlCommittatore/gson that referenced this issue Dec 2, 2024
@IlCommittatore IlCommittatore linked a pull request Dec 2, 2024 that will close this issue
@Marcono1234 Marcono1234 linked a pull request Dec 26, 2024 that will close this issue
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants