JsonReader with strictness set to LENIENT,

[d-william]

Gson version

2.11.0

Java / Android version

any version

Used tools

none

Description

With a JsonReader with strictness = LENIENT, all values after a null are read as a single string begining by "null".

Expected behavior

Values are correctly read.

Reproduction steps

public static void test(Stream<String> stream) throws IOException {
    Iterator<String> iterator = stream.iterator();
    StringWriter str = new StringWriter();

    try (JsonWriter writer = new JsonWriter(str)) {
        writer.setStrictness(Strictness.LENIENT);
        while (iterator.hasNext()) {
            TypeAdapters.STRING.write(writer, iterator.next());
        }
        writer.flush();
    }

    JsonReader reader = new JsonReader(new StringReader(str.toString()));
    reader.setStrictness(Strictness.LENIENT);

    System.out.println(TypeAdapters.STRING.read(reader));
    System.out.println(TypeAdapters.STRING.read(reader));
  }

With stream = Stream.of("value1", "value2"); -> OK

value1
value2

With stream = Stream.of("value1", null);, -> OK

value1
null

(the printed null is really a null value, not "null" String -> OK)

With stream = Stream.of(null, "value1"); -> KO

null"value1"
java.lang.IllegalStateException: Expected a string but was END_DOCUMENT at line 1 column 13 path $

With stream = Stream.of(null, "value1", "value2"); -> KO (and so on)

null"value1""value2"
Exception in thread "main" java.lang.IllegalStateException: Expected a string but was END_DOCUMENT at line 1 column 21 path $

With stream = Stream.of("value1", null, "value2"); -> KO

value1
null"value2"

With stream = Stream.of("value1", null, "value2", "value3"); -> KO (and so on)

value1
null"value2""value3"

Same behavior happens with other json primitive type but not with json object or json array.
Save behavior happens in previous gson version with #setLenient(true);.

[Marcono1234]

Not sure if this something which can be solved for JsonReader without causing backward compatibility issues.

Maybe the underlying issue here is how JsonWriter writes multiple top-level values: It just concatenates them without any separating whitespace. Not sure if that can be changed in a backward compatible way though.

Is your main use case to write multiple top-level values?
In that case a workaround (or maybe even proper solution) could be to create new JsonWriter for each top-level value (it wouldn't even have to be lenient then). However, this might cause some overhead due to all the needed new objects.

StringWriter str = new StringWriter();

while (iterator.hasNext()) {
    String value = iterator.next();
 
    JsonWriter writer = new JsonWriter(str);
    writer.setStrictness(Strictness.STRICT);
    TypeAdapters.STRING.write(writer, value);
    writer.flush();
    str.append('\n');
}

[d-william]

Actualy, fixing the thing on JsonReader side only is impossible i think, some cases will remain impossible to solve.
For example, a similare issue happens while writing two longs : write(1) then write(2) will produce nextLong() = 12 ...

I like your workaround, however, my case is to write and read multiple top-level values. So it is hard to implement the reading part because internaly JsonReader read into a buffer more bytes than necessary. So during the second new JsonReader(reader);, it will fail to read the next value because the inner reader has been consumed too much.

A solution in my implementation can be to write before each value an int indicating the number of bytes to read until the next value (instead of '\n'). And in the reading part, to give to JsonReader a Reader that artificially limit the number of bytes to read (by returning -1 when n bytes have been read).

Edit : Actualy using a single JsonReader with Strictness.LENIENT corretly handle multiple top level values separate by '\n'