cjson.safe encode and decode issues

[flowdopip]

Hi all.

I have a custom plugin to store the response body on Redis and in some use cases I get an error decoding the information stored on Redis with this error: bad argument #1 to '?' (string expected, got nil)

So when I get the response body with PDK I encode that on redis

  local body = kong.response.get_raw_body()
  local status = kong.response.get_status()
  if body then
    ngx.timer.at(0, store_cache_value, conf, ctx.req_body, status, proxy_cache)
  end

but when I fetch the data from redis with

local req_obj, error_decode= cjson.decode(req_json)

    if not req_obj then
... [here I get the error saying that I can not decode with ]bad argument #1 to '?' (string expected, got nil)

after further validation and test I can reproduce when the data as invalid json, but my point is why using kong.response.get_raw_body() I get invalid json.

thanks

[ProBrian]

Hi @flowdopip, seemskong.response.get_raw_body() doesn't guarantee that its return value is a valid json, it just returns a raw string. Could you try to print out the invalid json as string to make sure where the invalid logic happend: from the upstream server phase, the kong response phase, or the redis value storing phase?

[flowdopip]

Sure, I already did that; basically, What I got in debug mode was unfinished JSON valid, like missing a few properties and the last }'.

I write that debug log before the Kong response.get_raw_body(). To discard the upstream service, I run a bunch of calls to it through an automated tool, and I never get the unclosed JSON data. This has been happening for several upstreams on my Kong configuration. The ratio is around 1:1000.

[chobits]

I guess we should know what the raw body is when the problem occurs. Or adding some debug log or using some package capture tools to log the network flow. From current info, we could not detect whether the problem occurs inside Kong or Kong receives the invalid data.