Filebeat input parsing json error or escapement failure

[macaty]

1、Filebeat input parsing json error

error_message : Error decoding JSON: invalid character 'x' in string escape code

2、setps
-->Version:
docker.elastic.co/beats/filebeat:8.9.0

-->logs: log.txt
{"@timestamp":"2023-08-07T11:54:16+08:00","remote_addr": "101.89.13.92", "upstream_addr": "192.168.1.155:8030", "remote_user": "-", "host": "ds.xxxx.com", "time_local": "07/Aug/2023:11:54:16 +0800", "msec": "1691380456.103", "request": "GET /saas/log.gif?dcdid=239020&group=kankan&code=101&msg=beat%20accour%20timeout%20event HTTP/1.1", "status": 200, "body_bytes_sent": 0, "http_referer": "-", "http_user_agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36", "http_x_forwarded_for": "12.12.1.2","http_cookie": "tongji_globalid=BA9AF8E\xB2-6F54-6114-C32F-3514A5E39260", "request_time": 0.000, "upstream_response_time": "0.000", "upstream_cache_status": "-","http_Cdn_Tag": "TX", "http_cdn_src_ip": "12.12.1.2" }}
{"@timestamp":"2023-08-07T10:27:08+08:00","remote_addr": "22.22.22.22", "upstream_addr": "192.168.1.154:8030", "remote_user": "-", "host": "ds.xxxx.com", "time_local": "07/Aug/2023:10:27:08 +0800", "msec": "1691375228.917", "request": "GET /saas/waveuserstrategy?Version=2021032901&dcdid=1000326565&jjid=202000000&sjid=1239112&tid=154&agentid=110000031&clienttype=5&OutOfDate=0&pluglet=0&cid=1000326565&token=Lfokprd6wNaoTGnad%2bZGpBlmRTnVaEoLgubk1Ffu2O3QEdXFxPq9ApqQc7F%2fRsgiJ9a%2bXWnA3LYFyuBDWeqgAeUAN9bqtnvEP0sTMK2ZPB2Cvbci0UIoaUcr3KL%2baJmSkEUhga4Hb6Nmdwx8akBVbYx9MOme%2bRwivIz5TImPUjs%3d&bata=0001 HTTP/1.1", "status": 200, "body_bytes_sent": 73, "http_referer": "-", "http_user_agent": "Stock", "http_x_forwarded_for": "1.1.1.1","http_cookie": "TrainRedirctUrl_User=\xEF\xBD\x85\xEF\xBD\x8D\xEF\xBC\x90\xEF\xBC\x94\xEF\xBC\x93\xEF\xBC\x98\xEF\xBC\x93\xEF\xBC\x92; tongji_globalid=1474B015-C624-3D41-E535-B74C2CE4AA11", "request_time": 0.001, "upstream_response_time": "0.001", "upstream_cache_status": "-","http_Cdn_Tag": "TX", "http_cdn_src_ip": "1.1.1.1" }}
{"@timestamp":"2023-08-07T09:38:15+08:00","remote_addr": "61.151.164.141", "upstream_addr": "192.168.1.155:8030", "remote_user": "-", "host": "ds.xxxx.com", "time_local": "07/Aug/2023:09:38:15 +0800", "msec": "1691372295.048", "request": "GET /saas/waveuserstrategy?Version=2021032901&dcdid=1000326565&jjid=202000000&sjid=1239112&tid=154&agentid=110000031&clienttype=5&OutOfDate=0&pluglet=0&cid=1000326565&token=ptveUb%2fSwJkHp9CCqbQnX2IpjDGg0%2fiX2JDGrpgmx9kj9%2beCwSP5I2%2fSgE4jJ4PigN84d12IJF%2fXw5%2f1UB80gewCUqKWQPeYGDjBNWD5Kse7ZWRBanDt02zLdVzCn1lEgVzPQ4p9iGN4DdWi8AD6LdR0MDtAi1PjeZ0%2fE58hl7Y%3d&bata=0001 HTTP/1.1", "status": 200, "body_bytes_sent": 73, "http_referer": "-", "http_user_agent": "Stock", "http_x_forwarded_for": "1.1.1.1","http_cookie": "TrainRedirctUrl_User=\xEF\xBD\x85\xEF\xBD\x8D\xEF\xBC\x90\xEF\xBC\x94\xEF\xBC\x93\xEF\xBC\x98\xEF\xBC\x93\xEF\xBC\x92; tongji_globalid=1474B015-C624-3D41-E535-B74C2CE4AA11", "request_time": 0.000, "upstream_response_time": "0.000", "upstream_cache_status": "-","http_Cdn_Tag": "TX", "http_cdn_src_ip": "1.1.1.1" }}

-->config
filebeat.inputs:

• type: log
  enabled: true
  #tail_files: true
  paths:
  • /log.txt
    json.add_error_key: true
    json.keys_under_root: true
    json.overwrite_keys: true
    output.elasticsearch:
    xxxx

-->result
{"_timestamp":1691716066919000,"agent_ephemeral_id":"1e78c641-ec80-4c73-9192-851792eab6af","agent_id":"32d6213e-4919-473c-ae5d-663ffe408c4b","agent_name":"1da88ffb9649","agent_type":"filebeat","agent_version":"8.9.0","ecs_version":"8.0.0","error_message":"Error decoding JSON: invalid character 'x' in string escape code","error_type":"json","host_name":"1da88ffb9649","input_type":"log","log_file_path":"/emoney/openobserve/http.dssaas.xxxx.com.access.log","log_offset":2237205808,"message":"{"@timestamp":"2023-08-07T10:27:08+08:00","remote_addr": "22.22.22.22", "upstream_addr": "192.168.1.154:8030", "remote_user": "-", "host": "ds.xxxx.com", "time_local": "07/Aug/2023:10:27:08 +0800", "msec": "1691375228.917", "request": "GET /saas/waveuserstrategy?Version=2021032901&dcdid=1000326565&jjid=202000000&sjid=1239112&tid=154&agentid=110000031&clienttype=5&OutOfDate=0&pluglet=0&cid=1000326565&token=Lfokprd6wNaoTGnad%2bZGpBlmRTnVaEoLgubk1Ffu2O3QEdXFxPq9ApqQc7F%2fRsgiJ9a%2bXWnA3LYFyuBDWeqgAeUAN9bqtnvEP0sTMK2ZPB2Cvbci0UIoaUcr3KL%2baJmSkEUhga4Hb6Nmdwx8akBVbYx9MOme%2bRwivIz5TImPUjs%3d&bata=0001 HTTP/1.1", "status": 200, "body_bytes_sent": 73, "http_referer": "-", "http_user_agent": "Stock", "http_x_forwarded_for": "1.1.1.1","http_cookie": "TrainRedirctUrl_User=\xEF\xBD\x85\xEF\xBD\x8D\xEF\xBC\x90\xEF\xBC\x94\xEF\xBC\x93\xEF\xBC\x98\xEF\xBC\x93\xEF\xBC\x92; tongji_globalid=1474B015-C624-3D41-E535-B74C2CE4AA11", "request_time": 0.001, "upstream_response_time": "0.001", "upstream_cache_status": "-","http_Cdn_Tag": "TX", "http_cdn_src_ip": "1.1.1.1" }"}

[botelastic]

This issue doesn't have a Team:<team> label.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!