Complexe response format lead the container to run forever on CPU

[Rictus]

System Info

System:
Linux 4.18.0-553.22.1.el8_10.x86_64 #1 SMP Wed Sep 25 09:20:43 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Rocky Linux 8.10

Model: mistralai/Mistral-Nemo-Instruct-2407
Hardware:

• GPU: NVIDIA A100-SXM4-80GB
• CPU:

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              256
On-line CPU(s) list: 0-255
Thread(s) per core:  2
Core(s) per socket:  64
Socket(s):           2
NUMA node(s):        8
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               49
Model name:          AMD EPYC 7742 64-Core Processor
Stepping:            0
CPU MHz:             2250.000
CPU max MHz:         2250.0000
CPU min MHz:         1500.0000

Using text-generation-inference docker containers, the issue is reproduced with TGI 2.3.1 & 2.2.0

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

1. Run a TGI container

# Podman
podman run --device nvidia.com/gpu=4 --shm-size 64g -p 7085:80 -v /data/huggingface/hub/:/data ghcr.io/huggingface/text-generation-inference:2.3.1 --model-id /data/models--mistralai--Mistral-Nemo-Instruct-2407/snapshots/e17a136e1dcba9c63ad771f2c85c1c312c563e6b --num-shard 1 --trust-remote-code --env --max-input-length 63999 --max-total-tokens 64000 --max-batch-prefill-tokens 64000 --max-concurrent-requests 128 --cuda-memory-fraction 0.94 --json-output
# Equivalent using docker
docker run --gpus '"device=4"' --shm-size 64g -p 7085:80 -v /data/huggingface/hub/:/data ghcr.io/huggingface/text-generation-inference:2.3.1 --model-id /data/models--mistralai--Mistral-Nemo-Instruct-2407/snapshots/e17a136e1dcba9c63ad771f2c85c1c312c563e6b --num-shard 1 --trust-remote-code --env --max-input-length 63999 --max-total-tokens 64000 --max-batch-prefill-tokens 64000 --max-concurrent-requests 128 --cuda-memory-fraction 0.94 --json-output

2. Make a chat completion call with a complexe regex in response format:

curl -X 'POST' \
  'http://px101.prod.exalead.com:7085/v1/chat/completions' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mistralai/Mistral-Nemo-Instruct-2407",
    "messages": [
        {
            "role": "user",
            "content": "hello",
            "tool_calls": null
        }
    ],
    "temperature": 0.0,
    "top_p": null,
    "max_tokens": 2048,
    "stop": [],
    "stream": true,
    "seed": null,
    "frequency_penalty": null,
    "presence_penalty": null,
    "logprobs": false,
    "top_logprobs": null,
    "stream_options": null,
    "response_format": {
        "type": "regex",
        "whitespace_pattern": null,
        "value": "\\{([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\]([ ]?,[ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?|([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?[ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\]([ ]?,[ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?|([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?[ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\]([ ]?,[ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?|([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?[ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\]([ ]?,[ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?|([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?[ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\]([ ]?,[ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?|([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?[ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\]([ ]?,[ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?|([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?[ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\]([ ]?,[ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?|([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?[ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\]([ ]?,[ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?|([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?[ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\]([ ]?,[ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?|([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?[ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\]([ ]?,[ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?([ ]?,[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?|([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?[ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\]([ ]?,[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?|([ ]?\"a\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"b\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"c\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"d\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"e\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"f\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"g\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"h\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"i\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"j\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?([ ]?\"k\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\][ ]?,)?[ ]?\"l\"[ ]?:[ ]?\\[[ ]?((\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")(,[ ]?(\"([^\"\\\\\\x00-\\x1F\\x7F-\\x9F]|\\\\[\"\\\\])*\")){0,})?[ ]?\\])?[ ]?\\}"
    },
    "tools": null,
    "tool_choice": null,
    "logit_bias": null,
    "n": null,
    "parallel_tool_choice": null,
    "functions": null,
    "function_call": null
}'

JSON schema for above regex

{
    "properties": {
        "a": {"type": "array", "items": {"type": "string"}},
        "b": {"type": "array", "items": {"type": "string"}},
        "c": {"type": "array", "items": {"type": "string"}},
        "d": {"type": "array", "items": {"type": "string"}},
        "e": {"type": "array", "items": {"type": "string"}},
        "f": {"type": "array", "items": {"type": "string"}},
        "g": {"type": "array", "items": {"type": "string"}},
        "h": {"type": "array", "items": {"type": "string"}},
        "i": {"type": "array", "items": {"type": "string"}},
        "j": {"type": "array", "items": {"type": "string"}},
        "k": {"type": "array", "items": {"type": "string"}},
        "l": {"type": "array", "items": {"type": "string"}}
    }
}

Note that the issue is still present when passing the JSON schema instead of the regex, but for various reasons we compute the regex ourselves using outlines.build_regex_from_schema.

So when running above curl, TGI does accept the request and start processing. The %CPU goes to 99.7% and it stays like this for very long time until I decide to kill it. The container no longer accepts new requests and becomes useless.

Expected behavior

A timeout that would stop the processing before it runs for too long.
Not sure which kind of timeout is relevant but since no token is received, a timeout until first token is received would make sense for me.

Here's another schema that also triggers above issue but is more common

{
    "properties": {
        "main_intent": {
            "type": "string"
        },
        "expected_output": {
            "type": "string"
        },
        "key_entities": {
            "type": "array",
            "items": {
                "type": "string"
            }
        },
        "key_properties": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": {
                        "type": "string"
                    },
                    "comment": {
                        "type": "string"
                    }
                }
            }
        },
        "classes": {
            "type": "string"
        },
        "properties": {
            "type": "string"
        },
        "sparql_query_form": {
            "type": "string"
        },
        "variables": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": {
                        "type": "string"
                    },
                    "comment": {
                        "type": "string"
                    }
                }
            }
        },
        "properties_marked_as_optional": {
            "type": "array",
            "items": {
                "type": "string"
            }
        },
        "triple_patterns_for_relations": {
            "type": "array",
            "items": {
                "type": "string"
            }
        },
        "triple_patterns_for_classes": {
            "type": "array",
            "items": {
                "type": "string"
            }
        },
        "additional_properties": {
            "type": "array",
            "items": {
                "type": "string"
            }
        },
        "filters_and_constraints": {
            "type": "array",
            "items": {
                "type": "string"
            }
        },
        "output_variables": {
            "type": "array",
            "items": {
                "type": "string"
            }
        },
        "sparql_query": {
            "type": "string"
        }
    }
}