RequestMessage with tool_calls in Assistant message.

[Jeadie]

Describe the bug

• To follow OpenAI spec (and what other open source models, e.g. llama3.1), expect for Assistant messages with tool calls, messages should look like this

  "messages": [
    {
      "role": "assistant",
      "content": "",
     "tool_calls": [
        {
          "id": "call-243240c8",
          "type": "function",
          "function": {
            "name": "current_weather",
            "arguments": "{\"county\": \"somewhere\"}"
          }
        }
      ]
    }
]

In mistral.rs, we JSON string the full "tool_calls" value, e.g.

mistral.rs/mistralrs/examples/lower_level/tools/main.rs

Lines 127 to 139 in b6b722e

	messages.push(IndexMap::from([
	("role".to_string(), Either::Left("asistant".to_string())),
	(
	"content".to_string(),
	Either::Left(
	json!({
	"name": called.function.name,
	"parameters": called.function.arguments,
	})
	.to_string(),
	),
	),
	]));

In fact, MessageContent only supports String values.

This restricts the ability to feed the result of tool calls back to various models

Also value should be "arguments" and not "parameters" (parameters is used in JSONSchema for tools key.

[EricLBuehler]

Hi @Jeadie! Thanks for raising the issue. I was wondering if you could just modify the code like:

messages.push(IndexMap::from([ 
     ("role".to_string(), Either::Left("asistant".to_string())), 
     ("content".to_string(), Either::Left("".to_string())), 
     ( 
         "tool_calls".to_string(), 
         Either::Left( 
             json!({ 
                 "name": called.function.name, 
                 "arguments": called.function.arguments, 
             }) 
             .to_string(), 
         ), 
     ), 
 ]));

I think I'm a bit confused on the limitation here, as any JSON objects would need to be stringified eventually. Perhaps the above would work?

[Jeadie]

The issue is that certain common open-source Jinja chat_templates expect a non-serialised JSON object for tool_calls fields when converting RequestMessages to an unstructured string.

For example meta-llama/Llama-3.1-8B-Instruct has a chat_template, with the following

{%- if not message.tool_calls|length == 1 %}
    {{- raise_exception(\"This model only supports single tool-calls at once!\") }}
{%- endif %}

This is not checking the length of the JSON string is 1, but rather that the JSON array has one entry.