Help in getting a response that's not a tool_call

[marklysze]

When I am using tool_choice="auto" it is always returning a tool_call response. I am guessing it is a prompting issue, but is there some documentation on how to help the LLMs decide whether to make a tool call or not?

To Reproduce
I am testing a chess game between LLMs and there are a couple of functions which are used to 1. determine the available moves, 2. select and make a move. After those two have been called it should then respond with a message like "I have made my move, over to you!". However, it continues to respond with the second function.

import os
import json
import openai
import chess
import chess.svg
from IPython.display import display
from typing_extensions import Annotated
from together import Together

client = Together(api_key=os.environ["TOGETHER_API_KEY"])

# Initialize the board.
board = chess.Board()

# Keep track of whether a move has been made.
made_move = False

Here is where I'm up to with the messages - the next should be the text response rather than a tool call.

chess_messages = [
    {'content': 'You are a chess player and you play as white. First call the function get_legal_moves() first, to get list of legal moves. Then call the function make_move(move) to make a move. Finally, do not call a function just say "I have made my move, over to you!"', 'role': 'system'},
    {'content': "Let's play chess! Your move.", 'role': 'user'},
    {'tool_calls': [{'id': 'call_p1fla56o5jlugh9uekgo84c6', 'function': {'arguments': '{}', 'name': 'get_legal_moves'}, 'type': 'function'}], 'content': None, 'role': 'assistant'},
    {'tool_call_id': 'call_p1fla56o5jlugh9uekgo84c6', 'role': 'user', 'content': 'Possible moves are: g1h3,g1f3,b1c3,b1a3,h2h3,g2g3,f2f3,e2e3,d2d3,c2c3,b2b3,a2a3,h2h4,g2g4,f2f4,e2e4,d2d4,c2c4,b2b4,a2a4'},
    {'tool_calls': [{'id': 'call_lcow1j0ehuhrcr3aakdmd9ju', 'function': {'arguments': '{"move":"g1f3"}', 'name': 'make_move'}, 'type': 'function'}], 'content': None, 'role': 'assistant'},
    {'tool_call_id': 'call_lcow1j0ehuhrcr3aakdmd9ju', 'role': 'user', 'content': 'Moved knight (♘) from g1 to f3.'}
]

Here are the two functions

def get_legal_moves() -> Annotated[str, "A list of legal moves in UCI format (e.g. e2e4 or e7e5 or e7e8q)"]:
    return "Possible moves are: " + ",".join([str(move) for move in board.legal_moves])

def make_move(move: Annotated[str, "A move in UCI format. (e.g. e2e4 or e7e5 or e7e8q)"]) -> Annotated[str, "Result of the move."]:
    move = chess.Move.from_uci(move)
    board.push_uci(str(move))
    global made_move
    made_move = True
    # Display the board.
    display(
        chess.svg.board(board, arrows=[(move.from_square, move.to_square)], fill={move.from_square: "gray"}, size=200)
    )
    # Get the piece name.
    piece = board.piece_at(move.to_square)
    piece_symbol = piece.unicode_symbol()
    piece_name = (
        chess.piece_name(piece.piece_type).capitalize()
        if piece_symbol.isupper()
        else chess.piece_name(piece.piece_type)
    )
    return f"Moved {piece_name} ({piece_symbol}) from {chess.SQUARE_NAMES[move.from_square]} to {chess.SQUARE_NAMES[move.to_square]}."

and the tools construct to pass in:

tools = [
    {'type': 'function',
     'function': {'description': 'Call this tool to make a move after you have the list of legal moves.', 'name': 'make_move',
                  'parameters': {'type': 'object', 'properties':
                                 {'move':
                                  {'type': 'string', 'description': 'A move in UCI format. (e.g. e2e4 or e7e5 or e7e8q)'}
                                  },
                                  'required': ['move']
                                  }
                }},
    {'type': 'function',
     'function': {'description': 'Call this tool to make a move after you have the list of legal moves.', 'name': 'get_legal_moves',
                  'parameters': {'type': 'object', 'properties':
                                 {},
                                 'required': []
                                 }
                }
    }
]

And the create call:

response = client.chat.completions.create(
    model="mistralai/Mixtral-8x7B-Instruct-v0.1",
    messages=chess_messages,
    tools=tools,
    tool_choice="auto",
)

Any help would be greatly appreciated.

Desktop (please complete the following information):

• Ubuntu 22.04

Additional context
I'm working on a Together.AI client class for the Microsoft Research AutoGen project and would like to be able to demonstrate function calling effectively. Link to the working PR here