Add tools calling for dspy.LM

[chenmoneygithub]

Add tools arg for dspy.LM, since litellm natively supports tool calling. We may also consider have dspy.ReAct use the default tool calling from LLM providers for robustness.

A sample code looks like:

import dspy

lm = dspy.LM("openai/gpt-4o-mini")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    }
]

outputs = lm("What's the weather like in Paris today?", tools=tools)

print(outputs)

The output is:

(dspy) [add-tool-calling][~/Documents/genai/dspy]$ python3 tool_tmp.py
[{'text': None, 'tool_calls': [ChatCompletionMessageToolCall(function=Function(arguments='{"location":"Paris, France"}', name='get_current_weather'), id='call_s7pLflqklT2MekZt8eIOBQDU', type='function')]}]

[okhat]

Thanks @chenmoneygithub ! Right now, we have a general solution that works well.

I'm not sure we should replace it with a special solution that doesn't work for many model providers.

[chenmoneygithub]

Yes that's a good point. Yuki from Mlflow team reached out asking if we can use the standard tool calling supported by litellm so that we can trace the tool calls. Which kinda makes sense because now we are mixing the tools calls into the message.content, with this approach it's hard to trace the tool calls.

My plan is this:

• for OAI/Anthropic/Databricks providers, which has standard function calling, we can use the standard way, which is identical across these providers.
• for other models, e.g., local hosted models, we use our current logic.

I am seeing two downsides:

1. When we change the logic in dspy.ReAct, there might be some performance change (drop or increase, I am not sure), we need to be cautious.
2. dspy.ReAct will have 2 branches, one for LLms that support tool calling, the other for LLMs that don't support tool calling. So the code will be slightly more complex: https://github.com/stanfordnlp/dspy/blob/main/dspy/predict/react.py#L85-L96.

And I am seeing 2 benefits:

1. We can enable clean tool calling tracing, which is useful for debugging complex agents.
2. I expect the tool calling to be more robust with OAI/Anthropic if we use their protocal.

Please let me know your thoughts!

[okhat]

I think the if statements here need to be refactored. Right now, you can only ask for logprobs or tools, but not both.

I think we should move the if statement about logprobs and tools etc inside the loop over choices?