Structured_output in Agent

[acamenhas]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

llm_with_tools = self.llm.bind_tools(self.tools)
        prompt = ChatPromptTemplate.from_messages(
            self.aimessages
        )
        agent = create_tool_calling_agent(llm_with_tools, self.tools, prompt)
        agent_executor = AgentExecutor(agent=agent, tools=self.tools, verbose=False)
        response = agent_executor.invoke(self.params)

Description

I have this example of an agent with tools what works well.
How can I get a structured output of that?

System Info

langchain==0.3.12
langchain-community==0.3.12
langchain-core==0.3.25
langchain-google-genai==2.0.7
langchain-groq==0.2.1
langchain-openai==0.2.12
langchain-text-splitters==0.3.3

[mcavdar]

hi @acamenhas

1- You should pass llm to create_tool_calling_agent, not llm_with_tools.
From doc:
Note that we are passing in the model, not model_with_tools. That is because create_tool_calling_agent will call .bind_tools for us under the hood.

2- If you try to create a model with with_structured_output

model = ChatOllama(model="qwen2.5-coder").with_structured_output(Joke)
agent = create_tool_calling_agent(model, tools, prompt)

create_tool_calling_agent gives this error: ValueError: This function requires a .bind_tools method be implemented on the LLM.

I'm not sure if this explains it, but:
Under the hood, with_structured_output uses bind_tools to pass the given structured output schema to the model.

I'm not sure if this helps, but it works on my end:

from langchain_core.tools import tool
from langchain_ollama import ChatOllama
from typing import Optional
from pydantic import BaseModel, Field
from langchain_core.messages import HumanMessage

@tool
def change_cat_with_dog(a: str) -> str:
    """change cat occurrence with dog."""
    return a.replace("cat", "dog")


tools = [ change_cat_with_dog ]

class Joke(BaseModel):
  """Joke to tell user."""
  setup: str = Field(description="The setup of the joke")
  punchline: str = Field(description="The punchline to the joke")
  rating: Optional[int] = Field(
    default=None, description="How funny the joke is, from 1 to 10"
  )

model = ChatOllama(model="qwen2.5-coder").bind_tools(tools).with_structured_output(Joke)
response = model.invoke([HumanMessage(content="Tell me a joke about cats, and replace cat with dog")])

Output:
Joke(setup='Why did the dog stop in the middle of the road?', punchline='Because he saw a bouncy ball!', rating=7)

[acamenhas]

Big thank you for your time on this answer!

Great example, it worked perfectly!!!