How to retrun streamed response in a function

[navaneethstabilix]

async for chunk in llm.astream("write an essay on sachin in 200 words"):
chunks.append(chunk)
print(chunk.content, end="", flush=True)

how to write this inside a def function and use return keyword instead of print but response must be streamed . need to send this req to websocket

[dosubot[bot]]

I found a similar unsolved discussion that might be relevant: How return LLMChain.run() stream data?? it did not store stream text in variable rather it just print?? ^[1].

To return streamed responses from a function and send them to a WebSocket, you can use an asynchronous generator function. Here's an example of how you might implement this:

async def stream_to_websocket(llm, websocket, prompt):
    async for chunk in llm.astream(prompt):
        await websocket.send(chunk.content)
        yield chunk.content

# Usage example
# Assuming `websocket` is your WebSocket connection object
async for content in stream_to_websocket(llm, websocket, "write an essay on Sachin in 200 words"):
    # Process each chunk as needed
    pass

In this example, stream_to_websocket is an asynchronous generator function that streams the response from the language model and sends each chunk to a WebSocket connection. The yield statement allows the function to return each chunk incrementally, which can be consumed by an async for loop.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}