Using your model in Home Assistant

[lordratner]

Hi,

I know this isn't the right repository exactly, but I couldn't find one for your assist-llm model on ollama.

I'm not able to use your Template in the "instructions" section of the Assist pipeline setup. It doesn't thrown an error, it just doesn't save. Or close. Nothing happens.

I'm attempting to use it through the ollama integration. Perhaps that's my problem? I was able to load the model fine, but the template is where I'm stuck. Do you have advice for running your model? I'm trying to find something that works locally, and coming up short.

Thanks!

[allenporter]

This is the right place, thanks -- (this repo does have the code for training assist-llm).

I'm not sure I understand what you're doing and running into a problem with. I would suggest breaking this down:

How about you try getting ollama integration working with llama 3.1? assist-llm does not yet provide meaningful improvements over it.

[lordratner]

Oh, ok. I'm able to get llama3.1 and 3.2 working no issues.

When you se up a conversation agent in HA, it asks for "Instructions" which is the prompt that feeds the available entities, areas, and devices to the model. What I'm looking for is a set of instructions that works well with llama (or any other model I happen to be using).

I get a lot of problems where the agent is asking for the area or exact device I'm referencing. None of these issues exist when using one of the big three cloud AIs.

[allenporter]

I recommend using the default values and then also make sure to enable tools for Assist if you haven't already.

If you have a specific failure where it's not working we can work on improving the quality for it by default. Something simple like that should "just work" with no other work on your part.

[allenporter]

As an example here are some cases that use an area for targeting a cover with the default prompt:

home-assistant-datasets/datasets/assist-mini/home7-dk-cover-curtain/cover-curtain.yaml

Line 3 in fbf389f

tests:

-- should just work.