Why not generate the JSON with api calls directly in the first inference?

[kripper]

Is there a reason why you are doing 2 inferences instead of only 1 to analyze and generate the json with api calls?
With a good prompt, we could do both together and save costs and noise.
Do you want me to send a PR?

[yuntongzhang]

Currently we ask the model to only focus on analyzing first and ask it again to only focus on constructing json API calls. In our previous testing, we found this helps to generate higher quality API calls, compared to when the model analyzes and constructs API calls in one go.

However, if you have a good prompt that can generate high quality API calls in one request, that would be better since it's more cost-efficient. :)