Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix IndexError in LLM Reranking when handling malformed LLM responses #17353

Merged
merged 3 commits into from
Dec 26, 2024
+14 −2
Merged

Fix IndexError in LLM Reranking when handling malformed LLM responses #17353

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bf5af4f
Fix bug #17352 by catching errors for malformed LLM responses
okirmis Dec 22, 2024
e69030e
Fix formatting
okirmis Dec 23, 2024
a6e1377
Merge branch 'main' into bug/17352_gracefully_handle_malformed_responses
okirmis Dec 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions llama-index-core/llama_index/core/indices/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,20 @@ def default_parse_choice_select_answer_fn(
continue
answer_nums.append(answer_num)
# extract just the first digits after the colon.
_answer_relevance = re.findall(r"\d+", line_tokens[1].split(":")[1].strip())[0]
answer_relevances.append(float(_answer_relevance))
try:
_answer_relevance = re.findall(
r"\d+", line_tokens[1].split(":")[1].strip()
)[0]
answer_relevances.append(float(_answer_relevance))
except (IndexError, ValueError) as e:
if not raise_error:
continue
else:
raise ValueError(
f"Invalid answer line: {answer_line}. "
"Answer line must be of the form: "
"answer_num: <int>, answer_relevance: <float>"
)
return answer_nums, answer_relevances


Expand Down
Loading