-
-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
inconsistency in titles generated by GPT #fixed #1137
base: develop
Are you sure you want to change the base?
Conversation
...cation/src/main/java/org/togetherjava/tjbot/features/moderation/TransferQuestionCommand.java
Show resolved
Hide resolved
...cation/src/main/java/org/togetherjava/tjbot/features/moderation/TransferQuestionCommand.java
Show resolved
Hide resolved
...cation/src/main/java/org/togetherjava/tjbot/features/moderation/TransferQuestionCommand.java
Outdated
Show resolved
Hide resolved
.formatted(originalMessage); | ||
Optional<String> chatGptResponse = chatGptService.ask(chatGptPrompt, ""); | ||
String title = chatGptResponse.orElse(createTitle(originalMessage)); | ||
title = LEADING_TRAILING_QUOTES.matcher(title).replaceAll(""); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Problem is that you're also removing intentional quotations.
Consider the following example:
"hello" is being printed but not "hello world"
becomes hello" is being printed but not "hello world
'hello world"
this is also now just becoming hello world
You should probably let ChatGPT handle this entirely and update the prompt to ensure that and test it does it the way we intend it to.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In sense, you're really just trying the following on-top of ChatGPT:
if(str.startsWith("\"") && str.endsWith("\"")) {
str = str.substring(1, str.length - 1);
}
if(str.startsWith("'") && str.endsWith("'")) {
str = str.substring(1, str.length - 1);
}
but with uneeded complexity.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We tried to deal at prompt level, unfortunately GPT is very inconsistent and sometimes would end up adding quotations regardless. The best possible way forward is to let GPT do it's thing and then like suraj mentioned look for titles that starting with quotation marks and only remove those that way we handle the edge case.
#1134 fixed
Pull-request
Changes
Closes Issue: NaN
Description
This pull request addresses the issue where titles generated by the GPT model occasionally include quotation marks. To ensure consistency and clean titles, this PR includes changes to sanitize the generated title by removing any leading or trailing quotation marks (both single and double quotes). Additionally, the title is truncated if it exceeds the maximum allowed length (TITLE_MAX_LENGTH)