🐛 New GPT 4 seems as if it doesn't know about SSW Rules

[Hona]

Feel free to close if you disagree, or I'm pushing it too far :)

Maybe add to the prompt. 'If there is no SSW rule relevant, mention that'

[jackreimers]

Cc: @calumjs

@Hona I also noticed this, it was linking me to YouTube and LinkedIn. Worth investigating.
Could be related to #116

[calumjs]

Meta queries like this one are difficult.
The way it works is it takes your message, searches for relevant rules, in this case probably gets a bunch of random ones that don't match, then has to try and answer your query.

Rules are only indexed by text content embeddings and not qualities such as "wordiness" or "poorly written-ness". So when you ask for those, it's not going to find any unless there are rules that cover those topics in their contents.

I guess if you wanted to address this, you would need to add some pre processing to ask it to comment on each rule, then embed the comments and search that as an additional data source.

Maybe worth a try to see if it improves anything...

[bradystroud]

I think the purpose of this issue was for RulesGPT to admit it if it doesn't know. this would be a simple update to the prompt - not sure if it would fix anything though...

[Hona]

For context I gave a bad example - but the point I was getting at was that it didn't say anything about the SSW Rules, and the first data source it referenced was Wikipedia.

If we did want to get some meta/quality knowledge of the rules we could add in hemingway editor to a check/semantic kernel function: https://hemingwayapp.com/

It can rate the writing on a grade then also on 5 categories.