Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RELTask generates prompts that use ints instead of entity labels #366

Closed
peter-axion opened this issue Nov 13, 2023 · 1 comment · Fixed by #367
Closed

RELTask generates prompts that use ints instead of entity labels #366

peter-axion opened this issue Nov 13, 2023 · 1 comment · Fixed by #367
Labels
bug Something isn't working feat/task Feature: tasks

Comments

@peter-axion
Copy link

peter-axion commented Nov 13, 2023

The relation task tells the LLM:

The text below contains pre-extracted entities, denoted in the following format within the text:
<entity text>[ENT<entity id>:<entity label>]

However, it generates prompt text like:

well[ENT0:14862748245026736845] hello[ENT1:14230521632333904559] there[ENT2:149303876845869574]!

Reproduce via:

import spacy_llm
import spacy
nlp = spacy.load('en_core_web_sm')
doc = nlp("well hello there!")
doc.set_ents([
    spacy.tokens.Span(doc,0,1,"A"),
    spacy.tokens.Span(doc,1,2,"B"),
    spacy.tokens.Span(doc,2,3,"C")]
)
tsk = spacy_llm.tasks.make_rel_task(labels = ["A","B","C"])
for prompt in tsk.generate_prompts([doc]):
    print(prompt)

The problem is a single character difference here:

annotation = f"[ENT{i}:{ent.label}]"

It should be label_ instead of label. I'd open the PR, but it feels weird to have label be the non-string version of the entity so maybe the real problem is upstream.

print(doc.ents[0].label)
print(doc.ents[0].label_)

14862748245026736845
A

@rmitsch rmitsch added bug Something isn't working feat/task Feature: tasks labels Nov 13, 2023
@rmitsch
Copy link
Collaborator

rmitsch commented Nov 13, 2023

Thanks for bringing this up and identifying the problem! This is a known issue, I thought we fixed this already 🤔 #367 fixes this and will be part of v0.6.3 (released probably tomorrow).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working feat/task Feature: tasks
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants