\n", + " | question | \n", + "example_answer | \n", + "gold_titles | \n", + "observations | \n", + "pred_answer | \n", + "answer_exact_match | \n", + "
---|---|---|---|---|---|---|
0 | \n", + "Are both Cangzhou and Qionghai in the Hebei province of China? | \n", + "no | \n", + "{'Cangzhou', 'Qionghai'} | \n", + "[['Cangzhou | Cangzhou () is a prefecture-level city in eastern Hebei province, People\\'s Republic of China. At the 2010 census, Cangzhou\\'s built-up (\"or metro\") area... | \n", + "No, Cangzhou is in the Hebei province, while Qionghai is in the Hainan province of China. | \n", + "False | \n", + "
1 | \n", + "Who conducts the draft in which Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season? | \n", + "National Hockey League | \n", + "{'2017 NHL Expansion Draft', '2017–18 Pittsburgh Penguins season'} | \n", + "[[\"2017 NHL Expansion Draft | The 2017 NHL Expansion Draft was an expansion draft conducted by the National Hockey League on June 18–20, 2017 to... | \n", + "National Hockey League | \n", + "✔️ [True] | \n", + "
2 | \n", + "The Wings entered a new era, following the retirement of which Canadian retired professional ice hockey player and current general manager of the Tampa Bay... | \n", + "Steve Yzerman | \n", + "{'2006–07 Detroit Red Wings season', 'Steve Yzerman'} | \n", + "[['Steve Yzerman | Stephen Gregory \"Steve\" Yzerman ( ; born May 9, 1965) is a Canadian retired professional ice hockey player and current general manager... | \n", + "Steve Yzerman | \n", + "✔️ [True] | \n", + "
3 | \n", + "What river is near the Crichton Collegiate Church? | \n", + "the River Tyne | \n", + "{'Crichton Collegiate Church', 'Crichton Castle'} | \n", + "[[\"Crichton Collegiate Church | Crichton Collegiate Church is situated about 0.6 mi south west of the hamlet of Crichton in Midlothian, Scotland. Crichton itself is... | \n", + "Tweed River | \n", + "False | \n", + "
4 | \n", + "In the 10th Century A.D. Ealhswith had a son called Æthelweard by which English king? | \n", + "King Alfred the Great | \n", + "{'Ealhswith', 'Æthelweard (son of Alfred)'} | \n", + "[['Æthelweard (son of Alfred) | Æthelweard (d. 920 or 922) was the younger son of King Alfred the Great and Ealhswith.']] | \n", + "King Alfred the Great | \n", + "✔️ [True] | \n", + "
\n", + " | question | \n", + "example_answer | \n", + "gold_titles | \n", + "observations | \n", + "pred_answer | \n", + "answer_exact_match | \n", + "
---|---|---|---|---|---|---|
0 | \n", + "Are both Cangzhou and Qionghai in the Hebei province of China? | \n", + "no | \n", + "{'Cangzhou', 'Qionghai'} | \n", + "[['Cangzhou | Cangzhou () is a prefecture-level city in eastern Hebei province, People\\'s Republic of China. At the 2010 census, Cangzhou\\'s built-up (\"or metro\") area... | \n", + "no | \n", + "✔️ [True] | \n", + "
1 | \n", + "Who conducts the draft in which Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season? | \n", + "National Hockey League | \n", + "{'2017 NHL Expansion Draft', '2017–18 Pittsburgh Penguins season'} | \n", + "[['2017–18 Pittsburgh Penguins season | The 2017–18 Pittsburgh Penguins season will be the 51st season for the National Hockey League ice hockey team that was... | \n", + "National Hockey League | \n", + "✔️ [True] | \n", + "
2 | \n", + "The Wings entered a new era, following the retirement of which Canadian retired professional ice hockey player and current general manager of the Tampa Bay... | \n", + "Steve Yzerman | \n", + "{'2006–07 Detroit Red Wings season', 'Steve Yzerman'} | \n", + "[['Steve Yzerman | Stephen Gregory \"Steve\" Yzerman ( ; born May 9, 1965) is a Canadian retired professional ice hockey player and current general manager... | \n", + "Steve Yzerman | \n", + "✔️ [True] | \n", + "
3 | \n", + "What river is near the Crichton Collegiate Church? | \n", + "the River Tyne | \n", + "{'Crichton Collegiate Church', 'Crichton Castle'} | \n", + "[[\"Crichton Collegiate Church | Crichton Collegiate Church is situated about 0.6 mi south west of the hamlet of Crichton in Midlothian, Scotland. Crichton itself is... | \n", + "Crichton Collegiate Church is located in Midlothian, Scotland, near the hamlet of Crichton, about 7.5 miles south of Edinburgh. | \n", + "False | \n", + "
4 | \n", + "In the 10th Century A.D. Ealhswith had a son called Æthelweard by which English king? | \n", + "King Alfred the Great | \n", + "{'Ealhswith', 'Æthelweard (son of Alfred)'} | \n", + "[['Æthelweard (son of Alfred) | Æthelweard (d. 920 or 922) was the younger son of King Alfred the Great and Ealhswith.'], ['Æthelstan of Kent |... | \n", + "Alfred the Great | \n", + "False | \n", + "
\n", + " | question | \n", + "example_answer | \n", + "gold_titles | \n", + "rationale | \n", + "pred_answer | \n", + "answer_exact_match | \n", + "
---|---|---|---|---|---|---|
0 | \n", + "Are both Cangzhou and Qionghai in the Hebei province of China? | \n", + "no | \n", + "{'Cangzhou', 'Qionghai'} | \n", + "determine if both Cangzhou and Qionghai are in the Hebei province of China. We need to carefully analyze the information provided in the context to... | \n", + "No, only Cangzhou is in the Hebei province of China. Qionghai is located in Hainan province. | \n", + "False | \n", + "
1 | \n", + "Who conducts the draft in which Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season? | \n", + "National Hockey League | \n", + "{'2017 NHL Expansion Draft', '2017–18 Pittsburgh Penguins season'} | \n", + "produce the answer. We know that Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season. Looking at the context provided, we... | \n", + "The 2017 NHL Expansion Draft conducted by the National Hockey League filled the roster of the Vegas Golden Knights, including selecting Marc-Andre Fleury for the... | \n", + "False | \n", + "
2 | \n", + "The Wings entered a new era, following the retirement of which Canadian retired professional ice hockey player and current general manager of the Tampa Bay... | \n", + "Steve Yzerman | \n", + "{'2006–07 Detroit Red Wings season', 'Steve Yzerman'} | \n", + "identify the retired Canadian professional ice hockey player and current general manager of the Tampa Bay Lightning of the National Hockey League (NHL) whose retirement... | \n", + "Steve Yzerman | \n", + "✔️ [True] | \n", + "
3 | \n", + "What river is near the Crichton Collegiate Church? | \n", + "the River Tyne | \n", + "{'Crichton Collegiate Church', 'Crichton Castle'} | \n", + "identify the river near the Crichton Collegiate Church. We know that the church is situated in Midlothian, Scotland, and the River Esk flows through Midlothian... | \n", + "The River Esk | \n", + "False | \n", + "
4 | \n", + "In the 10th Century A.D. Ealhswith had a son called Æthelweard by which English king? | \n", + "King Alfred the Great | \n", + "{'Ealhswith', 'Æthelweard (son of Alfred)'} | \n", + "produce the answer. We know from the context that Ealhswith had a son named Æthelweard in the 10th century A.D. Now, looking at the information... | \n", + "King Alfred the Great | \n", + "✔️ [True] | \n", + "
\n", + " | question | \n", + "example_answer | \n", + "gold_titles | \n", + "rationale | \n", + "pred_answer | \n", + "answer_exact_match | \n", + "
---|---|---|---|---|---|---|
0 | \n", + "Are both Cangzhou and Qionghai in the Hebei province of China? | \n", + "no | \n", + "{'Cangzhou', 'Qionghai'} | \n", + "produce the answer. From the context, we know that Cangzhou is a prefecture-level city in eastern Hebei province, while Qionghai is one of the seven... | \n", + "no | \n", + "✔️ [True] | \n", + "
1 | \n", + "Who conducts the draft in which Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season? | \n", + "National Hockey League | \n", + "{'2017 NHL Expansion Draft', '2017–18 Pittsburgh Penguins season'} | \n", + "produce the answer. From the context, we know that Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season. The draft that... | \n", + "National Hockey League | \n", + "✔️ [True] | \n", + "
2 | \n", + "The Wings entered a new era, following the retirement of which Canadian retired professional ice hockey player and current general manager of the Tampa Bay... | \n", + "Steve Yzerman | \n", + "{'2006–07 Detroit Red Wings season', 'Steve Yzerman'} | \n", + "produce the answer. We know from the context that Steve Yzerman is a Canadian retired professional ice hockey player and the current general manager of... | \n", + "Steve Yzerman | \n", + "✔️ [True] | \n", + "
3 | \n", + "What river is near the Crichton Collegiate Church? | \n", + "the River Tyne | \n", + "{'Crichton Collegiate Church', 'Crichton Castle'} | \n", + "produce the answer. We know that Crichton Collegiate Church is located in Midlothian, Scotland, near the hamlet of Crichton. Since it is close to Edinburgh,... | \n", + "River Esk | \n", + "False | \n", + "
4 | \n", + "In the 10th Century A.D. Ealhswith had a son called Æthelweard by which English king? | \n", + "King Alfred the Great | \n", + "{'Ealhswith', 'Æthelweard (son of Alfred)'} | \n", + "produce the answer. From the context, we know that Ealhswith was the wife of King Alfred the Great. Therefore, in the 10th Century A.D., Ealhswith... | \n", + "King Alfred the Great | \n", + "✔️ [True] | \n", + "
\n", + " | question | \n", + "example_answer | \n", + "gold_titles | \n", + "observations | \n", + "pred_answer | \n", + "answer_exact_match | \n", + "
---|---|---|---|---|---|---|
0 | \n", + "Are both Cangzhou and Qionghai in the Hebei province of China? | \n", + "no | \n", + "{'Qionghai', 'Cangzhou'} | \n", + "[['Hebei | Hebei (; postal: Hopeh) is a province of China in the North China region. Its one-character abbreviation is \"冀 \" (Jì), named after... | \n", + "No, Qionghai is not in the Hebei province of China | \n", + "False | \n", + "
1 | \n", + "Who conducts the draft in which Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season? | \n", + "National Hockey League | \n", + "{'2017–18 Pittsburgh Penguins season', '2017 NHL Expansion Draft'} | \n", + "[['2017 NHL Entry Draft | The 2017 NHL Entry Draft was the 55th NHL Entry Draft. The draft was held from June 23–24, 2017, at... | \n", + "answer=\"Vegas Golden Knights\" | \n", + "False | \n", + "
2 | \n", + "The Wings entered a new era, following the retirement of which Canadian retired professional ice hockey player and current general manager of the Tampa Bay... | \n", + "Steve Yzerman | \n", + "{'2006–07 Detroit Red Wings season', 'Steve Yzerman'} | \n", + "[['Jay Feaster | Jay Harry Feaster (born July 30, 1962 in Harrisburg, Pennsylvania) is a National Hockey League (NHL) executive currently serving as the Executive... | \n", + "Steve Yzerman | \n", + "✔️ [True] | \n", + "
3 | \n", + "What river is near the Crichton Collegiate Church? | \n", + "the River Tyne | \n", + "{'Crichton Collegiate Church', 'Crichton Castle'} | \n", + "[[\"Crichton Collegiate Church | Crichton Collegiate Church is situated about 0.6 mi south west of the hamlet of Crichton in Midlothian, Scotland. Crichton itself is... | \n", + "River Tyne | \n", + "✔️ [True] | \n", + "
4 | \n", + "In the 10th Century A.D. Ealhswith had a son called Æthelweard by which English king? | \n", + "King Alfred the Great | \n", + "{'Æthelweard (son of Alfred)', 'Ealhswith'} | \n", + "[['Æthelweard (son of Alfred) | Æthelweard (d. 920 or 922) was the younger son of King Alfred the Great and Ealhswith.'], ['Ealhswith | Ealhswith or... | \n", + "King Alfred the Great | \n", + "✔️ [True] | \n", + "
\n", + " | question | \n", + "example_answer | \n", + "gold_titles | \n", + "observations | \n", + "pred_answer | \n", + "answer_exact_match | \n", + "
---|---|---|---|---|---|---|
0 | \n", + "Are both Cangzhou and Qionghai in the Hebei province of China? | \n", + "no | \n", + "{'Qionghai', 'Cangzhou'} | \n", + "[] | \n", + "no | \n", + "✔️ [True] | \n", + "
1 | \n", + "Who conducts the draft in which Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season? | \n", + "National Hockey League | \n", + "{'2017–18 Pittsburgh Penguins season', '2017 NHL Expansion Draft'} | \n", + "[[\"2017 NHL Expansion Draft | The 2017 NHL Expansion Draft was an expansion draft conducted by the National Hockey League on June 18–20, 2017 to... | \n", + "George McPhee | \n", + "False | \n", + "
2 | \n", + "The Wings entered a new era, following the retirement of which Canadian retired professional ice hockey player and current general manager of the Tampa Bay... | \n", + "Steve Yzerman | \n", + "{'2006–07 Detroit Red Wings season', 'Steve Yzerman'} | \n", + "[['Gretzky (disambiguation) | Wayne Gretzky is a retired National Hockey League player.'], [\"2006–07 Detroit Red Wings season | The 2006–07 Detroit Red Wings season was... | \n", + "\"Steve Yzerman\" | \n", + "✔️ [True] | \n", + "
3 | \n", + "What river is near the Crichton Collegiate Church? | \n", + "the River Tyne | \n", + "{'Crichton Collegiate Church', 'Crichton Castle'} | \n", + "[['Dumfries | Dumfries ( ; possibly from Scottish Gaelic: \"Dùn Phris\" ) is a market town and former royal burgh within the Dumfries and Galloway... | \n", + "River Nith | \n", + "False | \n", + "
4 | \n", + "In the 10th Century A.D. Ealhswith had a son called Æthelweard by which English king? | \n", + "King Alfred the Great | \n", + "{'Æthelweard (son of Alfred)', 'Ealhswith'} | \n", + "[['Ealhswith | Ealhswith or Ealswitha (died 5 December 902) was the wife of King Alfred the Great. Her father was a Mercian nobleman, Æthelred Mucel,... | \n", + "\"Edward the Elder\" | \n", + "False | \n", + "
\n", + " | question | \n", + "example_answer | \n", + "gold_titles | \n", + "observations | \n", + "pred_answer | \n", + "answer_exact_match | \n", + "
---|---|---|---|---|---|---|
0 | \n", + "Are both Cangzhou and Qionghai in the Hebei province of China? | \n", + "no | \n", + "{'Qionghai', 'Cangzhou'} | \n", + "[['Cangzhou | Cangzhou () is a prefecture-level city in eastern Hebei province, People\\'s Republic of China. At the 2010 census, Cangzhou\\'s built-up (\"or metro\") area... | \n", + "no | \n", + "✔️ [True] | \n", + "
1 | \n", + "Who conducts the draft in which Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season? | \n", + "National Hockey League | \n", + "{'2017–18 Pittsburgh Penguins season', '2017 NHL Expansion Draft'} | \n", + "[[\"2017 NHL Expansion Draft | The 2017 NHL Expansion Draft was an expansion draft conducted by the National Hockey League on June 18–20, 2017 to... | \n", + "George McPhee | \n", + "False | \n", + "
2 | \n", + "The Wings entered a new era, following the retirement of which Canadian retired professional ice hockey player and current general manager of the Tampa Bay... | \n", + "Steve Yzerman | \n", + "{'2006–07 Detroit Red Wings season', 'Steve Yzerman'} | \n", + "[['Tony Resch | Tony Resch is a retired lacrosse player, and current field and box lacrosse head coach. He is the former head coach of... | \n", + "Steve Yzerman | \n", + "✔️ [True] | \n", + "
3 | \n", + "What river is near the Crichton Collegiate Church? | \n", + "the River Tyne | \n", + "{'Crichton Collegiate Church', 'Crichton Castle'} | \n", + "[[\"Crichton Collegiate Church | Crichton Collegiate Church is situated about 0.6 mi south west of the hamlet of Crichton in Midlothian, Scotland. Crichton itself is... | \n", + "River Tyne | \n", + "✔️ [True] | \n", + "
4 | \n", + "In the 10th Century A.D. Ealhswith had a son called Æthelweard by which English king? | \n", + "King Alfred the Great | \n", + "{'Æthelweard (son of Alfred)', 'Ealhswith'} | \n", + "[['Steve Ellsworth | Steven Clark Ellsworth (born July 30, 1960 in Chicago) is the son of Dick Ellsworth and is a former Major League Baseball... | \n", + "answer | \n", + "False | \n", + "
\n", + " | question | \n", + "example_answer | \n", + "gold_titles | \n", + "rationale | \n", + "pred_answer | \n", + "answer_exact_match | \n", + "
---|---|---|---|---|---|---|
0 | \n", + "Are both Cangzhou and Qionghai in the Hebei province of China? | \n", + "no | \n", + "{'Qionghai', 'Cangzhou'} | \n", + "determine whether both Cangzhou and Qionghai are in the Hebei province of China. We know that Cangzhou is a prefecture-level city in eastern Hebei province,... | \n", + "No, only Cangzhou is in Hebei province, while Qionghai is not.assistant | \n", + "False | \n", + "
1 | \n", + "Who conducts the draft in which Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season? | \n", + "National Hockey League | \n", + "{'2017–18 Pittsburgh Penguins season', '2017 NHL Expansion Draft'} | \n", + "answer this question. We know that Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season. We also know that the Vegas... | \n", + "The National Hockey League (NHL).assistant` | \n", + "False | \n", + "
2 | \n", + "The Wings entered a new era, following the retirement of which Canadian retired professional ice hockey player and current general manager of the Tampa Bay... | \n", + "Steve Yzerman | \n", + "{'2006–07 Detroit Red Wings season', 'Steve Yzerman'} | \n", + "answer this question. We know that the Wings entered a new era following the retirement of a Canadian retired professional ice hockey player and current... | \n", + "Steve Yzerman. | \n", + "✔️ [True] | \n", + "
3 | \n", + "What river is near the Crichton Collegiate Church? | \n", + "the River Tyne | \n", + "{'Crichton Collegiate Church', 'Crichton Castle'} | \n", + "find the answer. We know that the Crichton Collegiate Church is situated near the hamlet of Crichton in Midlothian, Scotland. We also know that the... | \n", + "The River Esk. | \n", + "False | \n", + "
4 | \n", + "In the 10th Century A.D. Ealhswith had a son called Æthelweard by which English king? | \n", + "King Alfred the Great | \n", + "{'Æthelweard (son of Alfred)', 'Ealhswith'} | \n", + "find the answer. We know that Ealhswith was the wife of King Alfred the Great. We also know that Æthelweard was the younger son of... | \n", + "King Alfred the Great. | \n", + "✔️ [True] | \n", + "
\n", + " | question | \n", + "example_answer | \n", + "gold_titles | \n", + "rationale | \n", + "pred_answer | \n", + "answer_exact_match | \n", + "
---|---|---|---|---|---|---|
0 | \n", + "Are both Cangzhou and Qionghai in the Hebei province of China? | \n", + "no | \n", + "{'Qionghai', 'Cangzhou'} | \n", + "find the answer. We know that Cangzhou is a prefecture-level city in eastern Hebei province, while Qionghai is one of the seven county-level cities of... | \n", + "No | \n", + "✔️ [True] | \n", + "
1 | \n", + "Who conducts the draft in which Marc-Andre Fleury was drafted to the Vegas Golden Knights for the 2017-18 season? | \n", + "National Hockey League | \n", + "{'2017–18 Pittsburgh Penguins season', '2017 NHL Expansion Draft'} | \n", + "find the answer. We know that Marc-André Fleury was drafted to the Vegas Golden Knights in the 2017 NHL Expansion Draft, and we also know... | \n", + "The National Hockey League | \n", + "✔️ [True] | \n", + "
2 | \n", + "The Wings entered a new era, following the retirement of which Canadian retired professional ice hockey player and current general manager of the Tampa Bay... | \n", + "Steve Yzerman | \n", + "{'2006–07 Detroit Red Wings season', 'Steve Yzerman'} | \n", + "find the answer. We know that the Wings entered a new era, following the retirement of Steve Yzerman, and we also know that Yzerman is... | \n", + "Steve Yzerman | \n", + "✔️ [True] | \n", + "
3 | \n", + "What river is near the Crichton Collegiate Church? | \n", + "the River Tyne | \n", + "{'Crichton Collegiate Church', 'Crichton Castle'} | \n", + "find the answer. We know that the Crichton Collegiate Church is situated in Midlothian, Scotland, and we also know that the River Esk flows through... | \n", + "the River Esk | \n", + "False | \n", + "
4 | \n", + "In the 10th Century A.D. Ealhswith had a son called Æthelweard by which English king? | \n", + "King Alfred the Great | \n", + "{'Æthelweard (son of Alfred)', 'Ealhswith'} | \n", + "find the answer. We know that Ealhswith was the wife of King Alfred the Great, and we also know that Æthelweard was the younger son... | \n", + "King Alfred the Great | \n", + "✔️ [True] | \n", + "