Skip to content

Commit

Permalink
Added support for multiblock questions / answers in Markdown parser
Browse files Browse the repository at this point in the history
See interpreter/Readme and unit tests for documentation.
  • Loading branch information
andymatuschak committed Oct 14, 2024
1 parent 9039f59 commit ed5a530
Show file tree
Hide file tree
Showing 6 changed files with 506 additions and 239 deletions.
29 changes: 28 additions & 1 deletion packages/interpreter/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,34 @@ For example, here's an excerpt of some notes I wrote as I was studying Service W
>
> A. Two.
The empty line between the question and answer is optional. The question and answer cannot currently span multiple paragraphs: the paragraph including `Q. ` or `A. ` is extracted as that field.
The empty line between the question and answer is optional. So you can also write it like this, with a newline separating the question and answer.

> Q. How many dimensions are in a qubit's vector space? \
> A. Two.
If you'd like to make the question or answer field span multiple Markdown blocks, add a newline after the `Q.` or `A.` prefix, like this:

```
Q.
What is this a picture of?
1. Apples
2. Bananas
3. Pears
A.
Apples
Q. How to do the hokey pokey?
A.
1. You put your right foot in
2. You put your right foot out
3. You put your right foot in
4. And you shake it all about
```

As shown above, you can mix and match the "multi-block" style with the "single-line" style. In "multi-block" mode, all the content after the prefix will be included in the field, until the next question, heading, or horizontal rule (`---`).

#### Creating cloze deletion prompts

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import {
IngestibleSource,
IngestibleSourceIdentifier,
} from "@withorbit/ingester";
import mdast, * as Mdast from "mdast";
import * as Mdast from "mdast";
import { selectAll } from "unist-util-select";
import { Hasher } from "../../hasher/hasher.js";
import { InterpretableFile, Interpreter } from "../../interpreter.js";
Expand Down Expand Up @@ -78,13 +78,13 @@ function convertInterpreterPromptToIngestible(prompt: Prompt): TaskSpec {
type: TaskContentType.QA,
body: {
text: processor
.stringify(prompt.question as unknown as mdast.Root)
.stringify({ type: "root", children: prompt.question })
.trimEnd(),
attachments: [],
},
answer: {
text: processor
.stringify(prompt.answer as unknown as mdast.Root)
.stringify({ type: "root", children: prompt.answer })
.trimEnd(),
attachments: [],
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,12 +68,12 @@ test("cloze in backlink section", () => {
expect(prompts).toHaveLength(0);
});

test("QA prompt in blockquote", () => {
test("QA prompts aren't recognized in blockquote", () => {
const prompts = getPrompts(`# Heading
> Q. Test.
> A. Answer.
`);

expect(prompts).toHaveLength(1);
expect(prompts).toHaveLength(0);
});
14 changes: 7 additions & 7 deletions packages/interpreter/src/interpreters/markdown/markdown.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@ export interface ClozePrompt extends JsonMap {
}

export const qaPromptType = "qaPrompt";
export interface QAPrompt extends JsonMap {
export interface QAPrompt {
type: typeof qaPromptType;
question: mdast.RootContent & JsonMap;
answer: mdast.RootContent & JsonMap;
question: mdast.RootContent[];
answer: mdast.RootContent[];
}

export type Prompt = ClozePrompt | QAPrompt;
Expand All @@ -43,8 +43,8 @@ export interface ClozePromptNode extends unist.Node {
export const qaPromptNodeType = "qaPrompt";
export interface QAPromptNode extends unist.Node {
type: typeof qaPromptNodeType;
question: mdast.RootContent;
answer: mdast.RootContent;
question: mdast.RootContent[];
answer: mdast.RootContent[];
}

type NodeWithParent = mdast.Nodes & {
Expand Down Expand Up @@ -104,8 +104,8 @@ export function findAllPrompts(tree: mdast.Root): Prompt[] {
const qaPromptNode = n as QAPromptNode;
const qaPrompt: QAPrompt = {
type: "qaPrompt",
question: qaPromptNode.question as mdast.RootContent & JsonMap,
answer: qaPromptNode.answer as mdast.RootContent & JsonMap,
question: qaPromptNode.question,
answer: qaPromptNode.answer,
};
return qaPrompt;
});
Expand Down
Loading

0 comments on commit ed5a530

Please sign in to comment.