apply answerability-audit repair queue: ACCEPT variants, derivability fixes, rubric rewrites (46 masters)
- MISSING_ALTERNATIVES (63/64 applied): uncontracted/contracted forms, digit variants (05|5), synonyms and full-sentence variants, via ACCEPT lines and inline gap alternatives; skipped 'better/fine' for the Opposites drill (not opposites of "sick"). - NOT_DERIVABLE (16 fixed, 10 false alarms, 1 skipped): item-level fixes (visible word hints, missing premise, speaker clarifications "Joe (to Jim):", emphasis/cue corrections, un-gapped unguessable words, rounded-number accepts); converted the two book-10 paragraph clozes (Mary / The Mall) from CHECK: exact to CHECK: llm with logical-completion rubrics; cross-book recycled vocab (pilot, teller, barber, variety store, ...) left alone. - UNCLEAR_TARGET (10): instruction/intro corrections (Food "Type the word for the food.", Explain-with-Too intro), cue/emphasis fixes. - Rubric rewrites (57): applied audit-suggested grader rubrics (now effective with the pre-item RUBRIC parser fix); moved the two per-item rubrics of book-11 "Write the Main Idea" inside their items. No clip-bearing line changed; all masters validate (0 errors, 0 new warnings) and pass stage-master clip-identity checks. Full decision log: reactor-module-tools module-convert/answerability-audit/APPLIED.md Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -568,7 +568,7 @@ INTRO: Read each line from a dialog and make a question about the underlined phr
|
||||
INSTRUCTION: Make a question using the emphasized words.
|
||||
INPUT: speak
|
||||
CHECK: llm
|
||||
RUBRIC: The learner must form a question that targets the emphasized infinitive phrase (e.g. "What does Jim need to buy?").
|
||||
RUBRIC: Require a question targeting the emphasized infinitive phrase (e.g. "What does Jim need to buy?"), followed by a matching answer statement. Accept free phrasing, minor grammatical variations, and pronoun substitutions.
|
||||
|
||||
EXAMPLE
|
||||
TEMPLATE: Jim: To the BX. I *need to buy* some new shoes.
|
||||
@@ -583,7 +583,7 @@ RESPONSE: What did Joe learn to play? He learned to play tennis.
|
||||
TEMPLATE: Jim: I'd *like to learn* to play tennis.
|
||||
RESPONSE: What would Jim like to learn? He'd like to learn to play tennis.
|
||||
|
||||
TEMPLATE: Joe: Don't *forget to pick up* a pair of tennis shoes with your other pair of shoes.
|
||||
TEMPLATE: Joe (to Jim): Don't *forget to pick up* a pair of tennis shoes with your other pair of shoes.
|
||||
RESPONSE: What shouldn't Jim forget to pick up? He shouldn't forget to pick up tennis shoes.
|
||||
|
||||
TEMPLATE: Jim: I won't. I'll *remember to get* tennis balls too.
|
||||
@@ -592,7 +592,7 @@ RESPONSE: What will Jim remember to get? He will remember to get tennis balls.
|
||||
TEMPLATE: Sgt Mays: Sir, my ears *began to hurt* last night.
|
||||
RESPONSE: What began to hurt? His ears began to hurt.
|
||||
|
||||
TEMPLATE: Capt Coffey: Do you *need to go* to the doctor?
|
||||
TEMPLATE: Capt Coffey (to Sgt Mays): Do you *need to go* to the doctor?
|
||||
RESPONSE: Who needs to go to the doctor? Sgt Mays needs to go to the doctor.
|
||||
|
||||
TEMPLATE: Sgt Mays: Yes, sir. I'll *try to see* the doctor this morning and get some medicine.
|
||||
@@ -1717,7 +1717,7 @@ INTRO: For each paragraph, type its topic in a few words — what it's about, no
|
||||
INSTRUCTION: Type the topic in a few words.
|
||||
INPUT: type
|
||||
CHECK: llm
|
||||
RUBRIC: Treat the model response as the key: accept any 1-3 word phrase naming the same topic (e.g. "tickets", "a trip", "exercise", "a birthday dinner"). Reject topics about a different subject.
|
||||
RUBRIC: Treat the model response as the key: accept any short phrase (up to about 5 words) naming the same topic (e.g. "tickets", "a trip", "exercise", "a birthday dinner"). Reject topics about a different subject.
|
||||
|
||||
EXAMPLE
|
||||
TEMPLATE: Paragraph 1 — Paul, Barb, and the airline tickets.
|
||||
|
||||
Reference in New Issue
Block a user