book 11: regression fixes from old-generation comparison
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -293,8 +293,9 @@ REFERENCE: l4-s3
|
||||
INTRO: Answer questions about the six movies, aloud and in complete sentences. The advertisements stay one tap away if your memory needs a rerun.
|
||||
INSTRUCTION: Answer the questions aloud in complete sentences. Reopen the movie descriptions if needed.
|
||||
INPUT: speak
|
||||
CHECK: reveal
|
||||
CHECK: llm
|
||||
SHOW_PROMPT
|
||||
RUBRIC: Treat the model answer as the factual key; accept any complete sentence that gives the same fact, in any reasonable phrasing.
|
||||
|
||||
PROMPT: Are there different kinds of movies at the theaters?
|
||||
RESPONSE: Yes, there is a variety of movies at the theaters.
|
||||
@@ -1126,6 +1127,10 @@ EXAMPLE
|
||||
PROMPT: Ann: When does the movie start?\nBob: At 7:15, I think.
|
||||
RESPONSE: What did Bob tell Ann?\n— He told her that he thought the show started at 7:15.
|
||||
|
||||
EXAMPLE
|
||||
PROMPT: Mr. Hall: Can I pick up my car now?\nMechanic: No, it's not ready yet.
|
||||
RESPONSE: Did the mechanic tell Mr. Hall that his car was ready?\n— No, he told him it wasn't ready yet.
|
||||
|
||||
PROMPT: Waiter: How is your dinner, sir?\nCurtis: I'm enjoying it very much.
|
||||
RESPONSE: What did Curtis tell the waiter?\n— He told the waiter that he was enjoying his dinner very much.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user