JSON that always parses

Prompting, lesson 4

Schemas, JSON mode, and how to recover from bad output.

You know how a chat request is put together ("How a chat request is built"). It helps to have seen JSON, text like {"qty": 2}.

Part 1. Asking nicely for JSON

A shop reads order emails with a model. The email says: "Hi, please send 2 blue mugs to Pune." The prompt ends with: "Reply with JSON with the keys item, qty and city." Your code hands whatever comes back to json.loads.

Your turn

Five replies come back: the bare JSON; the same JSON after "Sure! Here it is:"; the JSON inside a markdown code fence; the JSON with a comma after the last value; and the JSON with "qty": "two". How many does json.loads accept?

Only the bare JSON and the "two" version parse. The chatty opener, the fence and the trailing comma all break the parser. The "two" one is worse than a crash: it parses, nothing complains, and later your stock code tries to subtract a word.

Show the answer

C. Only the bare JSON and the "two" version parse. The chatty opener, the fence and the trailing comma all break the parser. The "two" one is worse than a crash: it parses, nothing complains, and later your stock code tries to subtract a word.

Five replies through json.loads, in Python

import json

good = '{"item": "blue mug", "qty": 2, "city": "Pune"}'
replies = [
    good,
    "Sure! Here it is:\n" + good,
    "```json\n" + good + "\n```",
    good[:-1] + ",}",
    good.replace("2", '"two"'),
]
for r in replies:
    try:
        json.loads(r)
        print("parses")
    except json.JSONDecodeError:
        print("fails")
# parses
# fails
# fails
# fails
# parses

A prompt can only ask. Most replies will be fine, but code that runs thousands of times a day meets the odd one out every day.

Part 2. A schema says what shape

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come

A schema says what shape
Three ways to get structure
Schemas that help the model

See Pro plans I have Pro: sign in

Next lessonGrammars that constrain decodingMask invalid tokens at each step so output always fits a grammar. Start lesson 5

Prompting overviewSpotted a mistake? Tell us.

04 · JSON that always parses