LLM APIs and Prompting · Calling a Model · lesson 2 of 8
The messages list
about 16 minutes · free · runs in your browser
Step 1 of 2
A conversation is a list you keep
The API is stateless. It does not remember your last question, and there is no session to resume: the entire conversation is re-sent on every call, and "memory" is a list your program keeps.
Three roles matter:
| Role | What it is |
|---|---|
system | standing instructions — who the model is and what shape the answer takes |
user | what the person said |
assistant | what the model said last time |
Order is the conversation, so it is load-bearing. The list runs oldest to newest, and the last message is the one being answered. Append the assistant's reply before the next question or the model has no idea what it just told you.
messages = [{"role": "system", "content": "Answer in one sentence."}]
messages.append({"role": "user", "content": "What is rag?"})
reply = fake_llm.chat(messages)["choices"][0]["message"]["content"]
messages.append({"role": "assistant", "content": reply})
Your turn: write turn(messages, question) that appends the question, calls the
model, appends the reply, and returns the reply text. The list it is handed must come back
with both new messages on it.
You start from this, and edit it in the browser:
import fake_llm
def turn(messages, question):
"""Append the question, ask the model, append the reply, return the reply text."""
return ""
Step 2 of 2
What the model actually saw
When an answer is wrong, the first question is not "why is the model bad" but "what did I send it". Almost every prompting bug is visible in the messages list: the system prompt appended last, the history in the wrong order, an empty string where a variable should have been.
fake_llm.calls() returns the log of what was sent, which is the same thing you would
get from a request logger in production.
fake_llm.reset()
fake_llm.chat([{"role": "user", "content": "hello"}])
fake_llm.calls()[-1]["messages"] # exactly what went over the wire
Your turn: write first_role(messages) that sends the messages and reports the role
of the first message the API actually received. Read it from the call log, not from the
list you passed — those two disagreeing is the bug being hunted.
You start from this, and edit it in the browser:
import fake_llm
def first_role(messages):
"""Send these messages, then report the first role the API received."""
return ""