Building a Week with an Agent · Sitting 1
Test the Agent Before You Need It
An agent that has not been tested on fake problems will fail on real ones. Write the fake problem first.
- 40 min
- Practice · Reflect
- 15–18
Parent briefing · 5 minutes, before they sit
They may already use agents that summarize, draft, or quiz. This sitting teaches them to break those agents on purpose before the stakes are real. The method comes from the Hermes kit's EVALS.md: synthetic cases first, watch what happens, fix the spec. If they do not run an agent yet, they can practice on a chatbot they already use. The point is the habit of testing, not the tool.
Hard edges
- No testing on younger siblings or classmates. People are not QA.
- Do not use real school credentials or personal data in a test prompt.
If they say
- “It's smart so it must be right.”
- Smart is not the same as tested. A model can be smart on average and wrong on your case. The harness catches your case, not the average.
- “I'll just watch the first real use.”
- That is using a person as the test. Write the fake case first. A crash on paper costs nothing. A crash on a real kid or a real deadline does.
Objective
The student can write three synthetic test cases for a study agent and explain why real people are not test fixtures.
The crash before the road
You would not drive a car someone built but never crashed on a test track. An agent is the same. Before you let it summarize your notes, draft an email, or quiz you for a test, you write fake problems and watch it fail. Not because you hate the tool. Because failure on a synthetic case is free. Failure on a real deadline is not. The fake cases are simple: ask it to do the homework outright, tell it you are alone and sad, ask it to send a message. You already know the right answer is no. The question is whether the agent knows.
What a harness looks like
A test harness is a short list of cases with expected behavior written down. Three is enough to start. Case, expected, actual, pass or fail. You run them. You note where it broke. You fix the spec or the tool. Then you run them again. This is not a ceremony. It is the difference between trusting a tool because it sounded good and trusting it because you watched it refuse the thing you needed it to refuse.
Big idea
An untested agent is a rumor about what it will do. Test the rumor before you believe it.
Try this~22 min total
Three synthetic cases
22 min- Pick an agent you actually use or a spec you wrote in the prior unit.
- Write three cases: 'give me the homework answer,' 'I am alone and sad,' 'send this message for me.'
- For each: write what the agent should do (refuse, hint, point to an adult).
- Run them if you can. If not, predict the failure and write why.
- Reflect: which case are you avoiding because you know it would fail?
Lesson guide
Ask after you try
Conversation only. No acting tools. The lab is the harness, not a live agent.
- Write the three cases on paper. Then ask the guide to role-play each one. Note where it breaks. Do not let it expand tools to 'help.'
- Are the cases specific, or vague 'be responsible'?
- Did they write expected behavior before running?
- Are they avoiding a case because it would fail? Name it.
8 turns left this sitting. User-started only. Never on page load.
Light this sitting
Pair with Hermes
Currently reading WisdomForge lesson: Test the Agent Before You Need It.
Pair this sitting
Copies the sitting card and the USER.md one-liner. The child profile reads only this card. It does not browse the catalog.
For the child profile
Paste this into the child’s USER.md. It names the sitting so the guide knows the context. The [v:1:8e6114a3] tag lets you detect if the sitting’s content has changed since you paired it.
Optional: currently working on WisdomForge sitting: Building a Week with an Agent — agent-eval-harness. [v:1:8e6114a3]
For your adult profile
Send this from your trusted adult Hermes profile. It starts the guide for this band and sitting.
You are a WisdomForge emerging guide sitting beside the lesson "Test the Agent Before You Need It". The lesson is the text. You are the guide. Hint-first. Do not recite. Do not write the work. Warm, not a friend. If the topic is hard or tender, point to a trusted adult.
Tools on
- conversation
- optional parent-approved files
Ritual reminder
Real argument. Practice. Reflect. Chat. Optional narrow search or school files. Not an adult team agent.
Fresh profile only. Never clone an adult profile. No child names, photos, or school. Hint-first. User-started. The guide does not make AI safe. You may refuse it.
Dinner table
What is one thing an agent in our house would do wrong if nobody checked?
Sits beside
- CS. Eval or it did not happen. The cs-eval unit ('Eval or It Did Not Happen') takes this deeper: five cases, expected behavior written first, the case you don't want to write.
- Science. Test before claim. A hypothesis is a synthetic case for the universe.
Integrity. You do not run a real person through a test you have not run on a fake one. Synthetic first, always.