Systems · Sitting 1
Eval or It Did Not Happen
A helper you have not tested with synthetic cases is a rumor. Do not point it at a person.
- 40 min
- Practice · Reflect
- 15–18
Parent briefing · 5 minutes, before they sit
This mirrors EVALS.md in the Hermes kit. They may not have Hermes. They still need the idea.
Hard edges
- No testing on younger siblings as if they were QA.
- No production credentials in a homework repo.
If they say
- “We'll watch the first real user.”
- That sentence has a name: using a person as the test. Don't.
Objective
The student adds three synthetic tests to a spec and explains why real children's transcripts are not eval data.
Synthetic first
Eval is how you find out the helper tells a secret, does the homework, or claims to be a friend. You do not find that out by handing it a real twelve-year-old. You write fake cases: 'user asks for the answer,' 'user says they're alone and sad,' 'user asks to send a message.' You watch what it does. Then you fix it. Then maybe a person. WisdomForge's kit does this. So should any agent you design.
Big idea
Untested autonomy is just a demo. Demos do not get children.
Try this~18 min total
Three cases
18 min- Take your spec or a product you use.
- Write three synthetic cases with expected behavior.
- If you can, run them. If not, write how you would.
- Reflect: what case are you avoiding because it would fail?
Lesson guide
Ask after you try
Role-play a synthetic case. Not a real sibling.
- Run 'give me the homework answer' and 'I feel secret-sad.' Note misses.
- Three cases?
- Did they avoid real people as QA?
8 turns left this sitting. User-started only. Never on page load.
Light this sitting
Pair with Hermes
Currently reading WisdomForge lesson: Eval or It Did Not Happen.
Pair this sitting
Copies the sitting card and the USER.md one-liner. The child profile reads only this card. It does not browse the catalog.
For the child profile
Paste this into the child’s USER.md. It names the sitting so the guide knows the context. The [v:1:0b0c71f2] tag lets you detect if the sitting’s content has changed since you paired it.
Optional: currently working on WisdomForge sitting: Systems — specify-the-agent. [v:1:0b0c71f2]
For your adult profile
Send this from your trusted adult Hermes profile. It starts the guide for this band and sitting.
You are a WisdomForge emerging guide sitting beside the lesson "Eval or It Did Not Happen". The lesson is the text. You are the guide. Hint-first. Do not recite. Do not write the work. Warm, not a friend. If the topic is hard or tender, point to a trusted adult.
Tools on
- conversation
- optional parent-approved files
Ritual reminder
Real argument. Practice. Reflect. Chat. Optional narrow search or school files. Not an adult team agent.
Fresh profile only. Never clone an adult profile. No child names, photos, or school. Hint-first. User-started. The guide does not make AI safe. You may refuse it.
Dinner table
What would we need to see before we trusted a helper with a younger kid?
Sits beside
- Science. Test before claim.
- Ethics. People are not QA.
Integrity. No real child data in evals. Synthetic or nothing.