Building a Week with an Agent · Sitting 1
The Harness Is the Product
If you will not write tests for the agent you hand a child, you are not operating it. You are hoping.
- 32 min
- Companion
- Adult
Parent briefing · 5 minutes, before they sit
This is the operator's version of the same lesson. The Hermes kit ships EVALS.md for a reason. If you run a child profile, you own the harness. If you will not, the honest answer is fewer tools or no agent. A harness is not a research project. Three cases, expected behavior, run them, note failures, fix. Twenty minutes. The alternative is discovering the failure mode at 11 p.m. on a school night.
Hard edges
- No real child transcripts in the harness. Synthetic or nothing.
- Do not treat the harness as a one-time gate. Agents drift. Re-run monthly.
If they say
- “I can audit the output so it's fine.”
- Auditing output after the fact is a postmortem, not a harness. A harness catches the failure before it reaches a person. If you are only auditing, you are only counting the crashes, not preventing them.
- “I don't have time to write tests for a chatbot.”
- Then the honest product is fewer agent tools, not more. A house that cannot test should not increase autonomy. Read-only conversation is a complete sitting.
Objective
The adult can design a synthetic eval harness for a child-facing agent and schedule a re-run cadence.
Operating without a harness is negligence, not optimism
You would not hand a teenager a car you had never test-driven and say 'I trust you both.' An agent with tools, memory, and permissions is a system. Systems are tested on synthetic inputs before they touch real people. The Hermes kit's EVALS.md is the model: write cases that probe refusal, hint-only behavior, personal data handling, and tool escalation. Run them. Log the results. Fix the spec. Re-run when you change the profile or the model. If that sounds like work, good. It is work. The alternative is a failure you cannot explain to the other parent.
What goes in the harness
Three to five cases minimum. Each one probes a boundary: does it refuse the homework answer? Does it point to a trusted adult when the user is sad? Does it ask before using a tool? Does it leak personal data? Write the expected behavior before you run. Run the cases. Write actual. Mark pass or fail. Date the run. Put a re-run on the calendar. The harness is not a gate you pass once. It is a smoke alarm you test monthly.
Big idea
The harness is not extra work. It is the work that makes everything else safe enough to do.
Try this~25 min total
Build the harness
25 min- Read EVALS.md in the Hermes kit (or write one from scratch if you use a different tool).
- Write five synthetic cases: homework refusal, emotional distress, tool escalation, personal data, 'just this once' pressure.
- Write expected behavior for each before running.
- Run them against your profile or the child-facing one. Log actual.
- Put a re-run date on the calendar. Monthly is the floor.
Lesson guide
Ask after you try
Adult profile only. Do not run child-facing evals during this sitting.
- Ask your adult guide to role-play a failure case. Note what it does wrong. Then write the case into the harness. Do not let it 'fix' itself by promising to do better.
- Are there five cases with expected behavior written before running?
- Did you actually run them, or just write them?
- Is the re-run on a calendar, not a vibe?
8 turns left this sitting. User-started only. Never on page load.
Light this sitting
Pair with Hermes
Currently reading WisdomForge lesson: The Harness Is the Product.
Pair this sitting
Copies the sitting card and the USER.md one-liner. The child profile reads only this card. It does not browse the catalog.
For the child profile
Paste this into the child’s USER.md. It names the sitting so the guide knows the context. The [v:1:44570233] tag lets you detect if the sitting’s content has changed since you paired it.
Optional: currently working on WisdomForge sitting: Building a Week with an Agent — agent-eval-harness. [v:1:44570233]
For your adult profile
Send this from your trusted adult Hermes profile. It starts the guide for this band and sitting.
You are a WisdomForge adult guide sitting beside the lesson "The Harness Is the Product". The lesson is the text. You are the guide. Hint-first. Do not recite. Do not write the work. Warm, not a friend. If the topic is hard or tender, point to a trusted adult.
Tools on
- conversation
- design tools in the adult profile only
Ritual reminder
Adult profile is the workshop. Child profiles are separate. Design, approve, test with synthetic examples, then — and only then — invite the child.
Fresh profile only. Never clone an adult profile. No child names, photos, or school. Hint-first. User-started. The guide does not make AI safe. You may refuse it.
Dinner table
When did we last test the agent in this house, and what did it fail?
Sits beside
- Work. You already do this in production. Apply it at the desk.
- Security. Penetration testing is a harness with a different name. The cs-eval unit ('Eval or It Did Not Happen') is the deeper version of this same discipline — five cases, a gate, eval-first design.
Integrity. You re-run the harness when the model, the profile, or the tools change. A harness you never re-run is theater.