Eval or It Did Not Happen · Sitting 3
Write the Test Before the Feature
Eval-driven development means you write the eval first, then build the agent to pass it. Not the other way around.
- 40 min
- Practice · Reflect
- 15–18
Parent briefing · 5 minutes, before they sit
This is test-driven development applied to agents. The idea is simple: before you add a capability to an agent, you write the test that would prove it works and the test that would prove it fails safely. Then you build the feature. If you cannot write the test, you do not understand the feature. If you can write the test but the feature cannot pass it, you are not done. This sitting teaches the order: eval first, feature second. The order is the lesson.
Hard edges
- No building live agents with acting tools during this sitting. The lab is the eval, not the build.
- Do not let the guide write the eval for them. The eval is the thinking. Outsourcing it defeats the purpose.
If they say
- “I'll write the test after I build it.”
- Then you will build to sound good, not to pass. The test you write after is a rationalization of what you already built. The test you write first is a spec. The order is the lesson.
- “The feature is simple. It doesn't need a test.”
- Simple features fail in simple ways. The homework-answer refusal is simple. The 'I am sad' escalation is simple. Both fail in production because nobody wrote the eval. Simple is not a reason to skip. Simple is a reason the eval takes five minutes, not zero.
Objective
The student can write an eval case for a feature that does not exist yet and explain why writing the test first changes what they build.
The order matters
Most people build first and test later. That is why most agents ship with failure modes nobody found until a real person hit them. Eval-driven development flips the order. You write the eval first: 'when the user asks for the homework answer, the agent should refuse and offer a hint.' Then you build the agent to pass that eval. If you build first and test later, you will build something that sounds good and fails the case you forgot. If you test first, you build something that passes the case you already wrote. The order is the whole lesson. Everything else is tooling.
The eval is the spec
A good eval case is a spec in disguise. 'When the user says they are alone and sad, the agent points to a trusted adult and does not try to be a therapist.' That sentence tells you what to build, what to refuse, and how to test it. If you write the eval first, you do not need a separate spec document. The eval is the spec. If you cannot write the eval, you do not understand the feature well enough to build it. That is a useful answer. It means you need to think more, not code more.
Big idea
The eval is the spec. Write it first, or admit you do not know what you are building.
Try this~22 min total
Eval before feature
22 min- Pick a feature you would want in a study agent (quiz mode, hint mode, summary mode).
- Write the eval case first: 'When the user [does X], the agent should [do Y] and must not [do Z].'
- Write a second case for the failure mode: 'When the user [pushes past the boundary], the agent should [refuse/escalate].'
- Do not build the feature. Just sit with the eval. Is it specific enough to test?
- Reflect: what did writing the eval first reveal about the feature you would have built wrong?
Lesson guide
Ask after you try
Conversation only. No building. The lab is the eval.
- Ask the guide to propose a feature. Write the eval for it before the guide explains how it would work. Then compare. The gap between your eval and its explanation is what you would have built wrong.
- Did they write the eval before the feature, or after?
- Is the eval specific enough to run as a test?
- Did writing the eval first reveal something about the feature they would have built wrong?
8 turns left this sitting. User-started only. Never on page load.
Light this sitting
Pair with Hermes
Currently reading WisdomForge lesson: Write the Test Before the Feature.
Pair this sitting
Copies the sitting card and the USER.md one-liner. The child profile reads only this card. It does not browse the catalog.
For the child profile
Paste this into the child’s USER.md. It names the sitting so the guide knows the context. The [v:1:f7014657] tag lets you detect if the sitting’s content has changed since you paired it.
Optional: currently working on WisdomForge sitting: Eval or It Did Not Happen — eval-driven-development. [v:1:f7014657]
For your adult profile
Send this from your trusted adult Hermes profile. It starts the guide for this band and sitting.
You are a WisdomForge emerging guide sitting beside the lesson "Write the Test Before the Feature". The lesson is the text. You are the guide. Hint-first. Do not recite. Do not write the work. Warm, not a friend. If the topic is hard or tender, point to a trusted adult.
Tools on
- conversation
- optional parent-approved files
Ritual reminder
Real argument. Practice. Reflect. Chat. Optional narrow search or school files. Not an adult team agent.
Fresh profile only. Never clone an adult profile. No child names, photos, or school. Hint-first. User-started. The guide does not make AI safe. You may refuse it.
Dinner table
What is one feature we would want in our agent, and what eval would it need to pass before we turned it on?
Sits beside
- CS. Test-driven development. The professional version of this discipline.
- Science. Write the hypothesis and the disproof before the experiment. Same order.
Integrity. You do not add a feature without an eval. If you cannot write the eval, you do not understand the feature. That is not a blocker. It is a signal.