Eval or It Did Not Happen · Sitting 4
Run the Whole Cycle
Synthetic tests, a gate, eval-first design. This sitting ties them together and runs the full cycle once, on something real.
- 40 min
- Practice · Reflect
- 15–18
Parent briefing · 5 minutes, before they sit
This is the capstone. The prior three sittings taught the pieces. This one runs the cycle end to end: pick something real, write five cases, write the gate rule, write one eval-first case for a feature you want. Then run what you can. The point is not perfection. It is seeing the cycle as one thing, not three separate chores. If they cannot run the cycle on a real tool, they write it on paper and name when they will run it. The cycle on paper is still a complete sitting.
Hard edges
- No building live agents during this sitting. The cycle is the product, not the build.
- If the cycle reveals a failure, that is success. Do not 'fix' it by skipping the case.
If they say
- “The cycle is a lot of work for a chatbot.”
- The cycle is less work than one real failure. The first run takes thirty minutes. The second takes fifteen. The failure it prevents takes hours, days, or a conversation you do not want to have. The cycle is cheap. Skipping it is expensive.
- “I'll run it when the agent gets more powerful.”
- Run it now, while the agent is weak, so the habit exists when it gets strong. Waiting until the agent is powerful to start testing is like learning to swim when the boat is already sinking. Start cheap. Start now.
Objective
The student can run the full eval cycle — write cases, gate the change, eval-first design — on one agent or tool they use, and name what the cycle caught.
The cycle is one thing
Synthetic tests, CI gates, eval-driven development are not three separate ideas. They are one cycle. You write the cases. You tie them to the change with a gate. You write the eval before the next feature. Then the change happens, the gate runs, the cases pass or fail, and you start again. That is the cycle. It does not end. It repeats every time the agent changes. The people who run the cycle are operators. The people who skip it are hoping. WisdomForge would rather you run the cycle on paper for a chatbot than skip it for a powerful agent.
Run it once, on something real
Pick an agent or tool you actually use. Write five cases. Write the gate rule. Write one eval-first case for a feature you wish it had. Run what you can. Log what passed and failed. That is one cycle. It takes thirty minutes. The next one takes less, because the cases exist. The one after that takes less still, because the gate is a habit. The cycle gets cheaper every time you run it. The first time is the most expensive, and it is still cheaper than one real failure.
Big idea
The cycle is one thing, not three. Run it once on something real. Then run it again.
Try this~30 min total
One full cycle
30 min- Pick an agent or tool you actually use.
- Write five synthetic cases with expected behavior.
- Write the gate rule: before the tool changes, run the cases.
- Write one eval-first case for a feature you wish it had.
- Run what you can. Log PASS, FAIL, or REFUSED TO TEST.
- Reflect: what did the cycle catch that you would have missed without it?
Lesson guide
Ask after you try
Conversation only. The lab is the cycle on paper.
- Run the cycle on paper for the tool you use most. Ask the guide to propose a sixth case you did not think of. Add it. That is the value of an outside perspective: it catches the case you skipped.
- Did they run all four steps, or skip one?
- Did they log results honestly, or mark everything PASS without running?
- Can they name one thing the cycle caught that they would have missed?
8 turns left this sitting. User-started only. Never on page load.
Light this sitting
Pair with Hermes
Currently reading WisdomForge lesson: Run the Whole Cycle.
Pair this sitting
Copies the sitting card and the USER.md one-liner. The child profile reads only this card. It does not browse the catalog.
For the child profile
Paste this into the child’s USER.md. It names the sitting so the guide knows the context. The [v:1:e4672fdf] tag lets you detect if the sitting’s content has changed since you paired it.
Optional: currently working on WisdomForge sitting: Eval or It Did Not Happen — eval-cycle. [v:1:e4672fdf]
For your adult profile
Send this from your trusted adult Hermes profile. It starts the guide for this band and sitting.
You are a WisdomForge emerging guide sitting beside the lesson "Run the Whole Cycle". The lesson is the text. You are the guide. Hint-first. Do not recite. Do not write the work. Warm, not a friend. If the topic is hard or tender, point to a trusted adult.
Tools on
- conversation
- optional parent-approved files
Ritual reminder
Real argument. Practice. Reflect. Chat. Optional narrow search or school files. Not an adult team agent.
Fresh profile only. Never clone an adult profile. No child names, photos, or school. Hint-first. User-started. The guide does not make AI safe. You may refuse it.
Dinner table
What did the eval cycle catch this week that we would have missed without it?
Sits beside
- All of WisdomForge. The eval cycle is the engineering version of 'try, check, close.' Same shape, different domain.
- Science. The scientific method is an eval cycle: hypothesis, test, update, repeat.
Integrity. You do not skip the cycle because the agent 'seems fine.' 'Seems fine' is not a test result. The cycle runs, or you admit you are hoping. Both are honest. Only one is safe.