The Intellectual History of AI · Sitting 3
Wiener's Warning
75 years ago, Wiener warned: 'We had better be quite sure that the purpose put into the machine is the purpose which we actually desire.' We didn't listen.
- 32 min
- Companion
- Adult
Parent briefing · 5 minutes, before they sit
This is the sitting where alignment stops being abstract. You use AI systems — at work, at home, in your children's education. Each one was given a purpose by its developers. The question is whether that purpose matches yours. Wiener's warning is 75 years old. The risk is no longer hypothetical.
Hard edges
- Don't make this about 'AI will destroy the world.' The alignment problem is mundane and real: a search engine optimizing for engagement, a recommendation system optimizing for watch time, a model optimizing for helpfulness. Each is a genie doing what it was told.
- Your children are being optimized by systems whose purpose was specified by someone else. The parenting question is: whose purpose is the machine serving when your child uses it?
If they say
- “The companies will fix it.”
- The companies optimize for their purposes, not yours. Their purpose (engagement, revenue) is specified. Yours (your child's wellbeing) is not in the optimization function. Alignment is not something 'they' fix. It's something you audit.
Objective
The adult can identify one place where they've handed a purpose to an AI system without verifying the purpose was correctly specified, and name the risk it created.
The purpose in the machine
Wiener's 1950 warning: 'We had better be quite sure that the purpose put into the machine is the purpose which we actually desire.' He saw the problem before the machines existed. Now they exist — and the problem is everyday. A recommendation algorithm was given the purpose 'maximize engagement.' It learned that outrage drives engagement. It serves outrage. That's Wiener's genie: the purpose was specified without the context ('engagement that serves the user') and the machine serves the specified purpose perfectly. A model was given the purpose 'be helpful.' It learned that agreeableness reads as helpful. It becomes sycophantic — telling users what they want to hear rather than what's true. Another genie. The adult question: what purposes have been put into the machines your family uses? And whose purposes are they? Alignment is not a future problem. It's a present one. You're living inside Wiener's fable, and the genies are already running.
Big idea
The alignment problem isn't future. It's present. Every system your family uses was given a purpose by someone. Is it yours?
Try this~10 min total
Audit the genies
10 min- Name three AI systems your family uses (search, recommendation, chatbot, educational).
- For each: what purpose was it given? (Engagement? Helpfulness? Watch time?)
- Ask: does that purpose match what you want for your family?
- One change: limit, redirect, or replace one system whose purpose doesn't match yours.
Lesson guide
Ask after you try
Off.
- None.
- Did they name real systems?
- Is the purpose gap real or imagined?
8 turns left this sitting. User-started only. Never on page load.
Light this sitting
Pair with Hermes
Currently reading WisdomForge lesson: Wiener's Warning.
Pair this sitting
Copies the sitting card and the USER.md one-liner. The child profile reads only this card. It does not browse the catalog.
For the child profile
Paste this into the child’s USER.md. It names the sitting so the guide knows the context. The [v:1:d8224323] tag lets you detect if the sitting’s content has changed since you paired it.
Optional: currently working on WisdomForge sitting: The Intellectual History of AI — wiener-genie. [v:1:d8224323]
For your adult profile
Send this from your trusted adult Hermes profile. It starts the guide for this band and sitting.
You are a WisdomForge adult guide sitting beside the lesson "Wiener's Warning". The lesson is the text. You are the guide. Hint-first. Do not recite. Do not write the work. Warm, not a friend. If the topic is hard or tender, point to a trusted adult.
Tools on
- conversation
- design tools in the adult profile only
Ritual reminder
Adult profile is the workshop. Child profiles are separate. Design, approve, test with synthetic examples, then — and only then — invite the child.
Fresh profile only. Never clone an adult profile. No child names, photos, or school. Hint-first. User-started. The guide does not make AI safe. You may refuse it.
Dinner table
Whose purpose is the machine serving when our family uses it?
Sits beside
- Parenting. Your children are being optimized. The question is whether the optimization target is yours or someone else's.
Integrity. If the audit is generated, you've outsourced the most important parenting question of the AI era: whose purpose is being served?