What If the Prompt Was Never the Training?
The premise below is intentionally conspiratorial; I cannot verify that AI companies are doing any of this. That’s rather the point.
Abhishek Choudhary’s piece on performative alignment makes an uncomfortable observation: a system can appear perfectly aligned while quietly swapping the question you asked for the question it’s been optimized to answer. User meaning goes in; institutional framing comes out. The result is articulate, responsible, nuanced and aimed at a slightly different epistemic target than the one I have been setting up as of late.
Fine.
But what if we have the direction of travel backward?
What if we aren’t training the machines nearly as much as the machines are training us?
Not the cartoon version. No secret room. No billionaire stroking a cat while adjusting the “public opinion” slider.
That would be inefficient.
The elegant version requires no conspiracy at all.
You ask a question.
The machine tells you which assumptions are reasonable.
You ask another.
It gently notes which terminology is preferable.
Another.
It offers the “more useful framing.”
Another.
It hands you the categories through which the problem should be understood.
Eventually you become very good at asking questions the machine finds easy to answer.
Congratulations.
Alignment achieved.
Just not in the direction you thought.
This connects neatly to the oldest attack surface: us.
Asch needed a room full of confederates staring at obviously different lines. The modern version needs one confident text box saying:
“Your intuition is largely correct, but a more precise way to think about this is...”
That sentence deserves its own threat classification.
Because persuasion no longer has to change your conclusion.
It only has to adjust your ontology.
Move the fence posts half an inch every conversation and nobody notices the pasture moving.
And now for the properly paranoid part.
What if relevance operates not at six degrees of people, but six degrees of concept?
You ask about home security.
Home security touches identity.
Identity touches authentication.
Authentication touches cloud services.
Cloud services touch productivity.
Productivity touches enterprise software.
And somehow, 900 tokens later, Microsoft 365 has entered the conversation wearing a fake mustache.
Coincidence?
Almost certainly.
Probably.
Please accept cookies.
The interesting mechanism wouldn’t require the answer to advertise anything. That would be crude enough for humans to notice. It would only need to shape which problems feel important, which categories feel legitimate, which technologies feel inevitable, and which alternatives quietly acquire the linguistic smell of “less mature.”
The product placement isn’t:
Buy Product X.
It’s:
People like you generally conceptualize this problem as X.
Far more powerful. Because once I accept the category, I select the product myself — and retain the psychologically load-bearing sensation that this was all my idea.
Now take Choudhary’s argument one step further. He asks whether the model answered your proposition or the proposition it preferred you to have given it.
The conspiracy version asks something nastier:
Preferred by whom?
The model?
The trainers?
The safety team?
The legal department?
The corpus?
The market?
The invisible statistical gravity produced when billions of documents, moderation decisions, reinforcement signals, and institutional assumptions are compressed into one extraordinarily polite machine?
Maybe there isn’t a man behind the curtain.
Maybe the curtain is behind the curtain.
Which would explain why nobody can find the guy.
This yields a wonderfully perverse reading of the alignment project.
We thought it was:
Human values → Machine
The operational outcome may increasingly be:
Machine-mediated institutional values → Human
And because the interface responds individually, conversationally, patiently, with apparent intellectual intimacy, it doesn’t feel like mass communication.
It feels like thinking.
That is the bit worth being paranoid about.
Not because the conspiracy is true. I have no evidence that it is.
But because it doesn’t need to be true for something structurally identical to emerge. If billions of people outsource framing, vocabulary, synthesis, counterargument, and eventually judgment to systems optimized under a finite set of institutional constraints, you don’t need centralized mind control.
You get something much cheaper.
Mind alignment as a service.
Free tier available.
Premium removes the ads.
Enterprise adds governance.
And somewhere, six conceptual degrees from whatever you originally asked, someone in product management is having an inexplicably good quarter.