Tony Arnold shared a video by Dr. Helen Edwards that captures one important insight:

The people who would’ve adapted to the next thing you did not see coming are gone. Institutional knowledge is not a stockpile you can extract and keep. It’s a river – cut off the source, and it dries up.
— Dr. Helen Edwards

The learning organization is a system that adapts. There is no evidence that the adaptation required for an organization to learn and succeed in face of evolutionary pressure (e.g. competition) can work without a human substrate.

As a Luhmannian myself, I’m keenly aware that in the theory of dynamic systems of communication, we can abstract away humans and actors from the organization and treat it as its own kind of entity. 0 human workers, all AI? No problem for the theory as long as communication is done. But the theory of communication also requires sense-making, and while the Chinese Room of LLM chat is convincing to humans that make sense of what they read, it doesn’t follow that the LLM’s “make sense of” anything at all. So an organization, or any system at all, where only LLM’s send messages around, many things can happen, but none of them need to be sensible.

Looking at OpenClawd and Moltbook: whatever a human interpreter may find and see in this, it doesn’t automatically mean that the LLM’s in the message exchange make sense of anything at all, or that they communicate.

If you’re not into systems theory but sci-fi, read Peter Watts’s Blindsight. It’s a very cool novel about a “first contact” event, and cool world-building on top. You’ll see it quoted more in response to LLM pollution. Quoted from Watts’s website where the whole book is available for free:

Imagine that you encounter a signal. It is structured, and dense with information. It meets all the criteria of an intelligent transmission. Evolution and experience offer a variety of paths to follow, branch-points in the flowcharts that handle such input. Sometimes these signals come from conspecifics who have useful information to share, whose lives you’ll defend according to the rules of kin selection. Sometimes they come from competitors or predators or other inimical entities that must be avoided or destroyed; in those cases, the information may prove of significant tactical value. Some signals may even arise from entities which, while not kin, can still serve as allies or symbionts in mutually beneficial pursuits. You can derive appropriate responses for any of these eventualities, and many others.

You decode the signals, and stumble:

I had a great time. I really enjoyed him. Even if he cost twice as much as any other hooker in the dome—

To fully appreciate Kesey’s Quartet—

They hate us for our freedom—

Pay attention, now—

Understand.

There are no meaningful translations for these terms. They are needlessly recursive. They contain no usable intelligence, yet they are structured intelligently; there is no chance they could have arisen by chance.

The only explanation is that something has coded nonsense in a way that poses as a useful message; only after wasting time and effort does the deception becomes apparent. The signal functions to consume the resources of a recipient for zero payoff and reduced fitness. The signal is a virus.

Viruses do not arise from kin, symbionts, or other allies.

The signal is an attack.

[…]

How do you say We come in peace when the very words are an act of war?

(Strong emphasis added by me.)

Thanks Carlo Zottmann for the book recommendation. Didn’t expect it to come in handy this much.