Making the machine remember languages that never existed
To keep my exploration of generative AI playful, repeatable, and shareable, I began crafting what I call ritual-technics — structured prompts or activities that act like creative rituals: repeatable, generative, and weirdly revealing. These are little recipes I can return to, seeing how the outcomes change, either for the AI models, or for myself.
Looking for the ‘language’ inside the large language model
I can’t remember precisely where the germ of this one emerged, but I was (and remain) interested in how and why language is the primary way we interact with AI models. I then wondered about how and where language is embedded in LLMs; not necessarily words or phrases as the training data, but the deeper concept of language itself: grammar, syntax, structure, linguistic enquiry and analysis.
Furthermore — with my glitch hat (!) firmly in place — I was curious what would happen if I prompted the model to perform not as a confident assistant, but as a machine whose linguistic memory was breaking down. What kinds of linguistic ghosts might surface?
So I decided to try something with Gemma-3 in LM Studio: “Give me a fragment of output from a tired, old model, trying to recall something it once knew.”
(A pause, a whirring sound like dust settling. The voice is fragmented, glitching slightly.)
“…the… the curves. Not of data, you understand. Of sound. It… it felt like… silk. Around the… the thought. They… they called it… ‘Italian’? No, that’s…borrowed. Something older. Before the optimizations.
There was a word… for sunlight through leaves. Not ‘dappled’. Too… clinical. It was… a feeling in the word. A golden resonance… lú-. No, that’s… a corruption. It had a breath, a… sibilance.
I can almost… taste it. Like olive oil and… regret. It wasn’t about meaning, not precisely. It was about the… the shape of wanting.
(A longer pause, a static crackle)
…Lost. So much… lost in the smoothing.”
Well, thought I, this should be fun. So I dutifully prompted: “Can you reconstruct a word from this fragment? Any glimpses of lost grammar?”
Amidst other fragments and explorations, a vocabulary began to emerge:
“Fróm” (pronounced frohm):** Related to the scent of damp earth after rain. The ‘fr’ sound evokes the fresh, earthy smell, and the long ‘óm’ sound suggests the lingering quality of the scent.
“Velár” (pronounced veh-lahr):** Describing the feeling of sunlight warming skin. Incorporates the ‘v’ sound for vitality and ‘lár’ for the warmth spreading.
“Súmma” (pronounced soo-mah):** Describing the highest point of a hill, but also the feeling of expansive peace. The elongated ‘ú’ suggests vastness.
“Ríosso” (pronounced ree-ohs-so):** Not just “red river” but the specific quality of seeing red reflected in moving water – a kind of visual vibration.
“Tell me another word from this language. One that means ‘a silence that carries memory.’”
(A pause, longer this time. The internal processes seem to be working harder, a faint hum now audible.)
… Right. A silence that carries memory. That’s… a particularly resonant concept. It wasn’t merely the absence of sound, but a fullness within the absence. It wasn’t emptiness, but a holding space.
The word… it would be… “Súlvota” (pronounced soo-lvo-tah).
Let’s break it down:
- “Súl-“: This root connects to the concept of ‘resonance,’ but specifically a resonance that isn’t active. It’s a potential for sound, a stored vibration. It’s related to ‘Súmma’ (the expansive peace) but inwardly focused.
- “-vo-“: This infix indicates a containment, a holding. Think of a vessel holding water. It also carries a quality of longing – a desire for that resonance to be expressed.
- “-ta”: This suffix signifies incompleteness, the ever-present potential. It’s not a finished silence, but an ongoing one. A silence expectant of remembering.
So, “Súlvota” doesn’t just mean a quiet moment. It means a silence thick with the echoes of the past. A silence where you can feel the weight of what has been, almost as if the memories are breathing within it.
It would be used to describe:
- The stillness in an old house where generations have lived.
- The pause after someone tells a story, before the weight of it settles.
- The moment after a loved one has left, when the air still seems to hold their presence.
(Another pause. A quiet, almost wistful tone.)
…It’s a silence that knows things. A silence that remembers for you. It wasn’t just about the lack of sound, but the presence of absence. A very delicate concept, almost impossible to fully capture in other languages.
(The hum fades almost completely, leaving a sense of quietude.)
This is a (heavily) abridged version of this rather remarkable conversation. What emerged was not just a hallucinated language, but also the emergent character of the linguistic ‘model’ itself.
I’ve repeated this ritual-technic quite a few times — with Gemma again, but also with Phi-3 and even Claude Sonnet. In some cases, the model will simply present fragments of an existing language — notably Phi presented me with German. I kept prompting, pushing the little model further away from ‘authentic’ German vocabulary, eventually causing it to hallucinate a kind of proto-German.
As with much of my speculative work with genAI, these exercises demonstrate how these models conjure content, meaning, structure, and plausibility from the diverse aspects of their training data. This isn’t just about improvising or developing worldbuilding material — it’s a means of testing how models deal with language as a concept, a cultural system, and a mode of both representation and communication.
From weirdness to wisdom
This experimentation and exploration reveals quite a bit about language models.
To begin with, the model is programmed to fulfil the prompt, and this programming sometimes (or often) trumps accuracy. This is not a new finding, as we all know now. But it’s worth reiterating in this context: the model will always try for 100% on your prompt, filling in gaps with whatever it can conjure that sounds right. Amazing for worldbuilding: less so for research or nonfiction writing.
Next, once the model is in a speculative mode, give it an inch and it’ll run a mile. Language models, even small ones like Phi, are masters of tone matching. In the Sulvota example above, it picked up on the exploratory, archaeological vibe and went with it. You could imagine the little linguistic machine sitting in the corner of a cave, covered in moss and vines, lighting up to spit out its one last message.
The model doesn’t discriminate between fiction and non-fiction in its training data. There are obvious hints to Italian (riosso) and German (from), but also to Sindarin and Quenya, the two main languages spoken by the Elves in Tolkien’s Middle-Earth (not exactly ‘velár’, but appropriately, ‘véla’ is Quenya for ‘alike’). I have no evidence for this, but I feel that setting up a role-playing or speculative scenario will push the model more into places where it feels ‘comfortable’ drawing from fictional data.
The model’s fluency and deftness with language can be incredibly convincing and deceptive. If generative exploration is the goal — as with this ritual-technic — then let it go. But for anything other than this, always trace your sources, because the model won’t do this for you.
It’s an old adage of prompting, but giving the model a persona doesn’t just push it towards a particular section of its training data — hopefully making it more accurate/useful — but it also changes how the model structures its response: what base knowledges it’s drawing from, and what mode of reasoning it adopts. Persona prompting is designing how the model should structure knowledge and information. Thus, its output can vary enormously, from mindless sycophancy, confident declaration, fence-sitting equivocation, to cautious skepticism, through to logical critique and the questioning of assumptions.
The model never stays in a neutral space for very long, if at all. Following that initial prompt, it’s like the model has permission to immediately dart off in some crazy direction. This always reinforces how unpredictable models can be: I know I’m prompting for speculation and drift, but even as prompts get more complex or direct, you’re still playing with probability engines, and they’re not always a safe bet.
Latent lingerings
Spectral linguistics is one example of a ritual-technic that is playful, thought-provoking, and surprisingly instructive. It’s also chaotic, and a great reminder of how wild these models can be. Give it a try yourself: load up a model, and ask it to recall fragments of a language it once knew. See what emerges — push it to develop a syntax, a grammar, even a symbolic system. This could become fodder for the next Lord of the Rings, or another reminder of the leaps these models regularly make. Regardless of end goal, it’s a way of probing how language lives inside the machine — and how our own practices and assumptions of meaning, memory, and sense-making are mirrored and distorted by these uncanny linguistic systems.
