A tiny model,
a useful illusion.

Fugazi was one of the experiments that pulled me toward AI and ML. It is tiny by modern standards, and that is part of why I could follow what it was doing.

The useful illusion

Feed it text, ask for a few sentences, and it produces something with a familiar cadence. The words look intentional from a distance. Up close, the mechanism is wonderfully plain: it has seen which words tend to follow other words, and it makes the next choice from those observations.

Fugazi / sample run seed = 1

We built a platform. Platforms are wonderful because they can do anything.

The little trick is that it borrows rhythm without knowing what any of it means. That gap between how a sentence sounds and what is happening underneath is the interesting bit.

Two words at a time

With its default order of two, Fugazi remembers a two-word context. It counts the words that followed that context in the corpus, then samples a next word according to those counts. The chosen word shifts the context forward, and the process repeats.

A fixed random seed lets me repeat a run exactly. Starting at sentence openings keeps the result from beginning halfway through a thought. Neither choice makes the model understand the text, but both make the experiment easier to inspect.

Why it stayed with me

The part I like is being able to trace a result back to small rules. There is no hidden service to call and no large model to download. I can change the corpus, adjust the context length, or fix the seed and see what moves. That made this a useful doorway into bigger questions about language models.

It also taught me to look twice at fluent output. A sentence can sound sure of itself while carrying very little underneath.

Where it stops

Fugazi only has the short context I give it. Raise the order and it follows the source more closely; lower it and the output gets stranger. It also drops most punctuation, so clauses can blur together. Those limits are visible, which makes them good places to keep learning.