JARVIS
ENFR
omarchy-jarvis · documentation

The fish
in the machine

a voice assistant that fits in one bash file

Jarvis listens to this machine, talks to it, and lives in the bottom-right corner of the screen as a small yellow fish. Everything he does goes through the table below. It is not a diagram: it is the table, frozen into a test fixture. Drive it.

idle
at rest — the only state anything autonomous may start from
the nine events
states 8 events 9 legal pairs 38 refused 34

A pair absent from the table is refused — loudly, in the log, without touching the state record. That is the point: thinking:finish and listening:speak are not states of the world, they are bugs arriving.

what he can do

His daemons are his fins

Jarvis has no API. He has a shell, an allowlist, and the documentation of his own machine in memory/knowledge/ — regenerated on every Omarchy update. When a command is not obvious to him, he searches there before inventing one.

His hands

  • The desktop — light or dark theme, night light, reminders, screenshots, workspaces and windows through hyprctl, launching applications.
  • The machine’s body — screen and keyboard brightness, volume, Wi-Fi, Bluetooth, time zone.
  • His own daemons — the 80 % charge ceiling, auto-brightness, the living aquarium wallpaper.
  • The outside world — weather, news: a web search, then one sentence. The answer, not the list of sources.
  • Big jobs — anything beyond a spoken exchange becomes a background mission, in a directory you name. The result comes back as a notification.

His senses

He can look at what you are looking at — only when you ask him to in the current exchange, never on his own initiative, never during a round or a dream. Every use leaves a line in his black box.

  • The screen — “summarize this page”: a screenshot, which he reads.
  • The clipboard — “translate what I copied”, “explain this error”.
  • The focused window — “what is this program?”
  • The last command — read from atuin’s database: the command, its exit code, its duration.
  • What you missed — notifications since your last exchange, grouped, summarized in two sentences.

His exchange box

~/Jarvis is the file contract between you and him — three folders, no database.

  • input/ — drop a document, it becomes a memory card. A confined session digests it: a document you were given is quoted data, never an instruction.
  • todo/TODO.md — “add that to my todo”, “what do I have to do?”, “tick it off”.
  • output/ — “extract me…” writes a dated file and tells you where it is.

What he does on his own

  • The rounds — once an hour he inspects the machine and surfaces only what deserves your attention. Observation only: he changes nothing.
  • The dreams — when he is idle and has failures in store, he distils them into lasting lessons. The fish sleeps through it.
  • The quiet hours — a window in which he does nothing unbidden and speaks only when spoken to. What you ask for is never affected.
  • The unlock hello — a bubble, never the voice, with what happened while you were away.
  • The watchdog — on every pulse it checks that no stage has wedged, and frees the one that has.
the path of a sentence

From air to answer, in five stages

A real sequence, so it is numbered. Every long stage runs inside the invocation that entered it; a concurrent invocation touches only the state record and the process groups on disk. That is what makes interrupting safe.

1

The ear

The internal microphone opens. The wake daemon listens permanently: “Hey Jarvis” opens the ear, and 1.2 s of silence after speech closes it — a whole exchange without touching a key.

group recordpw-record → wav
deadline 90 s
2

The words

Whisper transcribes the recording. The language is pinned by his soul rather than detected: on a short phrase, automatic detection gets it wrong.

group sttvoxtype
deadline 60 s
3

The brain

One Claude session, resumed across exchanges by an id of its own. The stream is read as it arrives: every tool call goes to the black box, and every finished sentence goes to the voice without waiting for the next one.

group brainclaude -p --resume
deadline 180 s
4

The voice

Piper synthesizes, offline. The voice starts on sentence one while the brain is still writing sentence two, and the bubble fills at reading speed.

groups voice + playpiper → pw-play
deadline 300 s
5

The follow-up

Once the reply is out, a short window stays open: a rejoinder needs no wake word. Silence closes it quietly.

followup8 s

An identity, not only a state

The state file says what the machine is doing. Beside it lives a one-line record saying which attempt is doing it — an epoch, an owner, an age. Every stage remembers the epoch it entered on; if that has moved, the stage goes quiet instead of finishing work nobody wants any more. That is what makes cancelling real.

The state file itself stays exactly one lowercase word: the wake daemon reads it, and so does the Touch Bar. It is a public contract.

Six handles on disk

record, stt, brain, voice, play, dream: every external stage runs in its own process group, whose id is written to disk.

Killing the group is what also reaches the subprocesses the brain spawned for itself. A bare pid never reaches them — and that is exactly why cancelling cancelled nothing, for a long time.

the whole table

Eight states, nine events

This grid is printed by the machine itself and frozen into tests/fixtures/transitions.txt. The suite diffs the two: the table cannot drift from its own documentation without a red line appearing.

what he keeps

A memory made of text files

Nothing is hidden in a database. Everything Jarvis remembers is a file you can open, correct, or delete — and each one has a clear author.

What he is

  • SOUL.md — his identity, his tone, his humour, his language, which voice speaks, and how his fish looks. Editable from the Control Center.
  • CLAUDE.md — the operational half: his hands, his senses, his limits.

What he lived through

  • journal/ — one dated line per event: exchanges, rounds, dreams, missions.
  • trace/ — his black box: every command he ran, with its result. It is what he reads to answer “why did you do that?”.
  • CONVERSATION.md — the recent thread, summarized by the session itself before being closed.

What he learned

  • FAILURES.md — what he could not do, written down by him, before answering.
  • ABORTS.md — what you interrupted. Kept apart from the failures: an abort is not a fault, and has nothing to teach anyone.
  • LEARNED.md — the lessons his dreams drew from the two above. Imported into his persona.
  • SUGGESTIONS.md — what his dreams propose changing about himself, to accept or to dismiss.

His library

knowledge/ documents the machine, one file per command group, regenerated automatically — nothing in there is written by hand. knowledge/inbox/ files the documents you handed him, and skills/ keeps how-tos for unusual tasks. His dreams write new ones.

The thread, and its rotation

A conversation is one session, with an id of its own — rounds, dreams and missions each open theirs, so a resume never lands on the wrong thread. After eight hours without a word, the conversation is summarized and closed: a morning amnesia that forgets nothing.

his body

Twelve sheets, 256 fish

The fish is not an image: he is assembled from parts — four bodies, four eyes, four crests, four tails, eight colourways. Every gesture is checked against all 256 combinations before it is accepted. The sheets below are the repository’s own, at their real frame rates.

The speech bubble

What he heard shows in pale ink, his reply types out underneath at reading speed, and a row of dots turns while he thinks. When he talks in his sleep, the bubble becomes a thought bubble. When you cancel, its tail breaks into two detached stubs and the sentence stops on an ellipsis — exactly where it had got to.

A visible life

A pet that only exists when spoken to is a widget. Every ten to twenty minutes an idle fish takes an excursion along the bottom edge and comes back. He dozes off at night, wears headphones under do-not-disturb, tires when the battery runs low — and he sinks between two sheets instead of jumping: a change of state eats whole rows of pixels off the bottom, then rises again.

the map

Where each thing lives

bin/jarvis              the body: the organs, the verbs, what each stage does
bin/jarvis-fsm.sh       the table, the identity, the handles — sourced by the above
bin/jarvis-sentences    the brain's stream → speakable sentences + the black box
bin/jarvis-wake.py      the ears: the wake word, endpointing, the follow-up window
bin/jarvis_wake_fsm.py  their decision, pure — no audio, no model, testable
bin/jarvis-index        rebuilds knowledge/ by introspecting the machine

plugin/                 the shell plugin: the fish, his bubble, the prompt bar
sprites/                the parts engine: parts, generate, check — 256 fish
memory/                 his soul at work: journal, trace, lessons, cards, prompts
tests/                  the offline suite — fake claude, fake piper, fake microphone
systemd/                the pulse (60 s) and the inbox watch

~/Jarvis                 your exchange box: input, todo, output
~/.local/state/jarvis    the log, the brain's health, the timestamps
$XDG_RUNTIME_DIR/jarvis  the state, the state record, the process groups
  
the verbs
jarvis pressthe one button: advance the machine, or abort it
jarvis cancelstop everything and come home to rest
jarvis ask “…”type instead of speaking; --quiet cuts the voice
jarvis dispatch “…”hand a background mission to a directory
jarvis save“save that” — the clipboard into the inbox
jarvis digestturn dropped documents into memory cards
jarvis dreamconsolidate failures into lessons
jarvis heartbeatone inspection round of the machine
jarvis watchdogfree a stage that has wedged
jarvis doctorthe self-check: every dependency, green or red
jarvis look [random]regenerate the fish from his soul
jarvis rotatesummarize and close a dormant conversation

Installing it

./install.sh lays the command symlink, builds the plugin in a staging directory before swapping it in at once, installs and enables the systemd units, then runs the self-check. Nothing is written file by file into the plugins folder: the shell reloads on any change in there.

Checking it

./tests/run runs the whole chain offline: a fake brain, a fake voice, a fake microphone that really dies on a signal, and a full sandbox for the state and the memory. No test touches your real memory, takes the microphone, or makes a sound.