A Tutor That Remembers: A Subject Memory That Outlives the Context Window

The shape of the system

Three stores, one budgeted read per turn, one loop at night. Files feed the map; the conversation feeds the episode log; checks feed beliefs. Each chat turn reads a small, ranked slice of all three and puts it into the prompt inside a fixed token budget. Nothing in the read path calls a generative model. The one model it does call is the embedding, once per turn, shared by routing and recall; the only generative pass outside the chat is the one that reads a new file into the map.

Episodes: an append-only, hash-chained log

Every event becomes an episode: a question, an answer, a check with its verdict, a quiz result, a file arriving, something the student explicitly taught the tutor. An episode carries a one-line summary, the full text, the concepts it touched, a salience score set at write time, and its evidence: for an answer, the file and page it cited, stamped with the file's fingerprint at that moment. Episodes are never edited. Each one stores the hash of the one before it in the same chain, so the log is recomputable and any tampering, or any silent rewrite by a well-meaning migration, shows up as a break. That is tamper evidence, not tamper proofing: a rewrite is detected, never prevented, and only for as long as the chain or a checkpoint of it is kept.

Salience is set by what an episode is, not by a mood: a missed check (0.7) matters more than a passed one (0.6), and something the student taught the tutor (0.85) matters most, because it is the one memory a person has explicitly asked to be kept. A question is a modest 0.35: a student's own question is a correct recall and a useless reminder, which turns out to matter for the "what does this remind me of" path later.

Recall: relevance must beat recency

Recall is associative, not chronological. Candidates are gathered from several pools, each reaching the whole history, and only then scored: the most recent five hundred, every episode containing a distinctive word from the question, every episode touching the concepts the question was routed to, and the semantic neighbours from a vector index. Widening first and slicing second is the single most important design decision in the module. The first draft sliced first ("the newest 500") and could not recall a perfectly relevant memory from March at any relevance, because it was cut before it was scored.

Weights alone are not enough. When the question is about something and anything matches, episodes that match nothing are dropped rather than ranked: a fresh irrelevant note scores 0.15 × 1.0 on recency alone, while a 200-day-old exact match scores 0.55 × 0.156, and no weighting that lets a non-match outscore a match is right at any setting. Recency is a tie-breaker, not a ranker. Two questions a reader should ask about those numbers: where they came from, and whether we fitted them. They were hand-set, carried over from an earlier memory of ours built for a different product, and they have not been tuned on any of the data in this article. Nothing below was swept or fitted; the one change we made to the scoring came from a failure we could name, not from a search, and it was the gate rather than a weight. The gate has two exemptions: no query at all (then recency is all there is), and nothing matched (then the recent past is the honest best offer).

Two things real histories added

Repeats collapse. In one real subject 161 student questions were 56 distinct sentences: "what is a cell" asked nine times over three months. Nine copies are one memory, remembered nine times over, so recall collapses identical episodes and reports how many the survivor stands for. Time is a filter, not a hint. "What did we cover in July?" parses to a window and recall runs inside it first; if the window is empty the block says so explicitly (see § honesty). Month names work in the languages our students write in, as do "last week" and "yesterday".

The concept map

Passages answer "what does the file say here"; a map answers "what is this subject made of, and what builds on what". Each file is read once by a model pass that returns a bounded list of concepts: name, one-sentence gist, a short definition, the concepts it builds on from its own list, the pages where the file explains it. Section titles ("Introduction", "Common misconceptions") are rejected by a validator; a name that is really a chapter heading is the file's furniture, not the subject's ideas.

Concepts merge across files by a canonical key (a normalised name, script-aware) and by embedding similarity above 0.86. Provenance accretes: a chapter that re-explains osmosis adds itself to one node rather than creating a second, and every file that contributed stays listed with its pages. A prerequisite the file assumes but never explains becomes a node only when two concepts assume it; then the gap is real and worth showing; a one-off is recorded on the concept as "assumes" and drawn nowhere. The first policy document we read turned 11 concepts into 34 nodes before that rule existed.

Three derived numbers come out of the map. Centrality is a weighted PageRank over the edges (a prerequisite that everything builds on is a hub). Readiness per concept follows four bands: 0.35 named only, 0.60 explained by a file, 0.85 confirmed by a passed check, 1.0 taught by a person, and the subject's readiness is the centrality-weighted mean, so a hub the files do not explain costs more than a leaf. Teaching order is a topological layering: prerequisites first, hubs first within a tier; it replaced a model call that used to outline "what to teach" from fourteen passages.

SUBJECT MAP (16 concepts; the files explain 79% of it, this is about the material, NOT the student's mastery):
- Mitochondria: Mitochondria release energy from glucose. [taught: in our class we call the mitochondria the powerhouse of the cell] (builds on: Cytoplasm)
- Cytoplasm: Cytoplasm is the gel-like substance inside the cell membrane. (builds on: Cell Membrane)
- Chloroplasts: Chloroplasts are involved in photosynthesis. (builds on: Cell Wall)
- Endosymbiosis: Endosymbiosis explains the origin of mitochondria and chloroplasts. (builds on: Mitochondria, Chloroplasts)
…

That header wording was a lesson in itself. The first version said "79% understood", and a model reading it happily told a student "given your current understanding of 79%…". A readiness number that can be read as the student's mastery will be read that way; the line now says what it is about.

Teaching the subject

A person can add to the map by talking to it. "Remember that in our class we call the mitochondria the powerhouse of the cell" attaches a taught note to the concept it names (routing by name; an exact match only), sets that concept's readiness to 1.0, writes a taught episode, and puts the note into the brief. When the sentence names no existing concept, a new node is minted at the taught layer with a short name, never the whole sentence. Questions never teach; only an explicit "remember that", "treat X as", or its equivalent in the student's language does.

Beliefs that keep their history

A belief is what the tutor currently thinks about this student on this concept ("struggles with osmosis", "understands cytoplasm"), with a confidence and the episodes behind it. The rule that matters: rows are never updated into a different claim. When the tutor changes its mind, the old row stays exactly as it was, a new row is inserted, and the old one points at the new one. Recall reads only rows that nothing supersedes; a report can walk the chain when someone asks what changed and why. Retention and currency are different properties, and a memory that keeps both is one a report can be built on.

Beliefs are cheap where it counts: about twenty tokens each, and one stands for dozens of episodes. That ratio, not a bigger window, is what "remembering more than fits in the context" actually means. Episodes are not dropped (they are what a belief is checked against), but they come second and the caller spends its remaining budget on them.

Which topic is a belief about?

This was the part real data broke first. In teach mode, the "topic" of a check had been the student's own words, so replayed checks produced beliefs like understands ask me the questions and struggles with bueno ahora explícame todos los pasos. Now a check's topic is routed to a concept by name, and when the topic string is a sentence, by the tutor's own evaluation text (which usually names the concept) on an exact-name match only. A topic that maps to nothing and reads like an utterance (question words, verbs of asking, pronouns, fillers, in the languages our students use) is kept as evidence but never becomes a belief. A rule change like that needs a way to re-derive: retire the current beliefs (never delete), replay the checks on record under today's rules, touch no episode.

One turn, within a budget

Each chat turn asks the brain for one block, built off the request path with a hard time cap. Over 155 turns the block took 295 ms at the median, 403 ms at the 90th percentile and 445 ms at the 95th, with one build over a second and none over the 2.5 s cap, past which the bundle is dropped rather than awaited so a slow memory never sits in front of the first token. One embedding of the question serves both routing and recall. The block goes into the prompt's existing memory slot, ahead of the material passages, and nothing else about the prompt changes.

===== YOUR MEMORY OF THIS STUDENT AND SUBJECT =====
THE STUDENT IS ASKING ABOUT THEIR OWN RECORD. Answer from the beliefs and the earlier work listed here …
WHAT YOU CURRENTLY BELIEVE ABOUT THIS STUDENT (from their checks; confidence 0–1):
- understands cytoplasm (0.72)
- understands eukaryotic cells (0.72)
EARLIER WORK WITH THIS STUDENT (oldest first):
- 12 Jul · you were asked: Can we go over Eukaryotic Cells today? I keep mixing it up.
- 12 Jul · you answered: Of course. Eukaryotic Cells: Eukaryotic cells contain a nucleus and organelles. We worked through two examples and you got the second one right.
- 18 Sep · the student taught you: Taught: in our class we call the mitochondria the powerhouse of the cell
- 18 Sep · check: Check on cytoplasm: right (and 3 more times like it)
SUBJECT MAP (16 concepts; the files explain 79% of it, …)
…
HOW TO USE THIS: it is your own memory of past work with this student, not their material. When you rely on it, say when it happened ("in July we…"). Never claim a memory that is not listed here. …
===== END OF MEMORY =====

Order inside the block turned out to matter as much as content. For an ordinary question the map comes first. For a record question ("what did I get wrong so far", "what have we studied", "ماذا أخطأت"), the beliefs and the earlier work come first and the map last, with an explicit first line saying the student is asking about their own record. We learned this from a failure: the prompt also carried a legacy "assessment results" section that begins "NO QUIZ ATTEMPTS recorded", and the model answered a student who had answered twenty-eight checks in conversation with "you haven't attempted any quizzes yet". A model reads the first authoritative statement about a record and stops. The memory block now precedes that section, a bridging note ties the two together, and an assessment section that only says "nothing recorded" is dropped when the memory block exists.

Saying "I have no record"

The most instructive failure of the whole project was a scenario we wrote to pass: seed a July conversation, ask "what did we cover in July about X", check the answer. The answer began "In July, we covered X…" and was fluent, grounded in the file, and completely fabricated. The July memory had not reached the prompt, and the model narrated July from the question's framing. Two causes. The record-question mode had kept only checks and quizzes and dropped the conversation that was the record; and the tutor's own earlier narrations ("In July we covered…", stored as answer episodes) were being recalled as if they were memories of July, the brain remembering that it remembered.

Both cases are now scenarios in a repeatable runner: the July one passes only if the July memories are in the prompt (checked on the message's stored memory ids, not on the wording of the answer), and the empty-month one passes only if the answer says it has no record and does not describe a session.

Memory as a citation

Our chat already had a provenance panel: every claim in an answer is traced to a line in the student's files, and an answer with too few traced claims is marked as mostly ungrounded. That tracer only knew files. A correct "in July we covered X, and you got the second example right" would therefore read as not in your notes: the feature calling a true statement false. So the memories the tutor was reminded of are stored on the answer, and the tracer treats each recalled episode as one more source, labelled with when it happened.

The same grounding result gives each answer episode a stance (answered, partial, or cannot) stored after the fact. That is how the report learns which questions the files could not answer, without a second model call and without a hidden "frame" line in the stream, which we tried and rejected: in the plain answer mode nothing filters the stream, so a hidden line would have reached the student.

A memory that checks itself

A memory that cannot check itself gets more confident and less right with age. Because an answer's evidence records the file's fingerprint at the time, verification is a lookup, not a model call. Holds means the cited file is still there, unchanged. Changed means it was replaced or removed, and the note says which. Unverifiable means the memory cites no file, a conversation or a check. That third answer is a first-class result, not an error; reporting a conversational memory as "holds" would be the actual defect. A recalled episode arrives at the prompt already labelled: "[note: the file this cited has been replaced or removed since]".

The nightly loop

Once a night, for every student-and-subject pair active in the last day, a deterministic cycle runs with no model calls: compress the day into one consolidation episode (counts by type, the check tally, the topics, the five most salient headlines; a quiet day writes nothing, and the note never re-summarises its own kind); decay any belief untouched for 45 days by 15 % and flag it for the next check to confirm; calibrate the mastery the tutor predicted before each check against the verdict; verify a bounded batch of citations, never-checked first; rebuild the small working set; recompose the self-model.

Calibration is the honesty check on the tutor's confidence: before each check we know what mastery it predicted; after, we know the verdict. Bucketed, that gives a reliability curve and an expected calibration error, and the self-model reports "calibrated" only from five checks with a prediction. Until then its limits say so: treat confidence as a ranking, not a probability.

What real histories taught us

Scenarios prove mechanisms; they do not prove the thing works on the way people actually talk. So we built a replay: take the richest real student-and-subject histories, read-only, rebuild each one in an isolated copy of the platform, build the brain for it, and ask three real-shaped questions through the actual chat endpoint as that user. Six pairs, 65–142 messages each, 13–32 teach-mode verdicts each, one in Spanish. They are not a random sample and should not be read as one. We took histories with at least forty messages, at most two per student, test and internal accounts excluded, preferring the ones with the most graded answers, because a history with nothing in it exercises nothing. That biases every number here toward heavy users. A light user's subject would give the memory less to recall and less to get wrong.

It also produced the three fixes described above: beliefs on student sentences, the record answered from the map instead of the record, the legacy "no quiz attempts" line leading answers, and one confirmation we were glad to see: a consolidation flagged eleven memories whose cited file no longer existed. The verifier doing its job.

The algebra row is the honest one. Thirty-two checks and no belief, because none could be tied to a concept: the student answered arithmetic, the tutor's replies never named the chapter's concepts, and an utterance is not a topic. The report for that student says "32 checks are on record, but none could be tied to a concept in the map yet" rather than "no checks yet". Nothing invented, including the mastery.

A benchmark, not a demo

Everything above is a measurement or a sample: how long things take, what the replay produced, answers we liked. None of it establishes that the memory earns its place, because nothing was compared against the same tutor without it. So we ran the comparison. One pre-registered question set, four designs of the same tutor that differ only in what reaches the prompt, and scorers that are string and id comparisons rather than opinions, so the same answer always scores the same way.

Each subject here is one student's one subject, and its questions are generated from that subject's own recorded data, which is what makes them scorable: a question about what the student got wrong is checked against the checks actually on record, a question about a prerequisite against a real edge in the map, a question about a month against the months that subject was actually active. Where a subject supports more than one, it gets more than one, so three concepts and three empty months rather than one of each. Six categories, 420 scored questions per design, over 45 real student histories replayed in an isolated copy of the platform and asked through the ordinary chat endpoint as that student.

The four designs

All four run the same retrieval over the same files. Plain retrieval is the tutor with no memory at all. Window is the answer most people reach for first: the last twenty conversation turns, dropped in unranked. Episodes is ranked recall of the episode log alone, with no concept map, no beliefs and no special handling of questions about the record. Full memory is everything this article describes.

What the table says

Overall the tutor goes from 168 of 420 to 383, which is 40 percent against 91. The average is the least interesting number here; the shape of the difference is the finding.

A bigger window is not the answer. Dropping the last twenty turns into the prompt takes 40 percent to 65, and that is the whole of what it buys. It answers 5 of 32 questions about what the student got wrong, because twenty turns of chat is not where that lives, and it is still wrong about an empty month more often than not. This is the "just put more of the conversation in" reply, measured rather than asserted.

Ranked recall does most of the work. The episode log on its own reaches 83 percent. If you build one thing, build that: an append-only log, scored on relevance, is most of the value in this article.

The map and the beliefs are worth the other 33 questions, and they earn them in two specific places rather than everywhere. Asked what to revise before a topic, the designs without a concept map answer correctly about 55 percent of the time and the full memory 96, because prerequisites are edges and an episode log has none. Asked what the student got wrong, beliefs and the record branch take 24 of 32 to 30. Asked to explain a concept, the map adds nothing at all: 100 either way.

What it cost to buy that

What this benchmark does not show

The last of those numbers is the one to keep in mind while reading the rest. We asked the full memory the identical 420 questions a second time, and 44 of them scored differently. So run-to-run noise is about a tenth, the 33 question gap between the top two designs is comfortably outside it, and the 2 question gap between them on the empty month is not: on that category they are tied and we will not claim otherwise.

One category turned out to be worthless and we are leaving it in the chart rather than quietly dropping it. A month the student was active scores 44 of 44 for every design, including the one with no memory at all. It cannot tell grounded recall from a plausible description that happens to carry the right month, which is exactly the failure its sibling category catches. A test that everything passes measures nothing.

The scorers are string comparisons written by the same people who built the feature. They reward an answer that reuses the recorded words and cannot see a correct answer phrased entirely differently, and they cost us badly once: the prerequisite questions were generated with the edge direction reversed, so for a long time this category read as the memory's weakest when the tutor had been giving the map's own answer and being marked wrong for it. It is the map's strongest category. We found that by reading the failures rather than the totals, which is the only way these mistakes surface.

And the population is not random. These are 45 real histories chosen for having something in them: at least fifteen messages, files attached, and for the categories that need graded answers, at least five of those. A student who has barely started would exercise far less of this, and would see far less of the difference.