your agent remembers. your fine-tune doesn't.
edit: yeah we built the thing in this post. still mad about the problem though.
support has had a memory-backed agent running for like three months. works. incidents retained. deploy region flips retained. recall in prod is actually correct. rare, I know.
then ML wanders over. "hey can we get JSONL for a fine-tune."
what goes in the zip file?
chat export. messages[]. maybe someone regex'd out the system prompt. no memory IDs. no "fact A got superseded on Feb 3." no link to the document chunk anyone actually trusted.
you fine-tune. model comes back confident. cites jack shit. confidently wrong about a deploy region you fixed in the bank six weeks ago.
you open the memory UI. bank was right. dataset was fiction.
that's not "the LLM hallucinated." that's two teams shipping different religions and nobody noticed until after the GPU invoice.