Training-Serving Skew

Every pilot is built on data that someone looked at. An analyst exported the tickets, a domain expert removed the duplicates, an engineer fixed the encodings, and the team agreed on what a "valid" record looks like. The model, the prompts, the retrieval index, and the evaluation set all reflect that cleaned world. Then the system goes live and begins reading from the real stream: late events, half-filled forms, mixed languages, renamed fields, and values nobody anticipated. Quality drops, and the team's first instinct is to blame the model. Usually the model is the same; the data is not.…