Yeah this is something I've been noticing for years. Old buildings especially - churches, manor houses, that kind of thing. The cold spots aren't random, they cluster in the same corners or doorways every single visit, regardless of season or weather outside.
My thinking is it's got less to do with ghosts in teh traditional sense and more to do with the geometry of the space itself. Certain angles and materials might act as natural focal points for whatever energy or entities pass through. Norfolk has no shortage of old properties and I've catalogued probably a dozen sites round here where the cold spot is consistent to within a metre or two every time.
Anyone actually mapped these spots against the original architectural plans of a building? Curious whether they line up with older structural features - pre-renovation walls, original foundations, that sort of thing. Wouldn't surprise me if there's a pattern nobody's properly documented yet.