Never been to Remington specifically but I've had similar experiences after visiting a few sites here in Suffolk - one particular ruined priory where I came home and my camera was doing things it had no business doing. Orbs on every shot for about a week then nothing.
The "followed home" phenomenon is well documented in paranormal photography circles. Some researchers think it's residual attachment, others reckon you've just become more attuned and are picking up on things that were already there. Hard to know.
What exactly happened after you got back? And did you have any recording equipment with you at the asylum? That would help narrow down whether you're looking at something environmental or genuinely anomalous.