You're definitely not losing your mind. Shadow figures at mills and old industrial sites are genuinely some of the most well-documented cases in the UK and I've personally encountered something very similar at a derelict textile mill in Bury St Edmunds back in 2019. It was a solid dark mass, roughly human shaped, moving against the wall in a direction that made no sense with any of the light sources present.
What gets me is how consistent the descriptions always are across completely unrelated witnesses. Did it have any distinct features or was it more of a flat silhouette? And did it move fluidly or was there something jerky or unnatural about it?
Also worth asking - had anything been moved or disturbed in the mill before you saw it? In my experience these sightings often cluster around locations where objects have been relocated or the space has been recently altered. Would love to hear more detail from anyone else who's been out there.