Mate you are absolutely not losing it. The Hargrove mill has been on my radar for a while now and I've heard at least three separate accounts from people who've never met each other describing the same thing - tall, no distinct features, moves wrong somehow like it's not quite synced with the light around it.
What gets me is the poltergeist angle nobody talks about. A friend of mine went out there last autumn and said objects were displaced between visits, stuff he'd marked and photographed. Shadow figures showing up in the same location as physical disturbances is a pattern I keep seeing in these cases and I don't think its coincidence.
Did the figure respond to you at all or just ignore you? That detail matters more than people realise. Also what time of day and what were the weather conditions like? Barometric pressure and humidity genuinely seem to affect activity levels based on what I've read.