That feeling of not being able to stop thinking about it - that's the thing that always gets me. Your brain keeps going back because it knows what you saw wasn't normal, even if you can't explain it yet.
I had something similar a few years back out on the Pembrokeshire coast, just a shape sitting completely still above the hedgerow for what felt like ages before it just sort of drifted away without any sound at all. No drone noise, nothing. I'm no expert but I've been trying to photograph oddities like that ever since and the stillness is always what stands out. Normal aircraft don't just hang there.
Did it have any lights on it? And roughly how long were you watching it before it moved? Details like that really help when you're trying to work out what you actually saw. Would love to hear more about this one.