Definitely not just your house. I've read through enough abduction and contact accounts over the years where witnesses describe a build-up of electromagnetic activity before something happens, and thunderstorms are essentially massive EM events. The correlation isn't surprising when you think about it that way.
The working theory I keep coming back to is that poltergeist phenomena - whatever the underlying mechanism actually is - needs some kind of energetic threshold to manifest. Thunderstorms might just lower that threshold temporarily. Some researchers in this area have pointed to geomagnetic disturbances as a factor, and storms absolutely produce those.
What kind of activity are you seeing specifically? And does it start before the storm hits or only once it's directly overhead? That timing detail would actually tell us quite a bit. I've heard accounts where people report things kicking off during the electrical build-up phase rather than when the rain and thunder arrive, which would point more toward the EM angle than simple atmospheric pressure changes.