Yes, I know how frustrating it is. There used to be one just off the Totton junction of the M27 called "Swinges Junction" and it was brilliant. But as you rightly think, it was closed down due to local complaints etc. I don't know if there is a law or regulation but every time one pops up, it seems their time is limited due to local busy bodies pressure. Having said that, Swingers Junction did survive a few years.
I think travelling is you're only option at the moment, but good luck in your pursuit. I've not been, but I hear Kestrals near Heathrow is good, and is a bit nearer than the Midlands. I will try it soon and let you know if you're interested. . I did however try Bristol Gardens in Brighton, not a swingers club but a naturist spa with fun to be had if you wish, well worth a visit... Not the flea pit you mention in Bournemouth.
Again, good luck and I would be interested in hearing if you do find anything.
Hope this helps and welcome to Southampton ?
Kind regards
Mike