Welcome to the forum Gary!
The most obvious local club is Cheddar Caving Club, there's also the Axbridge Caving Group and up on the hills at Priddy is the Wessex Cave Club, the Mendip Cave Group, the Bristol Exploration Club, the Shepton Mallet Caving Club, and slightly further a-field are the Mendip Nature Research Committee and the Cerberus Speleological Society.
Mid week (Wednesday) is traditionally diggers' night and many clubs will be trying to fool you into joining their insane gruelling hell hole slavedom. You might like that kind of thing (I love it!) but perhaps what you're really looking for is some joyous and non-threatening sports trips. You might want to get involved in going to nice places, doing photographs and musing on how best to write a prize winning trip report.
Good luck in your Dark Side Missions!