As an advocator of the 'ploughing ones own furrow' approach (which is much more adventurous and ultimately rewarding) you would need to find a like minded friend of sound judgement, read as much caving literature as you can lay your hands on, have a healthy regard for prevailing weather conditions (rain), make sure you don't fall down any holes and keep looking back so you can find your way back out!!
Clothing needs to be warm and suitable (hence the reading caving literature) and lighting needs to be bombproof (ie good helmet mounted light plus spares).
Asking nicely would provide a whole host of suggestions for initial forays into simple caves from forums such as these, caving shops etc..
For anything more ambitious than simple trips you might consider a day or two with a professional caving instructor who could answer all your questions, provide non technical equipment (clothing etc) and technical equipment for SRT (ie abseiling down ropes and climbing back up them), accompany you on trips and generally mentor you in the right direction, giving a lot of practical advice and information with a very keen eye on safety. http://www.caveinstructor.org.uk/
Clubs, as Stu says can be cliquey and socially intimidating and don't suit everybody!! I know several people in well established clubs who have serious reservations about the technical competence of the people who lead and take responsibility for less experienced club members!
I was brought up on Merseyside and now cave mainly in the Yorkshire Dales, though the Peak District isn't far away from Merseyside. There is a lot more variety of caving in the Dales.