Good question, but you could also ask 'what constitutes a caving club', and 'according to whom', and 'for what purpose'.
Basically a caving club can be almost any kind of group - large or small. The point that was being made about the CNCC member clubs, (which is where this thread came from), was one of scale and voting power. A small club such as the Dent House Speleological Society has only 2-3 members left, but are still a club never the less. However, they have the same voting power as does a club like the Bradford or Craven which has several hundred members, or even clubs with 20-50 members for that matter. This is then a question of caver representation. In theory five mirco-clubs with no more than twenty members between them can out vote four large clubs with over five hundred members. This was the problem alluded to in the threads about CNCC representation and fairness.
A club can be anything, it is how the regional and national bodies see those clubs that really counts.