I don't suppose the officials at DEFRA really give a toss whether cavers enter caves on access land. They are probably just overstretched and under-resourced and don't want any to have divert time, manpower and money to resolving the thorny legal questions and issues it might throw up.
Let us imagine that the interpretation of the act is changed to include caving....
DEFRA receive a letter complaining that Upper Flood and Waterwheel are gated denying cavers their rights. What do they do? Probably nothing for days, weeks or months while they ruminate on it. Then they take the simplest option - write to the landowner, in this case Somerset County Council, and say we have this letter - what is your position? After more rumination of days, weeks or months, SCC write back with this response.... They point out that they own an outdoor centre (Charterhouse) in close proximity and that the children use the access land, unaccompanied, for orienteering. They point out they have a duty of care to keep the caves gated and locked. So where would we go from here? Which is the greater legal obligation - open access, or duty of care to the public, in particular children visiting the SCC owned centre?