I suspect we are approaching the key point I have been trying to get across for decades; EDUCATION!, if cavers understood farmers more, and farmers understood cavers more, what a happier world it would be for all concerned.
I recall a time when a cave owning farmer broke an arm in the middle of haymaking and was very worried and stressed, one of the local caving clubs pulled together to help him get his harvest in, result- no problems with access for that club, and eventual ease of access for all, he was educated that cavers could be useful!
Another local cave owning farmer was stretched for help at lambing time, a few cavers offered and gave assistance, result- access (and quite a lot of help from the farmer) to a previously forbidden dig and some passage found, along with a lot of understanding of the local hydrology.
All it takes is everyone understanding each other and appreciating each others way of life, if one caving numpty antagonises a farmer, its up to the more sensible ones among us (you) to discourage them and teach the farmers that we (you) are not all bad.
I understand both sides, as a Peak District farmer (thankfully on Gritstone, so no caves) and a Peak District caver of loong experience, I did however gain my farming experience on a cave owning farm.