Access info update, December 2016:
Having just returned from the island here's the most recent understanding on access:
All access to all caves is subject to permission, but gaining permission is quite involved and in some instances (National Park) it won't be granted unless it is something like a University Research Project, with a six page application being filled in beforehand. Some of the caves are in the National Park, some elsewhere are Natural Monuments, others are in relatively open areas and no problems have been reported or experienced.
All caves are off limits due to bird nesting season between (and including) January to May.
The island is a Biosphere Reserve with the following Protection Classifications (& number thereof)
National Park
Natural Park (2)
Comprehensive Natural Reserve
Specially Protected Bird Area (SPBA) (5)
Natural Monument (5)
Site of Scientific Interest (2)
Protected Landscape (2)
Marine Reserve
Site of Community Interest (SCI) (11)
Furthermore the island also recently became a Geoparque with a further tranche of funding and attendant restrictions on freedom of movement, principally with fines for wandering away from delineated paths.
Don't let any of this put you off visiting the magnificent island, though, as it has fabulous R&R, usually excellent weather and if you are a caver and like rocks (you do like rocks, don't you?), then you'll find it very rocky and therefore very up your street.