Only intentional disturbance is illegal.
For Schedule 1 birds:
The Wildlife and Countryside Act 1981 regulation 1 (5) was amended from 'intentionally' to 'intentionally or recklessly', so reckless disturbance (e.g. climbing when you know there are Schedule 1 nesting or young birds that could be disturbed by your climbing) is also illegal.
For bats:
The Wildlife and Countryside Act 1981 regulation 9 (4) (again as amended) makes it an offence to 'intentionally or recklessly... he disturbs any such animal while it is occupying a structure or place which it uses for shelter or protection'. So again, reckless disturbance is illegal.
The Conservation of Habitats and Species Regulations 2017 regulation 43 does say 'deliberately disturbs wild animals of any such species' which is probably poor drafting but I'm not a lawyer so wouldn't want to guess what you could get away with (and wouldn't want to anyway - I don't want to disturb bats!).
This legislation has a better definition of disturbance though:
"For the purposes of paragraph (1)(b), disturbance of animals includes in particular any disturbance which is likely—
(a)to impair their ability—
(i)to survive, to breed or reproduce, or to rear or nurture their young; or
(ii)in the case of animals of a hibernating or migratory species, to hibernate or migrate; or
(b)to affect significantly the local distribution or abundance of the species to which they belong."
The important thing is that normal caving in normal caves with the usual care around bats does not, AFAIK, cause disturbance.