A mine rather than a cave, and a wizard rather than witches, but...
"Some dwarf-gates will open only at special times, or for particular persons, and some have locks and keys that are still needed when all necessary times and words are known. These doors have no key. In the days of Durin they were not secret. They usually stood open and doorwards sat here. But if they were shut, anyone who knew the opening word could speak it and pass in. At least, so it is recorded, is it not, Gimli?"
"It is," said the dwarf. "But what the word was is not remembered. Narvi and his craft and all his kindred have vanished from the earth."
"But do not you know the word, Gandalf?" asked Boromir in surprise.
"No!" said the wizard.
(From The Lord Of The Rings, Book 2, chapter 4, by JRR Tolkien)