Another vote for merino thermals, but I do find smooth synthetic ones better for the leggings because the weave gives me sore knees when I'm crawling otherwise (I know, big softie). I sometimes push the sleeves of my thermal top and furry up, under my oversuit, if we're crawling through a wet section, to keep them dry longer. That seems to help.
I also wear a fleece buff round my neck and a thin one under my helmet, and I sometimes carry a balaclava too.
Neoprene cuffs make a big difference, and long wetsocks with wool socks underneath.
Some types of gloves (the blue plumbers ones) whip heat away - I like the dipped-in-rubber woven type.
A bivvy/bothy bag of any kind is really effective - the clear plastic over-the-head type are pocket sized and work really well (can you still get those?).
A friend wears two furries because they frequently get cold otherwise.
I've worn Rab Power Stretch salopettes/bib (not for caving) and they're amazing. I think they would be great under a furry for really cold trips.
A bit of bioprene certainly helps!