It really depends what sort of caving you start with.
I started in the local mines, undergound stone quarries - no fall potential apart from the shaft.
If someone had told me I needed to spend 200GBP on kit I would have laughed at them and found another hobby!
I used an old boiler suit I had, old wellies, a hard-hat I bought as I did not want to smack my head and a cheap head-torch. By far the greatest expense was 50m of sisal rope from the local DIY store to make a rope-ladder long enough to get down the shaft (No.3 - 17ft deep - same site as PB's picture from 1979)
Eventually I joined our club and as I met more people I gradually saw what others had, its benefits etc and quickly learnt the short-comings of my kit.
The one new item I bought fairly early on was a helmet - hard-hats have peaks which make crawling awkward as you cannot see where you are going.
Slowly over time I acquired a few second hand items such as oversuit and lamp - someone showed me how to cannibalise a flashlight to use the lead-acid cells inside.
By this time I was hooked on caving and sufficently experienced to understand what all the kit was for and what I really needed.
Only at this point was I ready to commit hard to come by cash for proper kit - my first trip to S.Wales involved an early morning stop-off at Dragon to get a furry and neoprene socks as I would be wet caving for the first time.
And then I got into SRT - no short-cuts there, my homemade battery pack proved too bulky and I treated myself to an FX Ion because of its tiny battery.
Now, a good few years later I have probably forked out around 500GBP for my full kit and various bits and bobs - none of this has been wasted and contrasts with the 30GBP or so it took me to get into the sport.
Only people with more money than sense (or extremely good fortune) would consider forking out that sort of money on a hobby they may not like.
Nowadays I have enough spare kit and others around me with the same that we can usually ensure that new people have all they need for reasonably comfortable and safe caving.
But despite the cost of the kit it lasts a long time, and overall caving is a remarkably cheap hobby.
Any old kit will do as long initially as you do not try to exceed its limitations - caving is all about the learning experience.