It all depends on whether you are rushing or just caving to your ability, some people can move rather quickly and have good enough coordination and stamina to move quickly though a cave and be in no more danger if they are used to moving at that speed, what you see as rushing might just be a fast pace to someone else but still well within there own safety limits. Obviously there are exceptions, such as when there are looks rocks, delicate formations, or SRT which should not be rushed in my opinion (I am referring to technical parts like getting on/off the rope, re-belays etc, I am not referring to prussicing as it's not dangerous to do that at high speed just tiring).