Is it safe to go caving yet? Modelling cave flood events in Yorkshishire

[Hall2501]

Hi Tony, yep to give you credit it did steal your ultra sonic sensor idea. I was going to use a pressure sensor and put the logging device under water, which would mean I could put it in more interesting places but the risk of damage/loss was too much for me to bear.



Sent from my AC2003 using Tapatalk

 

[Hall2501]

Just watching Andrew Smith talking about the cave science research center in Poole cavern now, it looks like it might be a good place to do an bench mark test my loggers.



Sent from my AC2003 using Tapatalk

 

[Edwardov]

Hello all,

I stumbled across this thread after I had an idea a few days ago for using AI to monitor environmental conditions in underground spaces, namely mine air quality, but I equally thought that it could be applied to caves.

I am a data scientist in my day job where I used AI to solve a wide variety of problems and I have been a caver for several years now, so it made sense!

A great place for global hourly weather data is here https://rda.ucar.edu/datasets/ds094.1/

I too have come up against the problem of having insufficient data of scenario (external conditions) and outcome (cave water depth/air quality) for time-dependent supervised machine learning, but I have found this additional source which got me interested in finding equivalent data for UK caves https://carrollcave.org/?p=3328.

If you want to talk about the AI side of things or if you have any questions, let me know and I'll happily help you out in exchange for some cave data.

Cheers

 

[AR]

I'd suggest getting in touch with the BCRA, they've recently set up a permanent logging station in Poole's Cavern, Buxton. I can't remember offhand all of what's being monitored but they are keen to increase usage of the data. I think you can also get datasets from the nearby Harpur Hill weather station.

 

[Bob Mehew]

I'd suggest getting in touch with the BCRA, they've recently set up a permanent logging station in Poole's Cavern, Buxton. I can't remember offhand all of what's being monitored but they are keen to increase usage of the data. I think you can also get datasets from the nearby Harpur Hill weather station.

The web site is at https://www.cave-science.org.uk/.  I also have some CO2 data for another cave which is puzzling.  PM me if you want more info.

 

[Edwardov]

I'd suggest getting in touch with the BCRA, they've recently set up a permanent logging station in Poole's Cavern, Buxton. I can't remember offhand all of what's being monitored but they are keen to increase usage of the data. I think you can also get datasets from the nearby Harpur Hill weather station.

The web site is at https://www.cave-science.org.uk/.  I also have some CO2 data for another cave which is puzzling.  PM me if you want more info.

Thanks for the info both. The CO2 and meteorological data for Poole's Cavern look really interesting and I can add additional data (wind etc), but this is missing visitor data to really understand the relationship as clearly this will have an effect on the CO2 levels. Any idea if such data exist?

 

[Bob Mehew]

... this is missing visitor data to really understand the relationship as clearly this will have an effect on the CO2 levels. Any idea if such data exist?

There is data from the early part of this year when no visitors went in due to the lock down.  Visitor numbers are probably tracked by the Buxton Civic Association but I don't know if they release them.  That will require some negotiation via the BCSC team.  See my PM. 

 

[Hall2501]

Hey Edwardov,

Yep, any help with the data interpretation would be great Ari Cooper Davis has expressed interest in helping with this also. I'm far more comfortable with the hardware side of things. And even then for this I'm merely modifying existing logger designs https://thecavepearlproject.org/how-to-build-an-arduino-data-logger/ with my primary motivation being to put together cheap and accurate enough loggers to get a better idea of how popular dales caves flood after it rains.

Quick update: I've now changed the design/code so I can turn the sensors and screen off when they're not needed to save power.

And done some experiments to see how efficient it is (see graph below), the bottom axis is about 20 seconds long*, I'm using a INA219 module to monitor current and voltage, the voltage holds steady at 3.3V while the current changes depending on what the device is doing. The device is in sleep mode by default, the plot shows 2 logging events, each event has 2 current peaks, the first is the sensor activating and taking a reading, the second is the SD card writing to file before going back to sleep I think.

In sleep mode it averages a power consumption of around 3 mA with 2 18650 cells so in theory should last around 2000 hours depending on the logging frequency.

 

[mikem]

Just to illustrate the problem of using rain gauge data -  in last 24 hours dartcom (at Princetown) recorded 40mm of rain, haytor is less than 15 miles away (still on the top of dartmoor, but in the rain shadow), but only got 10mm. Gauges on all sides of the moor (at lower altitudes) recorded no more than 5mm.

 

[Hall2501]

Just to illustrate the problem of using rain gauge data -  in last 24 hours dartcom (at Princetown) recorded 40mm of rain, haytor is less than 15 miles away (still on the top of dartmoor, but in the rain shadow), but only got 10mm. Gauges on all sides of the moor (at lower altitudes) recorded no more than 5mm.

I see, I've just applied for access to the Met Office's Nimrod System which apparently provides 1 km Resolution UK Rainfall Data from radar stations. There's almost certainly limitations to obtaining data in this way too but we'll see what it looks like.

https://catalogue.ceda.ac.uk/uuid/27dd6ffba67f667a18c62de5c3456350

I was going to get radar data from the Met Office DataPoint system but it looks like that is being discontinued soon. 

 

[mikem]

The two I mentioned are the only ones I know in UK that are actually on hilltops, whereas majority are in the surrounding valleys. It's quite normal for rainfall at Princetown to be twice that at Haytor, but on this occasion, it was four times & that was not because the storm missed that point, it was just heavier on the top (prevailing weather side) of the moor.

 

[Hall2501]

The two I mentioned are the only ones I know in UK that are actually on hilltops, whereas majority are in the surrounding valleys. It's quite normal for rainfall at Princetown to be twice that at Haytor, but on this occasion, it was four times & that was not because the storm missed that point, it was just heavier on the top (prevailing weather side) of the moor.

Kind of like this?

The good news is that this morning the MetOffice approved my application to access NIMROD high-resolution radar data until November 2023 when I will have to reapply, so will be interesting to see if this effect shows up on radar too.

 

[mikem]

It would be very interesting to see how the radar data for Princetown compares to what's actually recorded on dartcom, as a test of its accuracy. Is it only future events you'll have access to, or past as well? As there's not much rain forecast in immediate future (down here anyway).

 

[Andy Walsh]

note
Flooding can be delayed by 2 or more hours
See the Cave Explorers by Jim Eyres Pages 41-45 --A very violent thunderstorm + heavy rain before breakfast and a trip to a dry Easegill which flooded violently while they were underground a near miss
Also on page 45 he says that dry ground does not soak up water nor as he says does even dry sand-and it runs off if the ground is saturated so the most dangerous is after a prolonged drought or after prolonged drought A sudden thaw after a lot of snow on wet ground is impressive.Kingsdaleis quite good for delayed floods
Some places like Sleets Gill have a longish delay mainly due to delay supply from perculationwater supply

 

[Hall2501]

The radar data is past/present only, dates back almost 10 years with 5 min time resolution. Currently having trouble interpreting the files though, they're in a proprietary format that's hard to convert into something I can work with.



Sent from my AC2003 using Tapatalk

 

[Hall2501]

Hi Andy, yep I'm really interested to see just how quickly these flood events happen, I often see foam on the ceiling in wreached rabbit near the streamway after heavy rain and wonder what it would have been like a few hours earlier.

Project update:

I've applied for BCRA funding, hopefully I'll hear back around/soon after Christmas.

I've decided to try make the logger enclosure out of a short length of 2" PVC pipe with plastic welded screw cap fittings on either end, the sensor/display will be epoxided onto the outside hopefully giving a good waterproof seal.

I'll design a PCB to fit inside the pipe and maybe 3D print a holder for the PCB so it fits snug inside the pipe.

I've programmed the logger to shutdown peripheral modules when they're not in use. I'm considering adding some more circuitry to better regulate the power supply to the modules and improve the overall efficiency of the device during sleep mode, hopefully I don't go too far down a rabbit hole there.

If anyone's any good at python or matlab and fancies helping me convert this radar data into a more useable format give me a shout, there's a few example scripts on GitHub but I can't seem to get any of them to work.

 

[andrewmcleod]

If anyone's any good at python or matlab and fancies helping me convert this radar data into a more useable format give me a shout, there's a few example scripts on GitHub but I can't seem to get any of them to work.

What sort of format is it currently in, and what sort of format would you like it in?

 

[Hall2501]

It's not in a file format I've seen before apparently it's in a weird format unique to NIMROD, if you pm me your email I could send you an example file. I'd like it in a way I can interpret, maybe an array of numbers where the position of the numbers corresponds to their location and the magnitude equal to the mm of rain.

 

[mountainpenguin]

Been following this with some interest.
Have you considered using https://www.influxdata.com/. You could put it on an AWS instance and allow read only access to all (a good use of a BCA grant)
combined with grafana and pretty UIs there is also the Kapacitor part of the stack for processing.
This approach would also let you farm off things like the radar data to others who could feed into the influxdb

 

[andrewmcleod]

It's not in a file format I've seen before apparently it's in a weird format unique to NIMROD, if you pm me your email I could send you an example file. I'd like it in a way I can interpret, maybe an array of numbers where the position of the numbers corresponds to their location and the magnitude equal to the mm of rain.

Having had a look at the NIMROD data website thing, it looks like a custom binary format out of Fortran (which I used to do a lot of stuff in) but would take a bit of time to write a reader for (so may not get a chance). The fact that it came out of Fortran doesn't really matter in terms of reading it; you should be able to do that with anything.