Author Topic: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire  (Read 4590 times)

Offline blhall195

  • Brendan Hall
  • player
  • **
  • Posts: 100
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #75 on: November 04, 2020, 09:17:49 pm »
Hi Andy, yep I'm really interested to see just how quickly these flood events happen, I often see foam on the ceiling in wreached rabbit near the streamway after heavy rain and wonder what it would have been like a few hours earlier.

Project update:

I've applied for BCRA funding, hopefully I'll hear back around/soon after Christmas.

I've decided to try make the logger enclosure out of a short length of 2" PVC pipe with plastic welded screw cap fittings on either end, the sensor/display will be epoxided onto the outside hopefully giving a good waterproof seal.

I'll design a PCB to fit inside the pipe and maybe 3D print a holder for the PCB so it fits snug inside the pipe.

I've programmed the logger to shutdown peripheral modules when they're not in use. I'm considering adding some more circuitry to better regulate the power supply to the modules and improve the overall efficiency of the device during sleep mode, hopefully I don't go too far down a rabbit hole there.

If anyone's any good at python or matlab and fancies helping me convert this radar data into a more useable format give me a shout, there's a few example scripts on GitHub but I can't seem to get any of them to work.
« Last Edit: November 04, 2020, 09:33:46 pm by blhall195 »

Offline andrewmc

  • BCA ind. rep.
  • junky
  • ****
  • Posts: 940
  • EUSS, BEC, YSS, SWCC...
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #76 on: November 05, 2020, 10:21:27 am »
If anyone's any good at python or matlab and fancies helping me convert this radar data into a more useable format give me a shout, there's a few example scripts on GitHub but I can't seem to get any of them to work.

What sort of format is it currently in, and what sort of format would you like it in?

Offline blhall195

  • Brendan Hall
  • player
  • **
  • Posts: 100
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #77 on: November 05, 2020, 11:11:07 am »
It's not in a file format I've seen before apparently it's in a weird format unique to NIMROD, if you pm me your email I could send you an example file. I'd like it in a way I can interpret, maybe an array of numbers where the position of the numbers corresponds to their location and the magnitude equal to the mm of rain.

Offline mountainpenguin

  • regular
  • *
  • Posts: 63
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #78 on: November 06, 2020, 09:52:01 am »
Been following this with some interest.
Have you considered using https://www.influxdata.com/. You could put it on an AWS instance and allow read only access to all (a good use of a BCA grant)
combined with grafana and pretty UIs there is also the Kapacitor part of the stack for processing.
This approach would also let you farm off things like the radar data to others who could feed into the influxdb

Offline andrewmc

  • BCA ind. rep.
  • junky
  • ****
  • Posts: 940
  • EUSS, BEC, YSS, SWCC...
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #79 on: November 06, 2020, 12:35:54 pm »
It's not in a file format I've seen before apparently it's in a weird format unique to NIMROD, if you pm me your email I could send you an example file. I'd like it in a way I can interpret, maybe an array of numbers where the position of the numbers corresponds to their location and the magnitude equal to the mm of rain.

Having had a look at the NIMROD data website thing, it looks like a custom binary format out of Fortran (which I used to do a lot of stuff in) but would take a bit of time to write a reader for (so may not get a chance). The fact that it came out of Fortran doesn't really matter in terms of reading it; you should be able to do that with anything.

Offline aardgoose

  • player
  • **
  • Posts: 77
  • DCA DCRO
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #80 on: November 06, 2020, 03:10:57 pm »
It looks like they may have done the job for you already with the NIMROD data.

They have provided some reading routines in python among others for the data which looks like it includes modelled rain fall.

http://data.ceda.ac.uk/badc/ukmo-nimrod/software/python

One of the scripts produces a 'standard'  ASCII .asc raster file.

Offline blhall195

  • Brendan Hall
  • player
  • **
  • Posts: 100
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #81 on: November 06, 2020, 03:45:40 pm »
It looks like they may have done the job for you already with the NIMROD data.

They have provided some reading routines in python among others for the data which looks like it includes modelled rain fall.

http://data.ceda.ac.uk/badc/ukmo-nimrod/software/python

One of the scripts produces a 'standard'  ASCII .asc raster file.

Yep when I tried using those scripts though they didn't appear to work. I half managed to get the Matlab one to work with a few tweaks but I couldn't produce anything of use.

Offline aardgoose

  • player
  • **
  • Posts: 77
  • DCA DCRO
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #82 on: November 06, 2020, 03:53:24 pm »

A shame, although they probably provide a starting point.   

Offline blhall195

  • Brendan Hall
  • player
  • **
  • Posts: 100
Cave Logger Mk1 design
« Reply #83 on: November 10, 2020, 09:44:10 pm »
Cave Logger Mk1

It's been great to see so many people taking an interest in the project and the time to offer useful advice/ideas  :bow:

Now I've had some time to consider how to make the cave flood event monitoring device.

I've designed a prototype and come up with a list of attributes I want it to have...

Cave logger attributes

A waterproof ultrasonic or water pressure-based water level logging device for flood event monitoring, ultrasonic sensor range 6m +/- 2cm, pressure sensor range 250m water depth +/- 1cm (depending on how waterproof the housing is), still I need to test the devices before I can comment on how accurate they really are.

Flexible: Many Arduino compatible sensors can be plugged into the logger, multiple sensors can be placed on a single logger.

Easily to assemble: To overcome the tedious and annoying process of wiring the cave logger together I've designed a printed circuit board, simply solder on the components and it's good to go.

Small: A small device footprint has many advantages, the device is less likely to be noticed by cavers who might try to open it out of curiosity, remove it after mistaking it for rubbish or be offended by its presence. It also means it's easier to carry in and out of the cave. The existing design is a 6" length of grey 2" PVC pipe capped at both ends with plastic welded PVC screw cap fittings.

Cheap: The logger and housing should cost less than £50 to make, the other additional costs being sensors.

Logger + housing component list

Arduino pro mini: £2.00
SD card module: £0.60
RTC module: £1.00
Display module: £1.70
TTL adapter (PC interface): £2.00
3.3V regulator: £4.00
5V regulator: £4.00
PCB: £3.00
battery case: £1.80
transistors and resistors: £1.00
SD card: £2.00
switch: £0.10
power in: £0.10
2" PVC pipe section: £5.00
end cap adapters: £8.40
screw caps: £6.60
4xAA batteries: £4
Total: £48.30

By keeping costs low I should be able to afford more sensors and gather more data for the project, also means I'm not too heart broken if my devices die.
To save on shipping costs I bought multiple components in bulk, also ordered electronic components from china via snail mail.
The list doesn't include necessary tools and consumables such as glue, solder, soldering iron ect. 

Efficeient: the device almost completely shuts itself down between readings, using less than 1 mA of power when asleep, I can probably improve this further in the next design iteration. 

Simple: I'm sure there are many improvements I could make to the device to further improve its efficiency and lower it's cost but in the interest of saving time and my own sanity, I've opted for a more lego based approach where I exploit the efforts of other people. I have designed a device made up of a few cheap and readily available components. Libraries of code already exist and the Arduino IDE makes programming the device very easy.   

Rugged: the device housing will be constructed out of thick 10 bar pressure rated 2" PVC piping https://www.irrigationonline.co.uk/products/PVC-Imperial-Adaptor-Socket-to-Male-Fitting.html.
I'm hoping it will be able to withstand a decent amount of water pressure if submerged. I plan to have sensors sticking out of the housing like this (epoxy glue):




Current progress

I've finally finished designing the circuit design for the logger, I've sent it off to the PCB print press and it should arrive sometime next week.

The PCB preview is shown in the image below, component breakdown:

1. 3.3V buck regulator: I plan to use 4 AA batteries as these will fit nicely in a 2" PVC pipe, this step-down regulator turns 5-6V into 3.3V and should be more efficient than a linear regulator as power isn't wasted as heat, https://cdn-shop.adafruit.com/product-files/2745/P2745_Datasheet.pdf these regulators apparently have a quiescent current draw of 0.16 mA.

2. 5V buck regulator: Same thing again but for 5V, this is for powering an optional 5V ultrasonic sensor, the device will work without this voltage regulator

3. PC interface: port for a USB serial TTL adapter to interface the device with a PC for programming the Arduino.

4. Arduino pro mini: a cheap low power microcontroller

5. LED: pins for optional LED output, can be programmed to let the user know the status of the logger, blinking = recording, constantly on = powered but in an error state. Programmed to turn off after a set period to save power.

6. OLED display: useful if setting up an ultrasonic based water level sensor, this displays the time/date and data read from the sensor, the display automatically shuts off after a set period to save power.

7/8. I2C ports: up to two I2C devices can be attached directly to the board, more if you want to wire them on.

9. Realtime clock (RTC) module: a very accurate temperature regulated realtime clock module for timestamping data.

10. Transistors and resistors: these are so all of the modules can be turned off when the Arduino is in sleep mode to save power.

11. SD card module: allows data to be saved to an SD card.

12. Switch: an optional on/off switch can be placed here.

13. power pins/screws: leads to a battery pack with 4 AA batteries, I was originally going to use a 3.7V lithium-ion cell but decided it'd be safer to use AA's.

I've made the PCB Gerber files and Arduino code available on GitHub, I used the free PCB software EasyEDA for the circuit board design and ordered the boards from JCBPCB:
https://github.com/blhall195/cave_logger

PCB layout


Circuit design

« Last Edit: November 10, 2020, 10:04:41 pm by blhall195 »

Offline Chocolate fireguard

  • obsessive maniac
  • ***
  • Posts: 433
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #84 on: November 11, 2020, 09:05:08 am »
You have obviously put in a lot of thought and work on this - it looks good.

I still struggle with the idea that nowadays stuff that does such impressive things can cost the same as a short length of pipe, a couple of end fittings and a few batteries!

Online mikem

  • forum hero
  • *****
  • Posts: 3969
  • Mendip Caving Group
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #85 on: November 11, 2020, 09:35:27 am »
The data logger needs to be positioned in a free flowing section of streamway (preferably of relatively even width) to give easily interpreted results. If it's above a sump then the rise in level will not be directly proportional to the amount of rain that has fallen. However, a stream creates much greater pressures than a pool, so sump may be better for testing.

Offline blhall195

  • Brendan Hall
  • player
  • **
  • Posts: 100
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #86 on: November 13, 2020, 09:04:46 pm »
The data logger needs to be positioned in a free-flowing section of streamway (preferably of relatively even width) to give easily interpreted results. If it's above a sump then the rise in level will not be directly proportional to the amount of rain that has fallen. However, a stream creates much greater pressures than a pool, so sump may be better for testing.

That's a good point, I didn't consider that the movement of water might affect the pressure readings. I suppose I should do some experiments to work out how much of an effect this has, I've been talking with Nick Bairstow about the possibility of placing them in sections of a cave which become completely flooded, I'm thinking the water might be more static in a completely submerged cave passage.


Offline blhall195

  • Brendan Hall
  • player
  • **
  • Posts: 100
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #87 on: November 13, 2020, 09:05:48 pm »
Today the PVC pipe and fittings arrived, I plan to use another pipe cut in half length ways just big enough to fit into this pipe, and mount my electronics on that to stop them rattling around.



Offline Bob Mehew

  • forum hero
  • *****
  • Posts: 1380
  • breaking knots is fun
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #88 on: November 13, 2020, 09:34:11 pm »
The data logger needs to be positioned in a free-flowing section of streamway (preferably of relatively even width) to give easily interpreted results. If it's above a sump then the rise in level will not be directly proportional to the amount of rain that has fallen. However, a stream creates much greater pressures than a pool, so sump may be better for testing.
That's a good point, I didn't consider that the movement of water might affect the pressure readings. I suppose I should do some experiments to work out how much of an effect this has, I've been talking with Nick Bairstow about the possibility of placing them in sections of a cave which become completely flooded, I'm thinking the water might be more static in a completely submerged cave passage.
I don't think there is any easy link between water level in cave and rain fall because there will be several modes of the water getting to the sensor (percolation as well as direct stream flow).  And if stream flow is made up of several streams, then it will get even more complex. 

The pressure effect due to stream flow could be checked by a simple experiment of dragging the sensor through water at different speeds.  It won't be static unless you go for a sump pool with small inlet and outlet.

Offline Chocolate fireguard

  • obsessive maniac
  • ***
  • Posts: 433
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #89 on: November 15, 2020, 07:23:55 pm »
The data logger needs to be positioned in a free-flowing section of streamway (preferably of relatively even width) to give easily interpreted results. If it's above a sump then the rise in level will not be directly proportional to the amount of rain that has fallen. However, a stream creates much greater pressures than a pool, so sump may be better for testing.

That's a good point, I didn't consider that the movement of water might affect the pressure readings. I suppose I should do some experiments to work out how much of an effect this has, I've been talking with Nick Bairstow about the possibility of placing them in sections of a cave which become completely flooded, I'm thinking the water might be more static in a completely submerged cave passage.


The dynamic pressure produced by a moving liquid is equal to the kinetic energy of unit volume of the liquid, i.e. of 1000kg in the case of water.
If the water were moving at 0.1m/s the pressure would be 0.5*1000*0.1*0.1 = 5N/m^2.
This is 0.00005 atmospheres, so the depth error would be trivial at 0.5mm.
The KE is proportional to the speed squared, so at 1m/s the error would be 50mm.
Is it possible that with 2 pressure sensors at right angles you could get information on the speed of the water as well as its depth? Over a short time the effect of turbulence might average out to give an accurate answer.

Offline blhall195

  • Brendan Hall
  • player
  • **
  • Posts: 100
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #90 on: November 15, 2020, 07:26:04 pm »
The data logger needs to be positioned in a free-flowing section of streamway (preferably of relatively even width) to give easily interpreted results. If it's above a sump then the rise in level will not be directly proportional to the amount of rain that has fallen. However, a stream creates much greater pressures than a pool, so sump may be better for testing.

That's a good point, I didn't consider that the movement of water might affect the pressure readings. I suppose I should do some experiments to work out how much of an effect this has, I've been talking with Nick Bairstow about the possibility of placing them in sections of a cave which become completely flooded, I'm thinking the water might be more static in a completely submerged cave passage.


The dynamic pressure produced by a moving liquid is equal to the kinetic energy of unit volume of the liquid, i.e. of 1000kg in the case of water.
If the water were moving at 0.1m/s the pressure would be 0.5*1000*0.1*0.1 = 5N/m^2.
This is 0.00005 atmospheres, so the depth error would be trivial at 0.5mm.
The KE is proportional to the speed squared, so at 1m/s the error would be 50mm.
Is it possible that with 2 pressure sensors at right angles you could get information on the speed of the water as well as its depth? Over a short time the effect of turbulence might average out to give an accurate answer.


Could I get around the issue of turbulence by using a mini stilling well or something?

Offline Bob Mehew

  • forum hero
  • *****
  • Posts: 1380
  • breaking knots is fun
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #91 on: November 15, 2020, 08:07:11 pm »
Could I get around the issue of turbulence by using a mini stilling well or something?
How about just fitting a blanking plate with a small hole over the front of the sensor housing so it can't see the direct stream.  You could check on the venturi effect by the wafting it through water.

Offline Chocolate fireguard

  • obsessive maniac
  • ***
  • Posts: 433
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #92 on: November 15, 2020, 09:32:15 pm »
I had to look up stilling well!
Nice simple idea that should leave your sensor free to just register static pressure.

Online mikem

  • forum hero
  • *****
  • Posts: 3969
  • Mendip Caving Group
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #93 on: November 15, 2020, 09:37:15 pm »
Even a sensor at 90 degrees will still be subject to depth pressure.

Offline blhall195

  • Brendan Hall
  • player
  • **
  • Posts: 100
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #94 on: November 17, 2020, 08:17:33 pm »
PCB arrived today  ;D

I now have a slightly more condensed and robust version of the logger.

Works much better than the breadboarded version, I can pick it up without it falling appart. I'm still waiting on a few of the components to arrive so not quite finished yet. I might redesign the PCB too after noticing a few ways it can be improved.



Did some back of the envelope maths to estimate the logger battery life when I change the sample rate and improve the sleep state current draw.

I'll probably take a sample every 5 minutes so it's probably worth looking at improving sleep current, currently looking at integrating one of these modules https://www.adafruit.com/product/3435 

into the device. It's not too expensive and basically completely shuts off the device between samples.


Online mikem

  • forum hero
  • *****
  • Posts: 3969
  • Mendip Caving Group
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #95 on: November 18, 2020, 09:49:19 am »
EA are already doing predictions for some rivers, e.g. Trent:



https://www.ukriversguidebook.co.uk/forum/viewtopic.php?f=3&t=128160
« Last Edit: November 18, 2020, 01:11:58 pm by paul, Reason: Link modified at poster\'s request. »

Offline blhall195

  • Brendan Hall
  • player
  • **
  • Posts: 100
Re: Is it safe to go caving yet? Modelling cave flood events in Yorkshishire
« Reply #96 on: November 21, 2020, 07:25:09 pm »
EA are already doing predictions for some rivers, e.g. Trent:

https://www.ukriversguidebook.co.uk/forum/viewtopic.php?f=3&t=128160

Good to see the kayakers share my pain.

New PCB design is complete, also made a logo.



 

Main Menu

Forum Home Help Search