This is a showcase example of using AI to predict interesting things related to caves. I'm working on a few other projects but I thought I would share this one.
I have trained an AI to study weather and temporal patterns to predict CO2 levels inside Poole's Cavern. The CO2 data came from the BCSC, and I extracted hourly weather data from a numerical model output (https://rda.ucar.edu/datasets/ds094.1/) which matched the BCSC surface weather data but had better coverage throughout the timeseries and extended to the present. The entire dataset was resampled at 10 minute resolution to match the cave logger data which resulted in the interpolation of the hourly weather data.
Essentially I matched data of the scenario (external weather data and temporal patterns) to the result (cave CO2 data), and trained the AI to learn a function relating them. When tested, the trained AI performed very well when trying to predict the CO2 levels from some sample October 2018-October 2019 data that I held back which I obviously knew the CO2 levels for. I then trained the AI on the entire October 2018-October 2019 dataset and predicted CO2 levels for the time period of November 2019-present for which I had the required weather data. So see this as a prediction as to what the CO2 dataset might be like when it gets release by the BCSC (they are currently working on this). The algorithm performed on the test set with a mean squared error of 445, which is +- 21, quite good!
Though this is assuming that the underlying relationship between external weather and internal cave CO2 levels is the same for November 2019-present as it is in the data from October 2018-October 2019 from which it learned. Another assumption is that external weather and temporal patterns are the best predictors of CO2 data for this problem.
I used the following parameters to feed into the algorithm:
atmospheric pressure
wind speed and direction
dew point temperature
temperature
soil moisture content
precipitation
relative humidity
sensible and latent heat flux
hour of day
day of week
week of year
month of year
Just remember that while I can predict far into the future, things will have obviously changed from 2019 to 2020 because of no visitors being present in the showcave and whatever else might have changed that affects CO2. So I would expect my prediction to fall over at some point in the future unless it has been retrained on recent data to learn any changes.
This is just an example of what you can do with AI and some data engineering in Python. I used this sort of stuff in my day job to solve all sorts of problems, so I thought I'd give it a go for caves. If anyone has any interesting problems they have, I'd love to take a look. Or if anyone has any questions just give me a shout.
Cheers
Ed
Training CO2 data
Predicted Future CO2
I have trained an AI to study weather and temporal patterns to predict CO2 levels inside Poole's Cavern. The CO2 data came from the BCSC, and I extracted hourly weather data from a numerical model output (https://rda.ucar.edu/datasets/ds094.1/) which matched the BCSC surface weather data but had better coverage throughout the timeseries and extended to the present. The entire dataset was resampled at 10 minute resolution to match the cave logger data which resulted in the interpolation of the hourly weather data.
Essentially I matched data of the scenario (external weather data and temporal patterns) to the result (cave CO2 data), and trained the AI to learn a function relating them. When tested, the trained AI performed very well when trying to predict the CO2 levels from some sample October 2018-October 2019 data that I held back which I obviously knew the CO2 levels for. I then trained the AI on the entire October 2018-October 2019 dataset and predicted CO2 levels for the time period of November 2019-present for which I had the required weather data. So see this as a prediction as to what the CO2 dataset might be like when it gets release by the BCSC (they are currently working on this). The algorithm performed on the test set with a mean squared error of 445, which is +- 21, quite good!
Though this is assuming that the underlying relationship between external weather and internal cave CO2 levels is the same for November 2019-present as it is in the data from October 2018-October 2019 from which it learned. Another assumption is that external weather and temporal patterns are the best predictors of CO2 data for this problem.
I used the following parameters to feed into the algorithm:
atmospheric pressure
wind speed and direction
dew point temperature
temperature
soil moisture content
precipitation
relative humidity
sensible and latent heat flux
hour of day
day of week
week of year
month of year
Just remember that while I can predict far into the future, things will have obviously changed from 2019 to 2020 because of no visitors being present in the showcave and whatever else might have changed that affects CO2. So I would expect my prediction to fall over at some point in the future unless it has been retrained on recent data to learn any changes.
This is just an example of what you can do with AI and some data engineering in Python. I used this sort of stuff in my day job to solve all sorts of problems, so I thought I'd give it a go for caves. If anyone has any interesting problems they have, I'd love to take a look. Or if anyone has any questions just give me a shout.
Cheers
Ed
Training CO2 data
Predicted Future CO2