Author Topic: Cave registry GPX/KML/JSON files  (Read 411 times)

Offline Tseralo

  • player
  • **
  • Posts: 97
  • TSG
    • louisemcmahon.co.uk
Cave registry GPX/KML/JSON files
« on: January 14, 2020, 08:26:03 pm »
I've been building a tool for a little while now that converts the various cave registry's in the UK into GIS type files.

Curently the tool outputs
  • KML (for google earth)
  • GPX (for GPS devices)
  • JSON (for programatic use)

It should be trivial to add other file formats, the code is open source and PRs are welcome.

The output can be downloaded here https://github.com/LouiseMcMahon/UKCaveGIS/releases. Download individual files from the latest release not the source code zip. The full list of file types and regions/registys can be found here along with instructions to run the crawler yourself if you wish.

Feedback on the GPX files would be welcome, GPX is a strange format with many versions and I lack many test devices.

Note KML files exist for Yorkshire and the Matt Voysey registry's as documented here but Im not aware of others.

Online Martin Laverty

  • menacing presence
  • **
  • Posts: 207
Re: Cave registry GPX/KML/JSON files
« Reply #1 on: January 15, 2020, 02:31:19 am »
Very interesting, but
1)  might I suggest that you swap the order of the latitude and longitude for the KML - the current order in your sample files moves all UK caves to equatorial climes... GPX identifies lat and lon correctly and order in json is lat,lon: these should be fine, but KML uses lon,lat order in its <coordinates> element [https://developers.google.com/kml/documentation/kmlreference?csw=1#coordinates]
2) most of the json files are incomplete and not properly terminated - I think they must be failing part way through the spidering

Offline Tseralo

  • player
  • **
  • Posts: 97
  • TSG
    • louisemcmahon.co.uk
Re: Cave registry GPX/KML/JSON files
« Reply #2 on: January 15, 2020, 02:39:49 am »
Very interesting, but
1)  might I suggest that you swap the order of the latitude and longitude for the KML - the current order in your sample files moves all UK caves to equatorial climes... GPX identifies lat and lon correctly and order in json is lat,lon: these should be fine, but KML uses lon,lat order in its <coordinates> element [https://developers.google.com/kml/documentation/kmlreference?csw=1#coordinates]
2) most of the json files are incomplete and not properly terminated - I think they must be failing part way through the spidering

Good spot on the KML not sure how I missed that, its almost as if standards exist for a reason :confused:

Not sure whats going on with JSON ill look into it they look fine on my machine might be getting mangled in the upload.

Offline ILT

  • Newbie
  • *
  • Posts: 24
Re: Cave registry GPX/KML/JSON files
« Reply #3 on: January 15, 2020, 10:43:25 am »
Good spot on the KML not sure how I missed that, its almost as if standards exist for a reason :confused:

The world will accept the google standard eventually ;)

Some of the Cave Registry pages (Derbyshire and Mendip I am familiar with) already provide kml download facility - somewhat more useful as the pins will contain basic info and a link to the detail page.

If it is position only data that is to be harvested then why not output to a standard format such as CSV or TSV and then users can convert to other formats easily enough using gpsbabel (or one of the many websites which utilise it's functionality). This has the advantage that it addresses the issue of multiple variants of gpx-xml (where Garmin don't even use a single variant across their devices) as well as allowing people to convert the data for their TomTom, Memory Map etc quite easily?

Offline AlexR

  • Newbie
  • *
  • Posts: 19
  • BPC, TSG
Re: Cave registry GPX/KML/JSON files
« Reply #4 on: January 21, 2020, 10:57:15 am »
Get in there! As a technologically challenged person I tried to do this with gpsbabel a year or two ago and gave up, instead resorting to the extremely efficient method *cough* of manually putting POI's on my gps (so I could check out some esoteric holes on various walks through the Peak).
This is a little better, thanks!  ;D

 

Main Menu

Forum Home Help Search