Author Topic: 3D surveys in a web browser - now with satellite overlays  (Read 1618 times)

Offline aardgoose

  • regular
  • *
  • Posts: 42
  • DCA DCRO
3D surveys in a web browser - now with satellite overlays
« on: July 26, 2017, 08:46:59 pm »


Latest update to my web 3d cave viewer available at the link below. A recent addition is the ability to overlay images from Bing, Open Street Maps etc over terrain to allow views as in the attached image.

https://aardgoose.github.io/CaveView.js/ ( for an automatic demo )
https://aardgoose.github.io/CaveView.js/fulldemo.html (  several UK Survex and Therion models to play with )

Offline dave_the_cave

  • obsessive maniac
  • ***
  • Posts: 410
    • dave_the_cave's blog
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #1 on: July 26, 2017, 09:35:49 pm »
Fantastic - Can you explain how you did it

I found the hint "Requires a browser supporting WebGL (IE 11+ and most other recent browsers), no plugins required. Created using the THREE.js 3D library and chroma,js colour handling library."

Can you say more


Offline Swallowneck

  • addict
  • **
  • Posts: 120
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #2 on: July 26, 2017, 09:53:27 pm »
Liking this a lot, thanks for sharing.

Offline aardgoose

  • regular
  • *
  • Posts: 42
  • DCA DCRO
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #3 on: July 27, 2017, 11:49:10 am »
dave_the_cave

v brief answer : a lot of code.

brief answer:

From the bottom:

Most PC's/phones/tablets etc have 3d graphics processors of varying capability, This functionality usually exposed through 1 of 2 interface specs. (OpenGL & D3D (the microsoft proprietary version)).

Modern web browsers support 3d graphics through a spec 'WebGL 1.0', which supports a limited subset of OpenGL (so it can run on most phones on the market at the time the specification was defined). This allows Javascript programs to access high performance graphics, although with significant overhead compared with native applications. https://www.khronos.org/webgl/


WebGL isn't a friendly spec to use, so various code libraries have been created to make things more user/programmer friendly. Three.js is one of the leading libraries. It provides functions to do all the matrix and vector maths required, and provides mechanisms to assemble scenes and send the information to the graphics processor, as well as 3d primitives like spheres, boxes etc. https://threejs.org/ - lots of examples there which demonstrate the capabilities of three.js.

Incidentally Aven and Loch use similar C/C++ libraries to use OpenGL.


Iv'e written code to parse Survex .3d and Therion .lox files ( The .lox file isn't documented so I had to read the code, but it's a lot more straightforward than the .3d format), and transform the survey lines, LRUD walls and scraps for (.lox) files into lines and triangles required to create the scenes.  Getting the basic lines visible is quite simple, making it efficient and fast and avoiding crashing the browser  when using large surveys has taken a bit more work.


The Therion .lox files can contain terrains, Survex doesn't.  To put terrains on .3d surveys, I have created terrain tiles from various public sources ( SRTM mapping, Ordnance Survey 1:50k mapping, Environment Agency high res lidar data (I use the GRASS GIS https://grass.osgeo.org/ program to do this). By using the same tile sizes and locations as used by Bing/Google etc, I can then put various imagery over the terrain.  This is only possible with Survex surveys that specify the coordinate system in use using *CS OUT directives. I use another library proj4js to handle transformation between coordinate systems ( OSGB36 -> WGS84 etc).  Survex uses a related library proj4.


I'm looking at adding lines to indicate dye traces etc (obviously the real paths aren't known) - actually the code that does that is already there, it just needs a user interface, and improving tiling etc.  I would like to add the BGS 1:50k mapping, but currently it is only available in a form that browsers won't pass to the 3d functions.  Any features you would like to see?  There is a lot of open data out there like SSSI boundaries, Scheduled Monuments etc that could be used to make custom imagery layers.


That's a brief summary, lots of detail missed out - mainly performance related. The chroma.js library was used to generate smooth colour gradients. This is my first 3d graphics program so there has been a lot of trial and error.
















Offline marysboy

  • obsessive maniac
  • ***
  • Posts: 341
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #4 on: July 27, 2017, 12:10:11 pm »
wow  :beer2:

Offline Martin Laverty

  • menacing presence
  • **
  • Posts: 196
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #5 on: July 27, 2017, 04:01:36 pm »
Excellent work, and (as you said before) anyone with a .3d or .lox file can display it using drag and drop...

I had a play with a different approach last year by using Cesium (which is replacing Google Earth in some ways) to display the Ario caves in Spain in 3D on Bing imagery  - http://ariocavesproject.com/SBG/cesTst.htm

Cesium also builds on WebGL, provides its own (SRTM-derived), or other, terrain data, and has excellent support for WMS (Web Map Service) and similar tile providers....

Offline aardgoose

  • regular
  • *
  • Posts: 42
  • DCA DCRO
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #6 on: July 27, 2017, 08:27:17 pm »

Martin, thanks,

I discovered Cesium after I had the basic things working, and from what I remember was orientated to surface features rather than subsurface, in your demo are the cave passages rendered above the terrain? Obviously Cesium is a more professional code base and will have fewer bugs in it.

One of the big problems any 3d engine has to deal with it running out of Javascript heap or GPU memory, because Javascript doesn't provide any mechanisms* to detect either issue. *By design, to prevent device fingerprinting. I've contributed a few features to three,js to reduce memory consumption.

Somebody in the other SUSS has also been using three.js to render surveys - https://suss.caves.org.au/cavemodels.html. They appear to be using the JSON format output from Survex,which is actually where I started, before tackling the .3d binary format (smaller file size and more information).

Cheers



Offline wiltonp

  • player
  • **
  • Posts: 93
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #7 on: July 27, 2017, 10:34:05 pm »
This is really good by am I missing something?
I have coppied you entire page, built my site using your layout used the zip file but when I view my resulting page,I see my survey but with no UI (left menu or right depth legend) yet when I look at yours, it exists.
I've tried in latest chrome and IE.

Offline wiltonp

  • player
  • **
  • Posts: 93
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #8 on: July 27, 2017, 11:00:56 pm »
I also note that I get a 404 error saying GET xxxxxxxxx/Giants_Oxlow_Maskhill_System.json 404 (Not Found) This could be the issue. How would I generate these for my 3d files?

Offline Martin Laverty

  • menacing presence
  • **
  • Posts: 196
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #9 on: July 28, 2017, 01:18:06 am »
aardgoose: yes, the passages appear to be being drawn in 3d after the terrain has been rendered: this should probably not be allowed, so shouldn't be relied on although it does what we want!

I  appear to be exploiting a loophole in the Cesium viewer code which manifests when no 'baselayer' is specified. If a 'baselayer' is specified, the passages are (correctly, in general) hidden by surface culling,transparency not being allowed in a 'baselayer'. This is what prevented Google Earth from being used to display subsurface features too...

Incidentally, the survey data is a .kml file generated by therion, with subsequent editing to specify altitudeMode as absolute. KML is more widely accepted by GIS software than the cave-specific .3d or .lox (and can contain comments, always a useful attribute to have available, and notably excluded from .json as you say is used by the antipodean SUSS), so might be worth considering as an additional format for your application to recognise and load...

Offline aardgoose

  • regular
  • *
  • Posts: 42
  • DCA DCRO
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #10 on: July 28, 2017, 12:14:41 pm »
wiltonp


You have hit a bug, my local setup always uses loadCaveList(), so I missed a bug that crept into loadCave().

Now fixed with 1.1.1

Thanks

The error loading a <cave name>.json can be ignored.  The .json file is optional, and allows adding routes through a survey and eventually lines indicating hypothetical dye trace connections.

Offline wiltonp

  • player
  • **
  • Posts: 93
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #11 on: July 28, 2017, 01:07:24 pm »
Thanks I'll give that a go. I tried with both and the list one was much mite reliable. I've also found another possible bug. I'll send you a 3d file later which fails... I have a couple of them that fail so it is provably something to do with the cavern process as it is done with an older version of survex.

Offline aardgoose

  • regular
  • *
  • Posts: 42
  • DCA DCRO
Re: 3D surveys in a web browser - now with satellite overlays
« Reply #12 on: July 28, 2017, 04:25:13 pm »
wiltonp


I'm sure there are bugs! Most survex files i've tested have been the later version of .3d (Survex has produced various formats over the years) If you can send a failing .3d file, I can take a look. 

Cheers