I thought I'd start a thread specifically for reporting bugs and fixes for the cave survey data converter tool I am developing.

The latest reported bug is potentially quite serious as it can result in errors in the survey data.

When Disto/PocketTopo takes three readings near magnetic north, e.g. 359.8, 359.8, 0.1 the software sees the three as a leg which is OK, but the display on the PDA shows 359.8, 359.8, 0.1. The converter software then averages the three readings to 239.87 - not good

Now fixed. You can download from the usual place:
http://www.darkgem.com/wscc/caving-talk/index.php?title=CaveSurveyDataConverter

Sorry, my fix earlier today just moved the problem to a different compass bearing!  :-[

I've fixed it properly now. If you downloaded the Cave Converter earlier today then please get the updated copy I have just uploaded. You know you have the proper fix if the JAR file in the download has a time of 22:10 this evening (1st Sept. 2009). Watch out for web browsers which think you already downloaded the file earlier, so just give you the old file again! To force a fresh download, hold the CTRL key when you click on the download link. Then check the file time stamp of the JAR file inside the zip file.

OK - so I now have a folder on my desktop with CaveConverter.jar and the four .bat files in.

my exported .txt file is in the same folder.

I'm running from the command line prompt using the command lines as suggested on Darkgem pages....

java -jar CaveConverter.jar Bagresur1.txt survex.svx p s

and

this is from the output log

2009-09-16 14:33:23: Reading PocketTopo data file: Bagresur1.txt
2009-09-16 14:33:23: IO exception: Bagresur1.txt (The system cannot find the file specified)
2009-09-16 14:33:23: Processed 0 survey legs in 0 series.

Click to expand...

any clues

PS - how do you cut and paste when within the Command Prompt (never was any good at DOS  :).

OK, are you sure your pocket topo txt file in the same folder as the jar file and you typed the name correctly?
The output is telling you that it could not find the file. It will look for it in the folder you are running the command line from.

You can cut and paste in the command prompt using the edit menu on the command prompt window. There are some shortcuts on some versions of Windows too, but they do not work on all. There are:
<ENTER KEY> = copy selected text from command prompt to clipboard.
<Right mouse click> = paste clipboard into command prompt window.

You can also retrieve previously typed commands by pressing the up arrow key if this feature is enabled.

pretty sure it was all in the same directory/spelt right. hmmmm

cheers anyhow fleg - too late now  - will have a go tomorrow

I've found and just fixed a bug in the DXF conversion mode of my cave converter. It was rounding all station positions to the nearest metre during reading of DXF format data, giving rise to significant rounding errors.  :-[

This bug only affected conversions from DXF format to other formats. So thankfully it will not have affected anyone using the converter to get data out of PocketTopo to use in Survex.

I've uploaded a new version to the usual place:
Cave Survey Data Converter

I've just tried out the converter following the wiki instructions (http://ukcaving.com/wiki/index.php/Surveying#Cave_Converter) but got the following error

Daves-iMac:CaveConverter daveclucas$ java -jar CaveSurveyConverter.jar /Users/daveclucas/Documents/caving/surveys/kuching/test.txt test.svx p s
2009-12-10 09:31:28: Reading PocketTopo data file: /Users/daveclucas/Documents/caving/surveys/kuching/test.txt
2009-12-10 09:31:28: Data line    2: 1.0 0.000 0.000 0.000
2009-12-10 09:31:28: Data line    3: TRIP
Exception in thread "main" java.lang.RuntimeException: Invalid dot separated series and station name : TRIP
at footleg.cave.PocketTopoParser.splitSeriesFromStn(Unknown Source)
at footleg.cave.PocketTopoParser.parseFile(Unknown Source)
at footleg.cave.SurveyDataConverter.main(Unknown Source)

Did I do something wrong?

My converter assumes the pocket topo data file has station names which include a dot separated format indicating the station name and number. This what I get from PocketTopo by default when I generate data from a DistoX. If you changed the station numbering to a different format then it will not cope.

Here is some example data using the convention described above:

Piedra090416  (m, 360)

[1]: 2009/04/16    2.08 

    1.0              0.00          0.00      0.00   

    1.0                  0.300  311.00  -0.39  [1]   
    1.0                  0.600  42.13  86.72  [1]   
    1.0                  1.070  350.91  -84.96  [1]   
    1.0        1.1      0.750  238.72  -46.09  [1]   
    1.0        1.1      0.740  239.16  -45.92  [1]   
    1.0        1.1      0.740  239.27  -45.85  [1]   
    1.1                  0.230  319.26    0.28  [1]   
    1.1                  0.250  229.89  77.62  [1]   
    1.1                  3.640  246.69  -85.93  [1]   
    1.1        1.2      4.540  255.04  -72.97  [1]   
    1.1        1.2      4.540  255.35  -73.00  [1]   
    1.1        1.2      4.540  255.27  -73.03  [1]   
    1.2                  0.510  141.25    4.35  [1]   
    1.2                  0.200  304.63  14.40  [1]   
    1.2                  2.190  105.14  85.00  [1]   
    1.2                  2.280  37.36  -82.81  [1]   

This is one single series, so all stations start with '1.'
If I started another side passage I would number those '2.x'

Let me know if that works for you.

I've just uploaded an update to my converter program to add a couple of features and a bug fix for an issue we discovered when trying to process some resurvey data in Spain this Summer:

- Fixed bug where zero length legs to equate a station with one in a survey series not in the
  Pocket Topo txt file being converted would crash the converter if the station in the series
  not in the file was the second station in the leg. i.e. When in Pocket Topo you had tied in
  to another survey part way along the survey of new passages, or surveyed back from the end
  point in a new passage to tie in to a known passage at the end of the survey.
 
- The Survex writer will now output survey dates so that survex can process the date and
  allow the survey to be viewed by date in Aven.
 
- The Survex writer will now put SPLAY and NOT SPLAY flags around splay legs so that they do
  not get counted in the total length calculated by Survex.


These last two new features were things I always ended up having to do manually in the Survex files after converting from PocketTopo text files. Now it is all done automatically.

Download from the same place as before:  Cave Survey Data Converter

I have just uploaded a new revision. This one fixes the DXF file reading support so that DXF files which contain different data structures are not rejected by the tool. This update is only of relevance to people attempting to convert old survey data from Autocad DXF files into other formats. Any other data formats are unaffected by these changes.

After a busy Christmas break, I have published an updated version of my Cave Converter tool. It has a new home too, as the Wiki the help used to live on has gone. The new location for the help and download are here. Source code now available to under GNU GPLv3 license.

http://wscc.darkgem.com/caveconverter/

Some minor bug fixes in this release:
- PocketTopo trip comments are now supported.
- Survex files can now contain more than 26 splays per station without containing invalid characters.
- When run on Windows, output files will now have Windows style line endings.

Enjoy!

A new version of my Cave Converter tool is now available. It converts between various cave survey file formats. Primarily to enable users of DistoX devices to get their existing data into PocketTopo, and their new survey data out of PocketTopo and into Survex format. Help and download link on the home page. Source code available under GNU GPLv3 license.

http://wscc.darkgem.com/caveconverter/

A summary of the changes in this release:
- Added support for nested series in the data model. Survex reader and writer no longer flatten
  the survey series hierarchy when series are inside other series.
- Fixed issue where Survex output files put data inside duplicated begin/end blocks.
- Improved series naming in PocketTopo converted data. Series now have the series number as
  their name rather than the full cave name and series number, so the name is not duplicated
  in the outer series name and inner series names. The outer series name is now the full name
  from the input file (previously any numbers on the end of the name were removed).
- PocketTopo converted data links to series not included in the PocketTopo file now use the
  correct series number and station number for the linked external series instead of dummy data.
- Toporobot format generation is much faster, and handles more complex cave surveys which
  could crash the converter in previous versions.
- Fixed crash in Toporobot writer when converting directly from PocketTopo format.

Bug reports and requests to support other formats always welcomed.

Enjoy!

Footleg

Only ten days since I released an update to my Cave Converter program, but I just put out another update. Just one bug fix in this one, but it finally fixes the last niggling issues in converting survey data into Toporobot format (at least as far as I am aware) so I put it straight out. My most complex caves now all convert properly for import into PocketTopo for taking underground on a PDA for surveying trips. The change log entry is as follows:

16 Mar. 2013
- Fixed bug in Toporobot writer which left survey series unlinked when two series crossed over
  each other in the middle of each series (i.e. Not joined at the start or end of either series).

Same place as usual for the download:
http://wscc.darkgem.com/caveconverter/

A new version (20140412) of my Cave Converter tool is now available. It converts between various cave survey file formats. Enables users of DistoX devices to get their existing data into PocketTopo, and their new survey data out of PocketTopo and into Survex format. Plus enables migrating survey data from one program to another.

Help and the download link can be found on the Cave Converter home page. Source code is available under a GNU GPLv3 license.
http://wscc.darkgem.com/caveconverter/

A summary of the changes in this release:

• Adopted JScience.org libraries for handling measurements, and implemented length measurements
    as unitised lengths rather than plain decimal values.
• Added support for reading Compass format survey data files.
• Added substitution of illegal characters in series and station names to Survex format output.
    Compass files allow various punctuation characters in series and station names, which are not
    allowed in Survex format data files. These will be substituted by an underscore character followed
    by a two letter code indicating the character which was substituted (e.g. '!' becomes '_ex').
• Added additional command line options:
    splays/nosplays: Survex writer and Toporobot writer now have option to not output splays.
    lrud: Option to generate LRUD data from splays (can be used on any conversion).
• Added rounding of LRUD values to 2 decimal places in the Survex file writer.
• Changed LRUD generation to better select the best splay for each dimension, and to calculate
    the distance for the LRUD dimensions based on the angle of the splay to the passage direction
    instead of using the total length of the splay. The splay selected when there is a choice is now
    the splay giving the biggest dimension in the direction of the LRUD measurement rather than using
    the splay closest to the compass bearing of the passage dimension measurement direction. Passage
    direction is now calculated from the average bearing of the forward leg and the previous leg
    arriving at the station. When more than two legs join at a station then the previous leg will be
    the one with the closest bearing to the bearing of the onward leg.
• Splays fired as back shots are now excluded from LRUD generation.
• Splays are no longer removed from the output when used to generate LRUD dimensions.
• The Survex writer now groups passage data from LRUD dimensions into blocks of readings corresponding
    to the survey legs in the file (previously one block of passage data was output per series with all
    stations listed with no respect for their connectivity).
• Added duplicate and surface flags to leg class, and added reading of flags to Survex parser and writing
    them to Survex writer.
• Made the logging output less verbose, and fixed the survey summary details log output to include inner series.

Bugs fixed:

• Fixed case sensitivity bug in Toborobot file generation which left series unlinked if series or
    station names did not have identical case.
• Fixed bug in PocketTopo parser which repeated the station name given to the 'to' station for a splay
    leg when the splays were not all located together in the survey.
• Fixed bug where compass calibration was used instead of declination calibration in PocketTopo parser,
    Survex parser and Survex writer.
• Added date reading to Survex parser.
• Fixed bug in utility function to calculate the average of a set of compass bearings (thanks to Peter Kellaway
    for pointing me to a much more robust and elegant solution). This bug does not affect the use of the function
    in previous versions (where it was only used to convert PocketTopo triple leg readings into one average leg)
    but it did show up in the new LRUD generation code when calculating the average bearing of two legs either
    side of a station when the passage went round a sharp bend.
• Fixed bug in Toporobot Writer which caused passage dimensions data to be output on the wrong station
    (LRUD values for the 'from' station of each leg were being put on the 'to' station of each leg).
• PocketTopo file reader no longer reorders the splay legs for each station (something which used to happen
    due to the old code which sorted splays into groups for L,R,U,D measurements in this reader).

Bug reports and requests to support other formats always welcomed.

Enjoy!

Footleg