I am seeking a small piece of help from someone who currently uses and runs MatLab.  I would like to check on the capabilities for coping with large csv files (1MS by 5 columns in a 46MB file) to see if MatLab can read the file and then do a simple averaging routine.  The work is to do with seeking an alternative approach & software to process dynamic rope test data.  Could any one who is willing to spend probably no more than one hour on the task PM me? 

Many thanks in anticipation.

Bob

PS - Octave cannot cope.

I'm not a Matlab expert, and now retired, so I don't have much opportunity to play around with it.  But I used to read .txt files up to 4 GB into Matlab (not all at once, obviously) using the fscanf command.  Depending on how your data are formatted, this might work for you.

thanks for the encouraging reply.

Hi Bob,
I'll have access to Matlab next Wednesday so let me know if you're not sorted by then. When you say Octave couldn't cope, was it just taking too long, or not even managing at all?

Another option would be to write a quick program in 'C' to read it in and calculate the average. There are 1 million numbers to add up? How large are they? (i.e. what's a typical number)

Chris.

The purpose of the test program is to check for capacity of a program to cope with a large number of numbers before I move to purchase the software.  Whilst I am confident MatLab will be OK for doing the substantial task of data processing (currently involving several thousand lines of code in MathCad - have a look at http://www.roperesearch.co.uk/pdfs/the%20rig%20v2.PDF for an outline of the program) what is surprising is that MathCad is not capable of processing 1,000,000 rows of data.  I derived a test based on reading csv file as I understand that is fairly demanding in memory management.  MathCad failed the test, as did Octave.  (I was less surprised about Octave, recalling warnings about its limitations.) 

I have now had several offers of help which should be more than sufficient.

Thanks for the interest.

Bob