speed up saving data recieved via serial

[mh_51]

Hi there,

I hope you can give me some advise. Till now, I didn't find a solution.

I have a sensor connected per USB-to-serial convertor to the Computer (COM 2). After starting the measurement, the sensor is continuously sending data as a string. The sensor is capable to run at different speeds and it's controlled via strings sent to the device.

I now have programmed a small piece of code to read and evaluate the data of the sensor. There is no problem while viewing the data live in several plots. But, as soon as I try to save the data, I have speed problems. The loop is not fast enough to recieve the data as it is provided by the sensor. Temporarily deleting the plots and so on, doesn't help, so the problem is saving the data. As far as I know, I should use arrays as I did and I don't believe that editing the arrays should be the bottleneck, because they should be able to handle data in general much faster.

In the Hyperterminal coming with windows, the device is working fine.

I'm using VISA serial with a timeout of 2s and a buffer of 128. The baud rate is set to 460800.
The actual VISA read command is set to read max 50 bytes or stop at a termination char (newline).
After initialising and sending the start command to the sensor, the device begins to send its data..
The different speeds are:
1 -   4 strings per second
2 -  10 per second
3 - 100 per second
4 - 500 per second

Each string gets cutted, evaluated and plotted. Each value is also stored in an array (which would be saved after the program stops).

In the picture I deleted several commands sent to the sensor, because they are not relevant and it doesn't matter (for the failure) if I delete them in the program or not. The same goes for the plots. After setting the device to the highest velocity the program fails, because there are more data than labview reads. (I've already said, that without the arrays it works fine, haven't I?).

So what can I do? Am I doing something totally stupid? Is there another, faster way to collect the data with labview? I mean, 500 HZ, that should be nothing.

Thanks for your help.

Regards,

mh

PS I'm suing LabVIEW 8.6

Computer: Intel Core2 Duo E8500 3.16 GHz with 3 GB RAM

SW: Windows XP SP3

[Dennis_Knutson · Accepted solution]

You are using the Insert Into Array with each iteration and that forces LabVIEW to keep reallocating memory for the arrays. This is not very efficient. Instead, I would advise you to use the producer/consumer architecture and pass the new data to the consumer loop where you write the data. That would skip all of the Insert Into Array functions and shift registers. You are also using some local variables and property nodes. Neither one of those are very fast either. Can't tell from the image why those are being used. Also, consider using an event structure for those Booleans wired to the case structures. Do you really need to poll those continuously and execute the code with every iteration or do you just need them to execute once when the Boolean value changes?

[mh_51]

Hi Dennis,

thanks for your fast answer.

As I am new in programming with LabView, I haven't heard of the consumer architecture before. So that is a very interesting point I will check.

I'm using the local variables and property nodes to switch on and off the LEDs (=speed and setting indicators) and as well as give scale parameters to the plots or delete their history and such stuff. So I only need them changed if I send new parameters to the sensor (e.g. faster measuring). The same goes with the boolean case structures. I only need them if I want to change parameters of the sensor and not in every loop. So using an event structure is also a good advise and I will look into it.

Thanks again and have a nice day (it's already evening here ),

mh

[mh_51]

One thing to the arrays, I forgot to add (and the time to edit my post is over, so sorry for the double post):

I initialise the arrays before the while loop gets executed the first time. I thought that would circumvent the need for allocating every time new space for them, as they should be allocated at the time of the initialisation (here 4 times 1M double entries).

Cu

[OlderJohn]

I'm not certain on how LV performs with Array writes, but you may have wanted to use the Replace Array Subset function.  With the Insert Into Array function, you are adding your data values to the 1M element array, increasing its size. 

Message Edited by OlderJohn on 01-12-2010 11:56 AM

[Dennis_Knutson]

Yes, the initialization of those large arrays is not doing any good when you use the Insert Into Array function. You would probably have been off starting with empty arrays and using the Build Array function.

[mh_51]

Hi Dennis and hi John,

it seems that my producer and consumer structure is working now. I'm not totally sure, that I did it correct but there is no speed problem any more and all the data is saved in a file. Even with 1.6k samples in a second it works fine now. I also put all my button interactions in an event loop as you suggested, Dennis, and it works fine at the moment.

With the array I thought I override the existing entry (which should be quite fast) and not adding new values to it and increasing its size. I found that way in another thread but it didn't worked out as I thought. The producer/consumer way is the better way, I think.

I added an image of my edited vi. The stuff I cutted away is just initialising stuff, which is mostly uninteresting. The local variables and property nodes are only there to set the right scaling of the plots for different speeds and make some buttons (in)visible for different sensors. I would be pleased to hear it, if you find additional errors or have any hints to do it better.

Thank you very much for your help 

mh

PS I mark the queues as solution, because that worked for me 

EDIT: Forgot to add the image...

Message Edited by mh_51 on 01-15-2010 10:02 AM