Growing String Slowing vi

[KoJO]

---

I have a program where all data processing is based on processing an ASCII string. In the program, the measurement file is written to a txt file, the program does the calculation by reading the string, and the graph is also displayed by reading the values ​​from the string. The problem has become that the string grows so long that it slows down the state machine.

Attached is a simplified sample program where the phenomenon occurs. In the program, I increment the string at each round and plot it on a graph at the same time.

What do you think, is the problem because LabView dynamically allocates memory for the string variable, and at some point more memory is allocated each round, causing the iteration time to no longer remain constant?

In the real environment, I collect data from two measuring devices and combine the results into a graph. I have managed to eliminate all other possible causes and now it seems that the long string is causing the state machine RPM to slow down.

If I have to give up using string, the entire program architecture will have to be redesigned and I want to avoid that as long as possible. 

Is it possible to allocate a large enough memory for a string variable so that I can get rid of dynamic memory allocation, or is the problem somewhere else?

In the example program, the problems start at around 3000 iterations, at which point the delay I set as a constant becomes much larger.

Labview 2014 developper suite, 2.6GHz i5, 8G RAM

[santo_13]

String is not an efficient format to store numeric data. It is best to store as numeric array and convert to string as needed such as write to file.

[crossrulz]

The conversion to a string and then back to a number is also quite slow. The more conversions done, the worse your performance will be. As previously stated, keeping an array of your data would be a lot more efficient as the said conversions would not be required.

[Bob_Schor]

Did you know that LabVIEW is an acronym?  It stands for Laboratory Visual Instrument Engineering Workbench.  You have to learn to "think like an Engineer".  Suppose you have a signal you want to acquire and plot, and say you want to sample this signal at 100 Hz, or every 10 msec.  You have a loop (in your State Machine) that every 10 msec generates a data point.  What does it do with this data point?  If you try to save it, or plot it, or keep it in memory, you'll be accumulating data pretty quickly, and dealing with 3000 points (and growing) after acquiring 30 seconds of data are going to slow you down!

But this is LabVIEW, which lets you run several tasks ("loops") at the same time, one generating the data at 100 Hz (often called the "Producer"), and another one processing the data (often called the "Consumer").  This "dual-loop" technique is called, naturally, the "Consumer/Producer Design Pattern" -- if you open LabVIEW, click on "File", then click the second entry on the drop-down called "New...", expand "From Template" (click the "+" sign) and choose "Producer/Consumer Design Pattern (Data)", you'll see a pair of While Loops, the Producer at the top with a "Timing" function to control its speed, and the Consumer down below (which doesn't need a timer, as it "waits" for the Producer to send it data).

So now all you have to do is speed up the Consumer.  You do that by not keeping the data in memory -- write it to a disk file and also plot it on a graph or chart (I recommend a Chart, which you can set to "scroll", as though it was a pen drawing on a moving strip of paper, or an oscilloscope, with the beam showing time by moving left-to-right and showing the data values by moving vertically).

Bob Schor

[daveTW]

I understand your wish to not redesign your program.
But how dificult is it to change the datatype, which holds your data (currently string)? If you change it in all sub-VIs to the cluster with X & Y- data (make this cluster typedefed), then you will have faster access to the data by far (as crossrulz stated already). This would handle the first bottleneck.
The second problem when the amount of data rises, is the preformance of the XY-Graph. You try to display 50 times / second XY- data in a graph. I suppose, the x- data is not equidistant (else you would take a normal graph with much better performance), so it is a lot of work for the XY- graph- control. So you could lower the refresh- rate of the XY- graph.
The third problem is the amount of data, fed into the XY- graph. I suppose it is not really necessary to display 3000 values on 5 cm physical Monitor length. Consider to decimate the data which is to be displayed by the graph. Don't do it with every display cycle, instead use another decimated array (cluster of X & Y- values).
As a side effect to change the datatype to a typedefed cluster you get the opportunity to transport additional information / data with your measuring values. E.g. you could also put in the decimated arrays or the index of some active data set.

[avogadro5]

Fundamentally, you need to find a way to not retain data in memory indefinitely or you are guaranteed to run into problems as more data is added over time. If you want to keep using a big string to hold all data, you need to discard old data when the string reaches a certain size. The string subset node may help with that.

Here are some other ways to expire old data:

• Keep the data in a fixed size queue and use lossy enqueue so stale data is thrown out
• Circular buffer, here is an easy library that does this
• Use a chart instead of a graph, this comes with built in limits on history size, and you may not need to keep more than 1 sample at a time in your shift register