Higher sampling speed for accurate counter measurement

[ssomnath]

I placed the actual file-writing in a subvi and called it directly at first. I saw the same error as I have been experiencing before: 

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

"Error -200279 occurred at DAQmx Read (Analog 1D DBL 1Chan NSamp).vi:1

Possible reason(s):

Measurements: Attempted to read samples that are no longer available. The requested sample was previously available, but has since been overwritten.

Increasing the buffer size, reading the data more frequently, or specifying a fixed number of samples to read instead of reading all available samples might correct the problem.

Property: RelativeTo

Corresponding Value: Current Read Position

Property: Offset

Corresponding Value: 0

Task Name: _unnamedTask<11>

Possible reason(s):
Measurements: Attempted to read samples that are no longer available. The requested sample was previously available, but has since been overwritten.
Increasing the buffer size, reading the data more frequently, or specifying a fixed number of samples to read instead of reading all available samples might correct the problem.
Property: RelativeToCorresponding Value: Current Read PositionProperty: OffsetCorresponding Value: 0
Task Name: _unnamedTask<11>

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

So next, I made a second subvi that called the file IO subvi asynchronously. I still got the same error although I was able to sucessfully write two frames' worth of data to disk now.

Next, I removed the File IO subvi call and noticed that I still recieved the same error when duration between frame writes (I was just emptying the frame without IO now) was large.

This got me thinking that it probably has to do with some related operation in my VI that is still becoming the bottleneck and that this bottleneck tends to appear when a lot of data was collected in the frame data. I have spent quite some time trying to find out what this could be but I haven't been 

I would appreciate and guidance / debugging help I can get. Attached is a zip file containing my VI source code. 

[Michael G]

Greetings Suhas87.

I have taken a bit of a look at your code.  If I understand correctly, you have eliminated all file writing calls and still see the error detailed previously (although you are getting a small amount of data as compared to none like before).  I believe you are correct that there is a bottleneck in your code in each loop iteration that is preventing your reads from happening quickly enough.  Taking the file I/O out was enough to allow the reads to keep up for the first few iterations but that's all before it fell too far behind.  

As this appears to be a performance issue, you want to try to be as streamlined as possible.  The use of local variables can affect this and should only be used if absolutely necessary.  For instance, in one case in your VI, it seems that you read an array from a local variable, append to it, then write it back to the same local variable.  The better method to perform this type of operation would be to use a shift register or feedback node.

You might take a look at the following pages for additional insights:

VI Execution Speed

http://zone.ni.com/reference/en-XX/help/371361G-01/lvconcepts/vi_execution_speed/

Determining When and Where LabVIEW Allocates a New Buffer

http://digital.ni.com/public.nsf/allkb/C18189E84E2E415286256D330072364A?OpenDocument

Hope this helps a little.

-Michael G

[ssomnath]

Thanks for replying Michael,

I will try out these optimization techniques that you have told me. I am trying to develop data acquisition and live-visualization code for a raster-scanning application. However, is there some programming tool I am missing when desigining a vi for such an application? Is the whole set up of the VI inefficient in any way? I have read technical research papers wherein researchers were able to acquire an entire frame's worth of raster scan data, display it on the screen as a contour plot and save the data to disk within cycles lasting 20 msecs approximately, all using Labview 6.1 way back in 2005.

How dtho I get rid of this bottleneck problem?

Would it help if the data-saving code ran in a separate loop from the data-acquistion loop? I imagine I would be wasting precious time on synchronization, semaphore stuff.

I can try to simplify my requirements - 

1. acquire analog voltage data at high rates - few kS/sec for now (few tens of MS/sec in the near future).

2a. Once I get a pulse on the counter line, I need to chop up the 1D array of data I have collected so far to form a 2D array

2b. Clear the 1D array / buffer / shift register

2c. display the 2D array as a contour plot.

2d. if the user desires - save the 2D array to a file. 

Clearly 2a-2d can be done in a "separate execution thread" if you know what I mean. 

I would appreciate any advice for getting this done.

[ssomnath]

Hi I tested a new version of my simplified code (as described in my previous post). I am able to display my data using an intensity chart in real-time but the file output is still causing new problems. I am now getting this error:

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Error 1000 occurred at Invoke Node in Async_File_IO_caller.vi->06_02_06_No_Line_Trigger_Shift_Registers.vi

---------

Possible reason(s):

LabVIEW:  The VI is not in a state compatible with this operation.

Method Name: Run VI

VI Path: C:\Users\somnath2\Documents\My Dropbox\Labview\Scan_data_File_IO_minimal.vi

Error 1000 occurred at Invoke Node in Async_File_IO_caller.vi->06_02_06_No_Line_Trigger_Shift_Registers.vi---------Possible reason(s):LabVIEW:  The VI is not in a state compatible with this operation.Method Name: Run VIVI Path: C:\Users\somnath2\Documents\My Dropbox\Labview\Scan_data_File_IO_minimal.vi

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

I did google for this error and found that it may be occuring because of the fact that I am editing the properties of the VI. To my knowledge I do need to do this to update the data that the VI needs to write. I have set the auto-dispose VI to true hoping that a new instance of the file writing VI will be used I haven't found a way to work around this problem. Attached are the optimized VIs that I have been using. Any help here would be greatly appreciated.

[Matt_McLaughlin]

Hi Suhas,

The error you are getting is occurring because your code is attempting to call a new reference to a your subVI while it is already reserved for running, because the previous reference has not been closed. You can read more about this error here: What Is the Meaning of "Error 1000: The VI is not in a state compatible with this operation" in LabV...  With the false constant wired to the "Wait Until Done" invoke node, if the loop iterates again and the subVI is called again before the previous run has finished, this error will be thrown.

You can get around this error using LabVIEW Object Oriented Programming, but this does not address the main issue with the program which is the slow loop iteration.

Have you used Producer Consumer architecture in the past? This may be a good option for you depending on how long you plan on running this program. This architecture allows you to read in data in one loop and place it in a queue. Data is taken off the queue and the data operations are performed in a separate loop.

I would look into Producer Consumer architecture as a possible way to improve these issues and the overall structure of your code. Also, I saw that you have tick counts set up that you are likely using to measure loop iteration time. Try using these as a benchmark and implement some of the suggestions from the  VI Execution Speed link that Michael gave you. That link has many useful suggestions on improving performance that will likely increase the efficiency of your program.

I hope that helps!

[ssomnath]

Thanks for the producer-consumer tip. This seems to have solved the fast file writing problem (for now).