Possible threadsafe issues using Matlab scripts?

[cbrouse]

Hi all,

I'm developing an application in Labview 6.1 that makes extensive use of Matlab scripts.  The application executes the same set of scripts for multiple data sources, and keeps track of the inputs and outputs for each data source separately.  I've been noticing that the application produces the correct results when running only one data source, but that the results are often incorrect when I run it against multiple data sources.

I've done some investigation, and have found that all Matlab scripts share a common memory workspace in Matlab.  That is to say that if I have ScriptA that takes variable X as an input, and ScriptB that does not, ScriptB nonetheless has access to variable X in its workspace (from the last time ScriptA was called).  This doesn't pose a problem to me directly, but it has made me wonder if there are potential threading issues in the scripts.

Consider the following scenario with one script and two data sources.  I'm wondering if this scenario is possible in Labview:

1.  Thread1 calls ScriptA, passing data from SourceX.
2.  During Matlab execution (somewhere in the middle), Thread1 is suspended.
3.  Thread2 calls ScriptA, passing data from SourceY (data have the same variable names, since it's the same script).
4.  Thread2 finishes ScriptA execution uninterrupted with SourceY data.
5.  Thread1 is reactivated, but now the Matlab workspace data has been replaced with SourceY.  So, Thread1 started with SourceX data, but finished with SourceY data.
6.  An incorrect result for Thread1 is produced, because the wrong data was processed.

Clearly I could get around this problem with a semaphore, but first I'd like to know if this scenario can even happen.  Are Matlab scripts treated as atomic operations in Labview, or can the threads be suspended midway through execution?

Thanks for your help!

cjb

[JLS]

Hello,

If you don't need to parallel process them - that is, if you can tolerate computing your multiple data sources one by one - then you can simply put that code into a subVI, and call the same subVI in your code at will.  The idea here is that, if your subVI is not set to be reentrant, then since there will only be one copy of memory allocated for that VI, the first copy of that subVI to begin executing will essentially restrict others from executing until it completes - that's LabVIEW behavior.  Of course, you could force this also by wiring them in "series" (using dataflow) or your could use a trivial state machine (or even a sequence structure) to ensure they execute in the appropriate order, and you don't end up with a case such as you described.  Under the subVI premise, as long as you initialize your variables in your script then, I think your LabVIEW code will be insensitive to how the script is handled outside of LabVIEW, since it will only ever be using it once, and each time all necessary variables will be initialized.  Of course, if you define your VI to be reentrant, then LabVIEW will allocate new memory for each instance of your subVI, however, I don't know if it will be able to ensure new copies of memory for your script.  You can try this, just to be sure, but your best bet is likely to ensure that the script is only ever in use at one time, such as suggested above.

As another thought, have you explored the possibility of a setting which would all your script to get memory for each instance it is used, sort of like a reentrant VI?  Perhaps there is such a feature and/or some syntax you could use to enable this.  Another option would be to write your script to account for multiple uses, perhaps supporting some finite maximum number of calls, and allocating separate memory for each.

Thanks and I hope this helps!

Best Regards,

JLS

[cbrouse]

Thanks JLS.  You bring up a good point:  data from my various sources is executed sequentially inside a sequence structure, so that should eliminate any threading issues.

My funny results must be coming from somewhere else.

Thanks again for your help!

cjb

[JLS]

No problem, but you may still want to check the script to see that all variables are initialized explicitly at the beginning.  I can imagine a case where the script defines, uses, and manipulates an array (for example) but doesn't initialize it to, say, all 0s.  Then, if your data sources all assume that it will start with all 0s, but the first data source manipulates it, then subsequent data sources may not see the expected initialized state. 

Just something to keep in mind perhaps, although I am admittedly not such a script expert.

Best regards, and I hope you get some more clues to the underlying problem.  Feel free to respost if you think there's anything the community out here can help you with!

JLS

[cbrouse]

If you have another minute, JLS, I'd like to see what you think of something.  I have analyzed my data a little more, and I'm seeing two very distinct and unexplainable failures in my Labview/Matlab interaction.

I have a VI that receives some data and returns some data.  Inside is a loop (designed to iterate only once, so that I can have shift registers) that contains a Matlab script.  My shift registers are uninitialized, but I don't think this is a problem because my Matlab code is designed to recognize the first call and initialize the values itself.  Matlab performs some operations on the shift register data and outputs it back to the shift registers for the next call.  Some of this data is then passed out to the calling VI.  For debugging purposes, I've written a logger in my Matlab script to dump the inputs and outputs to disk for future analysis.

Normally this system works quite well, but on occasion I'm seeing some very strange behaviour.  There are two distinct cases that I find particularly disturbing:

1.  Matlab receives the data correctly, manipulates it, but then it seems to be removed from Matlab memory before being outputted back to Labview.  I know this because in my debug log the variable is received properly at the inputs but then is absent at the outputs (ie empty array).  The code does nothing to clear the workspace memory, and I'm confident that it is not a bug in my Matlab code.

2.  Matlab returns the data correctly (again, my debug log of the outputs shows the data is present and correct), and this is put into a shift register.  The next time the VI is called, however, the shift register no longer contains the data.  Matlab receives an empty array as input.  It seems Labview sometimes loses shift register data from one call to the next.  Nothing else is writing to these registers, so it's not a side effect of other code.

Both of these problems are intermittent, which is what led me to believe it may be a threading issue; however, with your help I am now confident that threads aren't a problem.

Have you ever seen or heard of such a problem before?

Thanks again for your help!!

cjb

[JLS]

Hello,

Interesting behavior indeed.  For the first issue, I am not sure why matlab seems to lose the data, but for 2, we should be able to find a solution.  Using unitialized shift registers should have the affect of retaining data across calls until the VI is recompiled (or closed and reopened).  Basically, while the VI is in memory and not modified, the shift register should retain data from the previous loop iteration, even across distinct executions of the VI.  Now, this also applies to subVIs, which is where I am going here...  I would recommend creating a so-called function global variable (also called a code module).  It is usually a subVI containing a while loop with a case structure in it, where the while loop only executes ONCE each time the subVI is called.  It may seem strange, but the idea is that you will always pass in the case to execute (with a numeric or enum or whatever... enums are best actually).  Then you use the while loop to hold a shift register - this way you can store data across distinct executions of the same subVI placed in muliple locations on a block diagram... the second instance can remember things from the first instances execution.  There are usually at least two cases though, one to initialize the shift register, and another to process or compute whatever the subVI's job is... and usually updates the shift register as well to "remember" computation across distinct instances/executions.  You will need to initialize the shift register so that an allocation takes place for the amount of data you are storing in the shift register.

Can you try integrating this idea into your VI.  It would allow you to call that subVI multiple times, and the subVI should remember the data from previous executions.

I look forward to hearing back from you, and hope we can solve the issue at least on the LabVIEW side.  Based on the behavior you describe, perhaps there is something very subtle occuring with threads... maybe the experiment I proposed will shed some light.  Also, let me know if you have trouble implementing the code module in your application, or if my explanation wasn't sufficient to get you started.

Thank you,

Best Regards,

JLS

[cbrouse]

Hi JLS,

My code is doing pretty much exactly what you've described, and normally behaves exactly as you've described.  I have a subVI with a while loop that iterates only once, so that I can store data in shift registers.  I have confirmed that most of the time the data remain in the registers across multiple calls, as they should.  I am not initializing my shift registers directly (unlike your recommendation); instead, my Matlab code (inside the dummy while loop) is performing the initialization itself during the first call.  I believe my setup should be equivalent to your description; indeed, most of the time this setup works properly.  The shift registers are initialized to the proper value (by Matlab) and retain the proper data across all calls to the subVI.

On occasion, however, shift register data is lost from one call to the next during execution.  The subVI is not being opened or closed, and no manual recompilation is being performed.  I don't believe Labview should be recompiling my subVIs during execution, so I doubt this is the cause of my data loss.  Are you aware of any case in which Labview would perform automatic recompilation during execution?

This is quite a mystery.  Your descriptions indicate that I have the correct implementation, and indeed it usually works.  Perhaps you are right that there is some subtle threading problem.  I think I am going to have to move from a Labview/Matlab hybrid architecture towards a pure Labview implementation.  This will create a lot of work, but I don't see any other way of solving this bug.  🙂

Thanks again for your help JLS,

cjb

[JLS]

Hello,

No. LabVIEW will definitely not compile a VI while it's running - I think that would be, even conceptually, impossible.  In fact, some properties (accessed via property nodes) are not editable at run-time for basically that reason, that a recompile would be necessary in order for the affect to take place, and the recompile can't take place at run-time!

It is indeed strange behavior, particularly because it is intermittent (which does hint at a threading issue, or some factor outside the development environment).  Can you try moving the initialization to LabVIEW?  I have attached an example of a so-called functional global variable (a simple code module in this case).  It has two "states" (which are cases) - one is initialize and the other is the "usually executing case."  The initialize case is used to initialize the shift register to an array of 50 doubles, where the other case will compute the running average, min, and max across the 50 most recent random numbers generated.  The 50 data point code module.vi is the code module, and it is called by Use 50 data point code module.vi.  When you run Use 50 data point code module.vi, be sure to toggle the initialize boolean so that the initialize case executes once to allocate and initialize an array with 50 elements; when you toggle it back then it will execute the other case on each iteration, and begin computing the statistics noted by keeping an array with the 50 most recent random data points generated.

I hope this helps and that you are able to incorporate a similar code module into your application to avoid the intermittent errant behavior!

Best Regards, and feel free to post again with updates!

JLS

[cbrouse]

Hi JLS,

Unfortunately I'm running an older version of Labview (6.1), so I can't open your files.  😞  Thanks for taking the time to make them for me though!

Also unfortunately, I can't really initialize my shift registers as I believe you are suggesting, because their size grows with time.  Every time a new data point arrives, it is appended to the vector or array, and there is no way to know a priori how many data points there will be.

Anyway, I suppose we will just have to move away from the Labview/Matlab hybrid implementation.

Regards,

cjb

[JLS]

Hello,

I understand your dilemma - in case it is useful, though, here are the screenshots for the VIs I attached.  They are simple, and it would be a fairly short exercise to recreate them in 6.1 if they are useful to you!

Best Regards,

JLS