How to handle massive array operations

[Gurdas]

Heres what we did and did not do:

1) Completely rewrote of the basic algorithm. The new algo is as different from the previous as chalk from cheese. This alone is responsible for 95% of the speed gain.

2) Never ever use "build array" inside a loop. We do use it (outside a loop) in a few locations where it converts a array from 1D to 2D for the next function. Also, stick (as far as possible) to "replace subset" and avoid "insert into array".

3) Always predict and initialise the array such that memory reallocation is next to nil

4) Avoid while loops and stick to For-Next loops

5) Avoid locals and stick to property nodes. Also avoid globals.

6) Send all repetitive computation intensive areas of the code to a subvi with no front panel display. The idea was to never display the huge array unless we are done with it. From sheer LV code improvement perspective, this was a most major gain.

7) Use the best fit number type. Most of the code runs on U16. At some stage it becomes DBL. We are still to test speed gain/loss if we keep it DBL from the start. However, we do not expect any surprising gain/loss. Any thoughts on this?

😎 Use state machine architecture and event structures wherever possible. This was just code maintenance improvement and probably does little in improving speed.

9) Never handle a huge array at a single go. Process sections and keep writing the result to some store.

Most of these thumbrules were known to us (except for 6) before this problem, but, at times, we compromised and flouted the rules. This time we were fighting for every second, so complacency went out of the window.

Must thank all the people who, at some time or the other, have introduced these good coding rules to me and my team (all of us have learnt LV hands-on and without any formal training).

Rgds,

Gurdas

[shoneill]

Well, all those points sound perfectly reasonable.

I think the team has come a long way in those 3 days of brainstorming.  Maybe we should be looking out for you in the next coding challenge?

Congratulations.

Shane.

[Gurdas]

Shane, thanks for the appreciation.

Our success is mostly attributed to common sense programming and clear engineering domain knowledge. Truly, we are not great at LV programming.

Would you believe that we have never used stuff like VI server, occurences, queues, notifiers, semaphores, CIN, data sockets? When I think of all this and other great LV tricks, I feel awed how much more there is to learn, happy that we know what is the right direction and hurried that time is running out. Its been great fun until now.

I do know of the coding challenge but feel inequipped to really make myself noticeable in the company of other LV giants. Correct me if I am wrong, but is not the coding challenge a test of ONLY coding skills?

Lets make things better (picked from Philips' punchline).

Rgds,

Gurdas

[Kevin_Price]

Just a couple comments:

5) Avoid locals and stick to property nodes. Also avoid globals.

Are you sure about this one?  I thought that writing to Value property nodes was supposed to be one of the slowest ways to update a control value.  Let's see, hmmmmm,  oh yeah, here's the thread...

7) Use the best fit number type. Most of the code runs on U16. At some stage it becomes DBL. We are still to test speed gain/loss if we keep it DBL from the start. However, we do not expect any surprising gain/loss. Any thoughts on this?

I would expect that you'll want to minimize or eliminate the need to allocate a huge array midstream in the processing.  Whether you convert at the beginning or middle though, you'll probably want the DBL array to be pre-allocated.

On the other hand, my only similar experience gave me surprising results.  I inheirited some image processing code that was very uncareful about datatypes.  There were at least 5 subvi calls that had type coercion dots on the ~2 megapixel image array.  The various routines used U8, U16, SGL, and DBL for no immediately apparent reason.  Since some of the early analysis needed better-than-integer precision, I rewrite all the downstream routines to use DBL and avoid all the automatic type coercion (and inline memory allocation).  SGL had resulted in too much cumulative round-off / truncation error.  Anyway, I only realized a very modest increase in speed like maybe 20% where I expected to be 2x-3x as fast.

I'm sure this is all very dependent on specific processing requirements, size of arrays being allocated, available memory in PC, etc.  So the bottom line is that you'll need to test it out to get a valid answer for your own situation.  Still, it sounds like your initial structure is very sensible -- perform as much processing as possible in integer form before switching to floating point.  Then once in floating point, stay there rather than reallocate back to integer. 

-Kevin P.

[Gurdas]

Kevin,

I too read that thread a few days back and yes, it was very informative. Infact I had posted some questions in the same thread (page 2). Maybe you could help me with those.

The thread says that Property Node is expensive because it spends time updating the control/indicator. But what if its a subvi with no front panel display? We assumed it will be OK to use property nodes for such cases even when we are trying to save time.

Any thoughts on how right we are?

Rgds,

Gurdas

[tst]

---

@Gurdas wrote:

But what if its a subvi with no front panel display? We assumed it will be OK to use property nodes for such cases even when we are trying to save time.

Any thoughts on how right we are?

---

You're probably very wrong as a property node is only something that is used to update a control. If you get rid of the front panel (when you build the application) you will probably get an error because LV can't find the control and if you do it in LV you will probably force LV to allocate the needed memory for the FP control (something it might do anyway, I'm not sure). In any case, using a property node forces LV to update the display and to go through the UI thread which definitely takes up time.

In general, the best ways of trasferring data (memory wise) is through wires, using an in-place global (lv2 global, same as JP's example from earlier) or (if you don't care about the copies) using locals\globals which would be the fastest after wires if you don't need to allocate a lot of memory. You can try reading the performance and memory management white paper found in the LV bookshelf in the help menu.

[Gurdas]

---

tst wrote:

You're probably very wrong as a property node is only something that is used to update a control. If you get rid of the front panel (when you build the application) you will probably get an error because LV can't find the control and if you do it in LV you will probably force LV to allocate the needed memory for the FP control (something it might do anyway, I'm not sure). In any case, using a property node forces LV to update the display and to go through the UI thread which definitely takes up time.

---

We have never seen that error when building and then running an exe/installer. I am not sure what you mean by "get rid of front panel". What we do is set the subvi property to not display the FP when called. Do you think the property node would still be going through the UI thread?

We would be implementing LV2 globals next version onwards.

Rgds,

Gurdas

[tst]

When you build an application, the FPs for some of the VIs get removed (you can see this in the VI Settings tab of the builder). I'm guessing that if you're using a property node the panel doesn't get removed because it knows it needs the FP. And yes, as far as I know, using a property node will always go through the UI thread and may even cause a copy for the indicator, even if you don't open it.

[Gurdas]

---

@tst wrote:
When you build an application, the FPs for some of the VIs get removed (you can see this in the VI Settings tab of the builder). I'm guessing that if you're using a property node the panel doesn't get removed because it knows it needs the FP. And yes, as far as I know, using a property node will always go through the UI thread and may even cause a copy for the indicator, even if you don't open it.

---

Hmmm... maybe my ignorance is preventing full grasp of your answer. While you are continuously refering to removal of the front panel, I am refering to disabling its display (VI Properties > Window Appearance) when the respective vi gets loaded.

I now have a few more questions:

1) Are we both talking of the same thing?

2) How does it matter if the FP is included into the build (as long as I do not load it on vi call)?

3) I thought only locals created copies and that property nodes NEVER created a copy (unless my thread branching forces this). Am I wrong?

Rgds,

Gurdas

[tst]

OK, here's a simple example. Watch the memory monitor. When running the VI without using the Value property, the usage is 16 MB. The usage will only go up after you open the FP of the subVI. When using it, it takes another 16. That's why you don't want to use property nodes to transfer data. Also, it's much slower because of going through the UI thread and it breaks your dataflow. One thing I don't understand is why the first time I run the VI the memory monitor doesn't show any memory consumption, but run it twice and you will see it.

To better see how using property nodes forces you to have the FP in memory, try putting a reference of the array in the subVI. Just create a reference of the array with nothing attached to it. At least in LV 7 this forces LV to create the FP and requires the extra 16 MBs each time you run the caller.

Try running the application builder for this and you will see that it removes the panel for the subVI. If you add a reference or a property node inside the subVI, though, it will not remove it. I'm pretty sure that if it had been removed, attempting to use a property node from the caller like I did would fail and return an error.