References vs dataflow: efficiency

[Svingen]

I have seen some relevant posts on this here before, but would like to hear some comments on this particular issue.

I have a program that use relatively large 1D arrays to pass data around. I have read somewehre in the manuals that globals are inefficient and should be avoided. But what about references vs efficiency? The arrays are within an array of strict typedefs (include alot of other things as well), so my program consist of a loop where the data is updated through ordinary dataflow using shift registers. I think probably the program would be cleaner and easier to maintain if i use references, but would this impose a penalty in efficiency? The program has to run fast.

Thanks

[Mads]

Use functional globals (search for the phrase here and you'll find lot's of descriptions). You say the arrays are withing an array....do you really mean within a cluster? LV does not allow arrays of arrays so I assume you are using a cluster to hold multiple arrays?

If you use a functional global you may find that having each of the cluster elements (i.e. arrays etc). in separate shift registers in the global instead can speed things up; the simpler the structure the better.

References will typically involve the use of property nodes and with them a number of side-effects that can reduce the efficiency of the code, especially when dealing with large arrays and clusters.

[Jim_Kring]

Take a look at this presentation and examples:

http://digital.ni.com/niweekpresentations.nsf/a3153572d74a4dd686256a8e00799a3c/d367799bdf83d38c86256cf3006235dd?OpenDocument

It discusses this.

[Greg_McKaskle2]

> I have seen some relevant posts on this here before, but would like to
> hear some comments on this particular issue.
>

This has been written about before, and my answer is the same.
References are to controls, not to data. The reference should be used
for UI programming and not to transfer data from one part of your
program to another.

The performance for copying data will be quite slow since they are
similar to updating the UI. Also keep in mind that LV diagrams make it
very easy to write parallel code, and references are an open invitation
to a race condition.

So, if you want to avoid race conditions and let the LV compiler do its
job by using primarily wires and shift regs.

Greg McKaskle

[Svingen]

Thanks all

Looking through the Large data handling at this moment.

I should maybe be more specific. The data is an array of clusters. Within that cluster is several arrays and other parameters. The array of clusters are not that large, but the arrays within the cluster can be large.

Splitting this data up into more pieces would be an option, but only when using references, not when using dataflow as this would completely clutter up the chart.

The best way to do this would be by using pointers to the data. I understand that globals make copyes of the data and therefore make the program slow, but do a reference make a copy of the data?

[Svingen]

I should mention also that the arrays within the clusters need to be referenced to each other in the form: G_A(2,6)=Q_B(4,9) for instance, where G and Q are two different clusters within the array of clusters. Actually much more complex so that it requires the use of formula nodes to be practical. For simple arrays it seems (although not completely tested) that this can be done within formula nodes without additional copies, if the input and output of the array is the same.

The functional Globals look very interresting, but really only for simple arrays it seems, and not when using formula nodes due to indexing.

If the data has to be splitted up into all it's pieces, then it seems the whole thing would be much better written in C or FORTRAN and linked
to a DLL or something, surely this can't be true? or.

[Mads]

Could you post a picture of the code you currently have, or the code itself? That way we can attack the problem more directly, instead of only giving general advice.

[Svingen]

This is an example of one function, there are several dozens of functions more or less similar in concept.

This is the "old" version where the data is in a single cluster. This function is inefficient since copies are made of each array (probably due to the bundle function to the right). Started out using index registers on each array, but this was really unpractical due to all the wires.

In the new version a different algorithm is to be used (algebraic time discretization vs space-time discretization, better for less detailed realtime survailance of very large systems, but that doesn't matter here). In this version data is best stored per element basis (array of clusters).

The same "conceptual" problem exist in both versions, but more so in the new v
ersion. In the old version functional globals could/should be used instead of the cluster i think. In the new version this would be difficult without some clever and completely rewritten indexing routines.

But, if the "reference" in LV really isn't a reference, then everything has to be split up as I see it, when data is of some size. This would be a real problem for me because it would restrict how data is stored and accessed, thus complicate the program tremendously. Besides, why does references or pointers destroy the dataflow paradigm? If the data is a reference, it is still flowing, isn't it?

Anyway, looks like functional globals could do the trick for now.

[Greg_McKaskle2]

> The best way to do this would be by using pointers to the data. I
> understand that globals make copyes of the data and therefore make the
> program slow, but do a reference make a copy of the data?

A functional global can be used for complex clusters as well as singe
arrays. It just means that the subVI interface will have more options.
Again, references within the cluster will make data copies, and they
will go through the same code as a UI update, meaning that if the window
is open there will be costs there, and if it isn't open you will still
pay some of them.

As mentioned in the other response, please give more details so your
problem is clearer.

Greg McKaskle

[Svingen]

I meant if the data itself is flowing, or a reference to the data is flowing, that would not matter regarding the dataflow paradigm other than semantically.

What i want is a large set of static data, preferably a structured one since the data is large and complicated, and structured in a practical orderly fashion that is easy to understand for humans. Then i want to operate on that data without making copies of it. No more, no less 🙂