LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Code structure for data gathering—ways to avoid dynamic array size?

Great, thanks for pitching in. Good point, the NaN flag, I've seen it used but I never really picked it up myself. I'll keep it in mind.

 

This much I can say to your questions: The number of simultaneous points to be tracked at a given time is relatively small, say below 30. It's definitely a very small fraction of the pixel count. The largest number of steps in one track, however, can be anything so we gotta take that into account. Most of the points should be still simply travelling from one boundary to another. Tracks cross, intersect, loop, falsely merge, all those things. Where I do would like to handle all such cases there's only some much time I can devote to hone the tracking algorithm. If you're interested, we can go there as well and take a look at it together.

0 Kudos
Message 11 of 21
(1,004 Views)

sorry it was Dr. Damien

0 Kudos
Message 13 of 21
(977 Views)

@Neos wrote:

https://forums.ni.com/t5/LabVIEW/Dr-Damien-s-Development-Memory-Allocation-in-Loops/m-p/107571

 


Excellent, thank you! That tip is most certainly a keeper. I'll see how I can make good use of it in my VI. As a side note, the link didn't work for me for some reason, went into a totally different page. I did find the right page in the end through search. This one works for me:

 

https://forums.ni.com/t5/LabVIEW/Dr-Damien-s-Development-Memory-Allocation-in-Loops/td-p/1075712

0 Kudos
Message 14 of 21
(964 Views)

How important is ordering? If not, then you can copy the last row to the current and use Split/Array subset.

/Y

G# - Award winning reference based OOP for LV, for free! - Qestit VIPM GitHub

Qestit Systems
Certified-LabVIEW-Developer
0 Kudos
Message 15 of 21
(945 Views)

As a side note, the link didn't work for me for some reason, went into a totally different page. I did find the right page in the end through search


Smiley Surprised

Smiley Happy

0 Kudos
Message 16 of 21
(941 Views)

I have thought about your problem and decided to try something.  

 

First, I defined the tasks:

1. Determine if new tracks need to be created.  This is implemented in your test VI as Random >= 0.99.

2. Add new point for each existing track.  (tracking algorithm)

3. Test new points for boundary limits.  (In Range & Coerce?)

4. Remove tracks which hit boundaries and send to parallel loop. (Right inner For loop)

 

Next I considered some alternate data structures:

1. Use integers? Join into one larger integer rather than cluster? { You are processing images, I think. Thus the coordinates of a pixel representing the position of the item to be tracked may be represented by 2 integers, probably 16-bit integers.  This is half of the data space required by two SGL coordinates. By combining to a single U32 (or U64 to include timing), the data can be processed as an array of integers rather than an array of cluster of two SGL. } 

2. Use complex?  { Credit altenbach for pointing out that complex can make things simple. The cluster of coordinates could be replaced by one CSG (Complex Single) and then processed as an array as with the integers above. If non-integer values are required this may be the preferred route. }

3. Use named queues with name representing start of track - location and time?  { Use Join Numbers to combine X, Y, and t values (of the first element of a track) to a U64 then typecast to string. This string will be unique for each track. Use this as the name of a named queue. The combined number can be the element of the queue because it contains all the information required.  Each time a point is added to the queue by the Tracking Algorithm, the X, Y, and t values for that point are placed into the queue. }

4. When removing a track just pass name of queue to other loop where it will be flushed and released.  { To remove a track, replace the name of the track queue with a default value which will not occur in the data and pass the name to the Consumer loop. In the consumer loop the named queue is Released and all the data extracted. }

 

I put together a VI which implements 1, 3, and 4 (struct.4.vi). I also modified your VI slightly (struct.2.vi) to get some information about how it worked.  I tested them with 20000, 200000, and 2000000 iterations.  I used the Profiler to test the time and memory use. While that has some limitations I think it is a way to make reasonable comparisons.

 

Iterations Time                       kBytes (struct.2)         Time kBytes (struct.4) 

20000        1                             8900                         5.5    13.6

200000      236                        20500                        54.7    13.6

2000000    Abort after 1936 s    53800                      550      13.6

 

For smaller sets the struct.2.vi is faster although the memory usage is much greater. The time of struct.2.vi appears to increase exponentially withthe number of iterations while the time of struct.4.vi increases linearly.

 

I have attached the VIs in LV2012. I used some features (Subdiagram Labels) not available in earlier versions. The image below shows the BD. The unshown cases are just wired straight through.

 

Note that the Enqueuing on the named queues is done using the Enqueue Element at Opposite End function rather than the standard Enqueue. The reason is  that this makes it LIFO (Last In First Out) buffer rather than the standard FIFO behavior of a queue. This makes it easy to get the previous value required for the tracking algorithm.

 

Note that the boundary tests take place in the same case as the tracking algorithm so that only one element coordinate need be present at any time.  This will be two coordinates in the real system if you also need to check X boundaries.

 

The Queue Names array is initialized with Max tracks elements of "No Queue". The constant "FFFF FFFF FFFF FFFF" in hex display mode serves as the No Queue default value. This value cannot occur in the combination of X, Y, and t. It is unlikely that the original images have 65536 pixels in each axis and the maximum value of an I32 ( the iteration count used as "t") is 2^31-1 = 7FFF FFFF.  When a track is added, the first No Queue in the array is replaced by the string created by combining integer values of X, Y, and t of the first element of the track. When a track is removed, the track name is repleced by No Queue again.

 

Lynn

 

Struct4.png

Download All
Message 17 of 21
(920 Views)

Oh wow, that is one extensive reply. Kudos whole post history.

 

Very interesting ideas, no doubt! I usually always start with a complicated but sort of intuitive data structures, like the coordinate cluster in this case, and sometimes I flatten it simply into 1D integer arrays (even X, odd Y etc.), I never thought of actually joining the coordinate numbers, neat! I wonder how big are the differences between all these? Nevertheless, great to learn new tricks.

 

Using a multitude of queues and an array of queue names never crossed my mind. For some reason, I never used queues for anything else but top level consumer/producer -structure with relatively large data chunks. I suppose I always thought the overhead is too large for such repetitive queue management? Here if addresses the problem with the overall data gathering structure well. Nice design!

 

Something acts up in the struct4 VI when I run it, though. First of all, the the Search and Replace String match returns offset 8 inside Add track(s) and 0 in the other one. Because of the replacement maybe? In any case, I switched the comparison to "Greater Or Equal To Zero?" instead of "Greater Than Zero?" in both. As far as I know that bit should work as intented then. However, I still run into either Error 1 (invalid parameter input) in the second "Enqueue Element At Opposite End" or Error 1100 (object not found) in the consumer "Obtain Queue".

 

I'll take another look on the evening, I don't really see why it shouldn't work. A big thanks anyway!

0 Kudos
Message 18 of 21
(901 Views)

Nevermind the second error, it was just caused by the queue releasing.

0 Kudos
Message 19 of 21
(896 Views)

... and the other error was came up every now and then when a track was sent to the consumer before the last elements were added. I connected the error line to force the order. This one works for me.

0 Kudos
Message 20 of 21
(888 Views)