Maps are too slow (?)

[pante]

I agree with Rolf.

But there is another trick you can use: Connect a map to the connector pane, and send an empty map, in all actions of the AE. Then have only one other action, called "Read whole map" and then send the whole map to that connector.

Pantelis

[UliMuc]

Thanks pante, this helped me. I first had connected the original map instead of an empty map.

[-Helmut-]

Hi,

I tried it in a small test-project (MapsPerformance attached, LV2021 64 Bit).
In a new application I have to store a lot of data (strings, numbers, array of anything, picture...) global so that other components 

have access. So I use a FGV with a map intern and as data a cluster of variant and an enum for the data type.

That question now is how big could be the map before running in problems ...

For example if I insert a few pictures into the map it slows down...

Has anyone experience in sharing a lot of data in one application?
Is using maps the right way?

[pante]

Hi 

I can not open your project, because i have LV 2020, but...

Maps seems very well implemented. I use them to store a few thousands of clusters, in an application, with no problem.

Now... pictures are something else. A picture can be a very big file. A lot of data.

I would suggest to make a separate database, only for the pictures.

You can make a test and compare a database with maps, and a database with arrays.

It is very important to have a good code inside the FGV. A small wrong connection can have a very big impact, because you call that vi, many times.

Normally i would say if a FGV with maps is slow, most probably there is something wrong with your code (as i had).

[altenbach]

---

@-Helmut- wrote:

That question now is how big could be the map before running in problems ...

For example if I insert a few pictures into the map it slows down...

---

Is it a question or an observation? The performance of a map (lookup, insert, delete) is linked to the map size and grows very slowly (log2(N). Of course if each entry is a truckload of data that needs to be shuffled around, copying the data out will be proportionally slower, but that is the same in any other way of storing the data.

Eventually, you'll be running out of memory but you should not run into problems. Maps (as e.g. opposed to arrays) don't need one contiguous block of memory.

(Sorry, don't have LabVIEW 2021 on this computer here. Will look at your code later)

[altenbach]

Your code is orders of magnitude too slow.

The outer subVIs (initmap, readkey, writekey) should be inlined.

for the FGV:

• disable debugging
• delete the map indicator, it is not needed!  The data is stored in the shift register, not on the front panel. (most important!!!).
• delete the disconnected boolean indicators since they are not needed.

replace the tick count with high resolution relative seconds and format the time indicators as e.g. "%.3ps" because both times will be sub-ms (<0.8ms, <0.3ms in my case). There is probably still some slack left to tighten up. 😄

(This is on a VM, so it probably would be even faster on a real computer).

[-Helmut-]

After deleting the map indicator the execution is one hundred times faster!

That is the solution, thank you very much.

[BertMcMahan]

I know this thread is long closed, but I wanted to throw out one more way to speed this up (in a more general sense). I needed a class to have a Map of values in its private data, which was being updated a few thousand times by a bunch of external function calls. Switching from a Map in the private data to a DVR of a map in the private data decreased my particular benchmarking case from about 1-2 minutes per run to about 60 milliseconds per run. This accomplishes the same effect as putting the map in an action engine, but this way is more useful for times when an AE doesn't make sense.

[Thomas_robertson]

We also were bitten by this sort of thing recently.   I wanted to summarize our findings.

After implementing maps in a small portion of our code we noticed an alarming 50% increase in the time to do some of our larger operations.   After tracing it down to the Map code we read a bit on the forums, most importantly this thread, and after some experimenting and learned some lessons which we thought we’d share:

• Passing maps around is dangerous because any copies appear to be very expensive.  I believe this is because LabVIEW has to do a complex copy of a linked list type data structure rather than a simple contiguous memory block copy.
• The safe way to pass a map around the software is to use a DVR.   All access to the maps should only happen within the in place structure, passing it out of the structure risks making copies.  For encapsulation and ease of maintenance we believe the best choice is to create a class to wrap your map and hold the DVR privately with member methods for inserting and searching with the map.
• For reasons that we don’t quite understand, it appears that having the map drawn to a control/indicator is expensive.  This performance hit occurs even in EXE’s when passing the map via control/indicator through subvis despite the controls presumably being stripped out at compile time.  I’m having a hard time believing our data on this topic because it defies my understanding of how LabVIEW compiles code to EXE, but we are measuring it.    Again using a DVR passed around directly or within a wrapper class can eliminate this concern but there may be a performance hit if you wire the map to an indicator for debug purposes. 

[altenbach]

---

@Thomas_robertson wrote:

• For reasons that we don’t quite understand, it appears that having the map drawn to a control/indicator is expensive.  This performance hit occurs even in EXE’s when passing the map via control/indicator through subvis despite the controls presumably being stripped out at compile time.  I’m having a hard time believing our data on this topic because it defies my understanding of how LabVIEW compiles code to EXE, but we are measuring it.    Again using a DVR passed around directly or within a wrapper class can eliminate this concern but there may be a performance hit if you wire the map to an indicator for debug purposes. 

---

If you use a map in a connector to a subVI, the overhead is not redrawing the front panel element (and you correctly state, this gets stripped out), but the compiler still might think that a data copy must be made. It probably depends on the overall code complexity and optimization thresholds as well as your entire architecture. One option would be to set the subVI to be inlined, but that will also increase the code complexity of the caller.

The LabVIEW compiler is an absolutely amazing piece of software engineering. A good read about the compiler is this page. Also have a look at this presentation .