Why this type-coercion dot at WRITE FILE?

[CoastalMaineBird]

I wonder how much performance improvement are you looking into

I have not timed the thing at all. The file-writing of the real thing is not written yet. I am looking to avoid things that I know are time-consuming (and memory consuming), such as the unnecessary-copying syndrome.

If writing the file was all I had going on, I wouldn't worry about it. But I have TCP connections and other data collection and screen updates to keep going while this is happening. So anything I can do to save time contributes to the betterment of the whole system.

Plus there's the fact that it's just a mystery. I have noted before that the type of a VALUE property is different from the type of a terminal, if the control is a typedef. I don't know why.

Another question that comes to me is: Is this the best way to do it? If the write-it-all-as-one-big-chunk hogs the CPU for this two seconds or whatever and starves other threads, maybe I should find another way to get it done.

[CoastalMaineBird]

I ran your code and got similarly inconsistent results. The property-node tends to be higher than the others but not in every run. The average is about 1.1 seconds.

Maybe this should tell me that I don't need to worry about it.

Still, I'd like to have something more than a gut feel to go on. My gut was apparently wrong earlier (about the buffer- allocation).

[CoastalMaineBird]

I will also question if there really is any data transfered in the case of the "New File".

A fair question, but the buffer allocation tool shows that a buffer is created, so it thinks it needs to.

The NEW FILE function needs to know only the data-type, but the property node doesn't necessarily know that.

[DFGray]

If you would like to use the buffer viewer in 7.0, check out Determining When and Where LabVIEW Creates a New Buffer at the following link:

http://digital.ni.com/public.nsf/3efedde4322fef19862567740067f3cc/c18189e84e2e415286256d330072364a?O...

You can also use your OS memory utility/task manager (Task Manager on Windows) to find out if you really are allocating the extra memory or not. Start up the memory viewer and find LabVIEW (sort descending by memory and LV will be close to the top). Note the initial memory reading. Create a large memory object in your wire of interest (1MByte or more is usually best). Now turn on execution highlighting and watch the memory reading for LabVIEW in the OS memory utility as you single step through the nodes on your code. When you hit the write VI, you should clearly see whether it makes a copy or not.

Another option is to use the profiler (Tools>>Advanced>>Profile VIs...). Turn on the memory profile before you start, as it is not on by default. This will also highlight where the time issues with this application are.

Another note - writing 20MBytes to disk at once is inefficient. You get faster results if you write it in chunks of about 65,000 bytes (just under the 64kByte boundary or 65,535 bytes).

Finally, doing benchmarks for disk writing in most OSes is very difficult due to the complex disk caching algorithms used. You usually get differenct results based on the previous disk writing history. Virus checkers and other background processes can seriously change your results. Don't be surprised if you get inconsistent results. Turn off all background processes before doing a disk benchmark, flush your cache, and be mindful that the first few thousand bytes will go direct to the memory disk cache, not the disk.

[CoastalMaineBird]

Thanks for the link to that tool - I did not know about it.

I had read that help article a while back. The trouble is that it says things like this:

However, LabVIEW analyzes the block diagram and sees that the original value of the input array is only needed once and then it is not needed again. Therefore, LabVIEW realizes that it is acceptable to modify to input array directly, rather than first making a copy of it. This is more efficient because LabVIEW can skip the data copy.

The implication is that this is a universal fact. As we have seen, it apparently is not true in this case, for reasons I don't understand.

[CoastalMaineBird]

Another note - writing 20MBytes to disk at once is inefficient. You get faster results if you write it in chunks of about 65,000 bytes (just under the 64kByte boundary or 65,535 bytes).

I have, in the past, used "hybrid" files. I created the file as a datalog file, wrote the header stuff as a cluster and closed the file. Then I opened the SAME FILE as a bytestream file, skipped to offset X (past any possible header stuff), and wrote data as a bytestream.

I handled all the calculations of where sample #X of channel # Y were in the file. It gave me several advantages: only the correct files appeared in the file-selection dialogs, simple file I/O (for the header, anyway), keeping all data in one file, not having to assemble all the data into one huge array.

It worked great, EXCEPT:

When NI changed the format of their datalog files (between LV4-5 and again between LV5-6, IIRC), they provided a converter to read the file and rewrite it in the new internal format. Of course, the converter knew nothing about my hybrid files, and truncated the data portion. But I built a converter of my own to handle that.