error 2 at TDMS write

[krivan]

I am using a producer-consumer architecture and buffer the data in the consumer loop. I buffer it 10 times that results in a 100x20480 array of SGL type [~72MB]. When I try to write the given TDMS file I always get Error 2 which says

Labview: Memory is full.

Ni-488: No listeners on the GPIB.

Well, the memory is not full by far and I have no GPIB devices connected to the computer.

The VI and screenshot of the error message is attached. Can somebody please advise why I keep getting this error message.

Thanks.

[Bo_Xie]

krivan,

I can not reproduce this issue because VI_error2.vi depends on some SubVIs (e.g.,Wait for Empty Queue LoopCtrl.vi) which are not attached.

Your chunk size (~72MB) seems too big. Could you reduce it to 16 MB and try again?  BTW, did you run this VI_error2.vi on host, or on RT?

Best Regards,

Bo Xie

NI R&D

[krivan]

Bo,

thanks for the answer.

Due to computer access problems I cannot send you the files at the moment but the ones you need can be downloaded from here. These are the files for the producer-consumer architecture. I run the VI_error_2.vi on the host as I wanted to try only whether the host is able to do the task at all. The host should receive a 7.81MB big data package from the RT target over TCP in every 10ms, the data package should be a 2048000 element long 1D array of SGL type.

Now, avoiding to use the hard drive too often I tried to buffer it. In another non-RT application I stream out ~32MB packages so I thought 78.1MB should not be an issue either.

Is there a limit in TDMS regarding the package size?

BTW, is there a way to precisely check whether the RT target sends out its packages over TCP? I mean a software where I can see the incoming packages and their timing through network.

Thanks,

Krivan

[DFGray]

What version of LabVIEW are you using?  Later versions include TDMS 2.0, which will buffer the data for you, if you set it up correctly.

Given your error message and LabVIEW code, it appears you have fragmented memory and TDMS is unable to find a contiguous memory block the size of your chunk.  Generally, for file I/O, keeping chunk sizes in the 65,000 byte range will give you best performance.  Keeping chunk sizes low will also reduce your memory issues.

If you remove the buffering loop in your TDMS loop, you have a good chance of succeeding.

You may want to read Managing Large Data Sets in LabVIEW (also available in the LabVIEW help of later LabVIEW versions).

[krivan]

---

@DFGray wrote:

What version of LabVIEW are you using?  Later versions include TDMS 2.0, which will buffer the data for you, if you set it up correctly.

I'm using LV2009 SP1 and I think it uses the TDMS 2.0 by default. What do you exactly mean by that it buffers the data in case it's set up correctly?

The link was very informative, thanks.

[DFGray]

---

@krivan wrote:

What do you exactly mean by that it buffers the data in case it's set up correctly?

---

The open primitive has an input called disable buffering? (T).  By default (T), it turns off operating system buffering and turns on internal buffering.  This internal buffer ensures that all writes are the same size as a block on your disk, making your writes more efficient.  Block size is typically about 4k, so this is not a very big buffer.  Wiring  FALSE to this input allows the operating system buffer to handle disk buffering.  For multiple reads, this will be more efficient, but not for writes.  See the LabVIEW help for more details of when and why to use this input.

[Bo_Xie]

Krivan,

Did you run this issue on Windows XP? We found a similar issue on some Microsoft operating systems (e.g., Windows XP) and the root cause was: Windows API WriteFile could not handle big chunk (e.g., 72MB in your case).

Refer to MSDN http://msdn.microsoft.com/en-us/library/aa365747(VS.85).aspx which mentions the chunk size limitation:

x86 (32-bit) 67076096 bytes
IA64 67051520 bytes
X64 33525760 bytes
This limitation occurs when the file is opened with FILE_FLAG_NO_BUFFERING. 

Then the workaround might be one of the following:

1. Set "Disable Buffering (T)" to be "false" for "TDMS Open".

2. Run your VI on Windows 7. We found Windows 7 has not this chunk size limitation.

3. Reduce your chunk size from 72MB to 16MB.

I am not sure whether your issue is exactly our found issue. Please let me know whether the above workaround works.

Thanks!

Bo Xie

NI R&D