Loading ATF File Fails Until DIAdem is Restarted

[Chris_P_SD]

Running Windows 7 64 bit with DIAdem 2017 SP1, I occasionally get the "Access Violation" error shown below.  This occurs whether loading the ATF file from a script, or simply dragging the file into the Data Portal.

After closing and reopening DIAdem, the same file loads without a hitch.

Any ideas as to what could be causing this?

[AndreasK]

Is it possible that your DIAdem progress runs out of memory.

Could you please check in Process monitor if DIAdem eats memory over time that is not comming back when freeing the Data portal ...

[Chris_P_SD]

Hi Andreas, I have been waiting for this to happen again so I can check the memory.  The computer has 16 GB of physical RAM and is running Windows 7 64 bit.  The ATF files are no larger than 200 MB.

[Brad_Turpin]

Hi Chris,

Please specify how much RAM is available/free at the moment that this loading issue occurs.  The fact that a computer has 16 GB before the OS is loaded and an unknown number of other programs are started is at best a rough indicator.  Also, though the file may be only 200 MB on disk, if the IEEE data types used are integers instead of doubles, it could easily take up 2x , 4x or more memory in RAM when loading.  DIAdem always balloons all loaded data arrays into doubles, regardless of the starting data type in the file format.

I agree that your specifications indicate that it's unlikely, but it's still conceivable that <= 1 GB of RAM is available at that moment and that the loading data file takes > 1 GB in RAM.

Brad Turpin

DIAdem Product Support Engineer

National Instruments

[Chris_P_SD]

Just experienced this issue again while trying to load the same ATF file back to back.  The first attempt failed and memory usage never exceed 9.0/15.8 GB.  No data was loaded in the Data Portal, no scripts were loaded, and no templates were loaded in DIAdem.  After closing and reopening DIAdem, the same file loaded without issue.  The second time, memory usage again did not exceed 9.0 GB of 15.8 GB available.

[AndreasK]

I am not sure wheater you are using the 64bit version of DIAdem 2017. If you are using the 32bit version the maximum will be 4GB.

Maybe you can check with one of your files if memory is lost by running the following loop.

Option Explicit
dim atfxFilePath : atfxFilePath = "C:\Users\Public\Documents\National Instruments\DIAdem 2017 (32-bit)\Data\Demo2.atf"

data.Root.Clear
dim i : for i = 1 to 1000
  MsgLineDisp "Load " & i & " time " & CStr(SystemInfo.ProcessMemoryUsage)
  call dataFileLoad(atfxFilePath, "ATFX")
  data.Root.Clear
Next

[Chris_P_SD]

Hi Andreas,

Thanks for following up on this.  I'm currently running the loop you provided to load 360 MB ATF file.  It's currently displaying "Load 5 time 196088" in the message line at the bottom.  I haven't seen this number exceed 200000.

I did notice the other day that even though the DataPortal had been cleared, I couldn't delete an ATF file from Windows until closing DIAdem.  That struck me as abnormal since usually clearing the data portal releases files.  Any idea why that would have happened?

I'm not sure if I mentioned this earlier in the post, but I'm running DIAdem 2018 x64 on a Windows 10 computer with 16 GB RAM.

Cheers,

Chris

[Chris_P_SD]

"Load 8 time 383980"

"Load 9 time 175932"

"Load 10 time 171268"

"Load 11 time 173584"

..."Load 35 time 189248"

..."Load 38 time 430404"

..."Load 42 time 437720"

..."Load 68 time 380012"

..."Load 82 time 463880"

This has been running on a single file for more than four hours now.  I am going to stop it because I need to use my computer.

[AndreasK]

> I did notice the other day that even though the DataPortal had been cleared, I couldn't delete an ATF file from Windows until closing DIAdem.

This points to somhow lossing memory or file handles which will sooner or later cause the behavior you described.
Is it possible to find a file that causes this behavior that could be given to us to do further checks to build a potential fix for future versions or find a workaround?

[Chris_P_SD]

Hi Andreas,

No problem, how would you like me to send you the file?  It's curious that a given ATF file doesn't always cause an error when loading.