memory allocation within TestStand teststeps

[mkossmann]

Are there any tricks or switches, which can be used to maximize the amount of a single memory chunk that can be allocated by code run as Teststand Teststep ?

Background for that question: We have observed that a testroutine run in a Teststand CVI code module shows that you can allocate as maximum a block of about 250MB size ( That means malloc() calls with bigger sizes will fail) when Windows Tastmananager still showing 600 MB of  free physical memory. And that maximum size doesn't change even if we put 512MB additional physical memory into the machine,which is running Teststand .

However at the same time ( Teststand running and stopped by breakpoint in the CVI code module ) a standalone CVI program using the same testroutine  is able to allocate  a 1GB chunk . So it seems that neither Windows nor CVI restricts that maximum size  but Teststand is the culprit.

[Norbert_B]

This is a common issue to 32bit applications, esp. running on 32bit OS.

See this KB from Microsoft.

You see that a 32bit process running on 32bit Windows can only allocate up to 2GB of virtual memory. On Windows 64bit, you can allocate complete 4GB (2^32) of virtual memory.

Please note that i am writing "virtual memory", not "RAM". Physical RAM only defines, how often you have to "swap page files (page faults)" when accessing virtual memory. More physical RAM, less page faults; hence, the system should get faster. But it does not change the limit of addressable memory for a single OS process.

Memory is getting fragmented during access. This is the reason why you most likely cannot allocate a contiguous fragment of memory larger than 300-400MB.

Note that you can allocate one chunk of memory of about 1GB once, but memory must not be fragmented.

Using CVI via TS mostlikely already fragmented virtual memory as such that you cannot allocate this chunk anymore.

If you want to find out about memory fragmentation on your specific machine, i ponder you will find VMMap a suitable tool.

hope this helps,

Norbert

EDIT: So all in all, this is kind of expected behavior, wether you like it or not. Sorry.

[mkossmann]

VMMap shows hat indeed memory frachmentatin seems to be the culprit. But How to workaround this?

Could the /3GB memory option help, because  with more virtual memory there is a chance to have bigger free memory chunks. And is TestStand aware  of that option ? ( mightbe after using imagecfg.exe on the executable ?)

[mkossmann]

VMMap shows hat indeed memory fragmentation seems to be the culprit. But How to workaround this?

Could the /3GB memory option help, because  with more virtual memory there is a chance to have bigger free memory chunks?  And is TestStand aware  of that option ? ( mightbe after using imagecfg.exe on the executable ?)

[Norbert_B]

The most promissing step is to use a 64bit OS. You will have complete 4GB with it for TS+CVI.

The /3GB entry in the Windows boot.ini does only help for applications which are "Large Address Aware". TS is officially not large address aware, therefore i would expect this entry not to change anything for your TS application.

Sorry,

Norbert

[dug9000]

You can edit the seqedit.exe binary if you have visual studio using the link.exe utility in order to add the large address aware flag. This is however not supported by National Instruments and there are some known issues with the sequence editor UI (all versions 4.0 and higher), and .NET adapter (versions earlier than 2010). It might work ok for you in many cases though so it's up to you.

Also, if you are the one writing the code that requires such memory, if you can write it to allocate multiple smaller blocks rather than one big block, you can avoid the memory fragmentation issue.

Hope this helps,

-Doug

[mkossmann]

---

@dug9000 wrote:

You can edit the seqedit.exe binary if you have visual studio using the link.exe utility in order to add the large address aware flag. This is however not supported by National Instruments and there are some known issues with the sequence editor UI (all versions 4.0 and higher), and .NET adapter (versions earlier than 2010). It might work ok for you in many cases though so it's up to you.

Is there some description for these UI issues available ?  And does a operator interface, which is only minimally modified from the NI example code, also suffer from these issues ?

Also, if you are the one writing the code that requires such memory, if you can write it to allocate multiple smaller blocks rather than one big block, you can avoid the memory fragmentation issue.

 Unfotunalety, the code , which finally needs the memory, is 3rd party code.

 .

[Norbert_B]

---

@mkossmann wrote:

[..] And does a operator interface, which is only minimally modified from the NI example code, also suffer from these issues ?[..]

---

Starting an EXE creates, in general, a new process within the OS. The discussed option (large memory awareness, /3GB option) has to be set individually for each process.

Afaik, the NI UI examples do not have this option configured by default so it would apply to them as well.

Norbert

[mkossmann · Accepted solution]

Replying myself: After using the imagecfg.exe utility on seqedit I can see with vmmap an additional 900M chunk of free space, which seems to solve the allocation problems. And I didn't see the UI issues so far.

[dug9000]

The ui issues will not always occur. It depends on whether or not an address in the negative range of a 4 byte integer gets used for some of the memory involved. That will likely be rare until most of the address space for the process is gone. There is a Windows registry setting that can be used for testing that causes the OS to start with the negative range of addresses when allocating memory (see this page for details http://msdn.microsoft.com/en-us/library/aa366521.aspx ). With that enabled you will quickly see the problems. If you are willing to live with the risk of rare intermittent UI problems then maybe it is ok, but it is not officially supported.

Here are examples of some of the known issues:

- The right click context menu for panes in the sequence editor do not appear.

- Click on step settings pane. The pane undocks & moves to one corner of the screen. I am unable to position it where i want. Dragging it to the center of the screen moves it towards the corner again.
- Right click on the Sequence file pane (or Sequences pane) to bring up the context menu. Click on something that is a sub-menu (ex: Step List configurations). Observe that the context menu disappears instead of opening the sub-menu.
- Drag some steps to the Templates pane. Hover over any folder that is collapsed. The folder does not expand.
- Hitting CTRL-Z (or CTRL-Y) is bringing up the floating pane that i had closed.

- Clicking on several panes takes the cursor to the origin of the screen.

Another potential option is for you to run the code which requires lots of memory in its own process. This can be done in many ways, whichever is easiest or best depends on what programming languages you are using. For example, you can call VIs out of process by calling them using the labview development environment.

Anyway, it's up to you. Just letting you know the details.

-Doug