heap allocation

[Chris (UK Rochester)]

I can run the following code sucessfully if I select "Run in an external instance of CVI". If I run in process, the pop-up occurs. Debugging in the external instance of CVI shows the same memory is being requested (for the instrument to DMA into). This code started life as part of a stand alone CVI project and has been ported to a dll so it can be called from TestStand :-

void __declspec(dllexport) __stdcall TS_FireWire_Record (int icuNumber, double capturetime,
       unsigned int filter_list [MAXIMUM_NUMBER_OF_SEQUENCER_FILTERS], char *dataFilename,
       CAObjHandle seqContextCVI, char *reportText, short *errorOccurred, long *errorCode, char *errorMsg)
{
   int status [FW_BRANCHES] = { SUCCESS, SUCCESS, SUCCESS };
   int status_brn_A = SUCCESS;
   int status_brn_B = SUCCESS;
   int status_brn_C = SUCCESS;
   int overall_status = SUCCESS;
   unsigned int branch,filter_index,filter_count;
   int seq_status;
   SequencerStatus sequencer_status [FW_BRANCHES];
   double timeout = 0.0;
   unsigned int memory_size,maximum_no_elements;

   // Get the recorder memory size. Convert Mb to bytes.
   memory_size = FS_GetRecorderMemoryAllocation () * 1024 * 1024;

   // Create recorder buffers if they do not yet exist
   if (!recorder_data_branch_A) recorder_data_branch_A = malloc (memory_size);
   if (!recorder_data_branch_B) recorder_data_branch_B = malloc (memory_size);
   if (!recorder_data_branch_C) recorder_data_branch_C = malloc (memory_size);

   // Set recorder buffer parameters
   maximum_no_elements = memory_size / sizeof (SequencerData);
   position_branch_A = 0;
   position_branch_B = 0;
   position_branch_C = 0;

   // Create sort buffer if it does not exist
   if (!recorder_sorted_index) recorder_sorted_index = malloc (sizeof (SequencerData*) * maximum_no_elements);

   // Memory protection
   if ((!recorder_data_branch_A) || (!recorder_data_branch_B) || (!recorder_data_branch_C) || (!recorder_sorted_index)) {
      sprintf (errorMessage, "Unable to allocate 1394 recorder memory for %u bytes per channel and %u elements.\n", memory_size, maximum_no_elements);
      MessagePopup ("Error", errorMessage);                               
      overall_status = -1;
   }

...

The CVI compile options are the same as the Stand alone project. I can not find anything in the CVI compile options or the TestStand options that allows me to tune my run time heap allocation. I have also looked at the documentation and searched using the keyword "heap". Can anybody tell me where the controls are ?

[James_Grey]

What is the value of memory_size that you allocate 4 times, and do you make other huge allocations? If the allocations are more than 100mb a piece, you might be running into problems with virtual address space fragmentation. This occurs when you have enough memory, but not enough contiguous virtual address space in a particular process due to the locations of the other allocations (heap, dlls, etc) in the process. If this is the case, a possible fix is to allocate the same amount of memory in smaller chunks.

I'm just guessing, and if the allocations are small, you should ignore this theory....

[Chris (UK Rochester)]

Sorry about the delay in the response, but it has taken me a while to get the information together.

The pop-up message I get is "Unable to allocate 1394 recorder memory for 277872640 bytes per channel and 259209 elements." and (re-stating the above) I have also checked that when running in an external instance of CVI, when I step in I request the same amount of memory.

It has been a bit of a struggle to try and run up TestStand with an external debugger in place so I can see what happens when the dll is run in process. I eventually managed to get a break point to work in the pop-up. At that point there was sufficient continuous physical memory available for one copy of the data and sufficient continuous virtual memory available for all 4 sets of data.

The data from the instrument was split into 4 for real time recording to work when the stand alone CVI project was created and made to work.

It still appears that TestStand either does not use all the virtual memory available on the PC or the heap/stack split is not correct for my application.

I therefore believe I am back to square one and need help on where the controls are inside TestStand.

[James_Grey]

You are allocating four 256mb buffers. This is half of your available virtual address space of 2gb. The remaining 2gb belongs to the system. It is possible to get more virtual address space, but I've never tried it: http://msdn2.microsoft.com/en-us/library/aa366521.aspx).

Your virtual address space must hold all heap allocations, thread stacks, and code for the executable plus code for every dll and all of their dependencies. Depending on the order of allocations and deallocations, your 2gb of address space can easily have more than 1gb free but still not have four contiguous 256mb blocks.

I don't think TestStand is doing anything other than creating some threads and loading some dlls you would not otherwise have in your process. I suspect your allocations are too close to the edge of what is possible and those allocations combined with TestStand push you over that edge.

Some ideas:

1) try the link to get 3b of address space

2) rewrite the code to allocate smaller buffers (the total amount of allocation is probably ok)

3) run your test in a separate process. If your test module is a dll called via the CVI adapter, just select Configure>>Adapters>>LabWindows/CVI>Configure>>Execute Steps in an External Instance of CVI. Note that this will push all of your CVI code modules into a separate process.