CVI Memory Management

[jr_2005]

Is there a limit on the size of a bitmap file that can be loaded with GetBitmapFromFile() ? This function returns -12 (out of memory) when attempting to load a large file (9 Mpixels JPEG), despite Windows Task Manager reporting over 1.2Gbytes of physical memory still available at the point it fails. CVI version 7.1.

JR

[LuisG]

Hi JR,

If you can reproduce the same error with the current version of the CVI runtime, we can enable some internal logging that will give us a better idea of where that error is coming from. Let me know if this is a possibility for you, and we can then go over the next few steps.

Generally speaking, I've looked into several out-of-memory errors when dealing with bitmaps, and in almost all cases, it really has been the case that the CVI runtime could not allocate enough memory from the system (via the Win32 API) to process the bitmap. I suspect that the requirements for a single large, contiguous block of memory (9 Mps x 4 bpp = 36 MBs) is what is causing the problem, even though the total amount of free memory in the system could easily accomodate it.

Having said that, 7.1 is pretty old, and I know that several bugs have been fixed in the imaging code since then, so there is also the possibility that this was being caused by one of those bugs.

Luis

[jr_2005]

Luis,

Unfortunately I cannot load a newer rutime onto our locked-down system. What you mention about fragmented memory may indeed be a clue - tomorrow I will try to repeat the experiment on a newly-booted PC and see if it makes any difference. Are there any utilities that you are aware of that can indicate how fragmented Windows free memory is?

7.1 may be old, but then so are my slippers - and I find them quite comfortable too!

Thanks

JR

[LuisG]

Sorry, I'm not aware of any such utility (which doesn't mean that they don't exist. It just means that I don't get out much).

And far be it for me to stand between a man and his slippers. It's a dangerous place to be in. One gets stepped on a lot.

Luis

[jr_2005]

On a newly-booted machine, the exact same program works just fine, successfully loading the same image file that would not load yesterday. So, it seems that fragmented physical memory is indeed the problem.

Why this should be the case, is a separate question. When any Windows process starts up, it is given a clean, linear, unfragmented virtual memory space of 2Gbytes to play in - the vast majority of time the actual mapping of virtual to physical memory is irrelevant to an application. (Obviously it gets more important at the hardware driver level, but to a pure software application it should not matter.) At the same time that CVI could not load this file, I tried several other image editing/viewing applications, and they all managed it just fine. With the same fragmented physical memory available to them, of course.

It must be a quirk of CVI (7.1), that for some reason it is requesting physically contiguous memory to load the file, even though it should not be necessary. This means that I will have to dust off my Visual Studio system and figure out how to do my application from there instead...

JR

[LuisG]

Hi JR,

What you're saying makes sense, and all I can say is that I know of no reason why the CVI runtime would be any more susceptible to the state of the physical memory than any other application. Yesterday when I was mentioned the fragmentation issue I was thinking of the process memory possibly being fragmented, rather than the physical memory. But like you said, that doesn't explain why the same program works fine on a different computer.

I've been assuming that our internal image processing code is failing in the same place where I've see it fail before, but this assumption might not be correct. If you would like to either attach here, or submit, that image for me to test it out, I might be able to find out some more information.

Luis

[jr_2005]

Luis,

There is nothing special about the 9M pixel image, except that it happens to be the largest one in our test suite. Even so, it is not that large a jpeg by today's standards. The really annoying thing is that the prototype code has been working just fine for months on a number of different deployment machines (using the same image test suite) and never missed a beat. So I give a demo, on my network development PC (that had not been restarted for a couple of months), to a prospective client and it falls over. (I rebooted this same machine the day after and the program then worked as it should - there is nothing to debug!)

I had tried changing the order of test images, to see if process memory fragmentation (caused by repeatedly loading/unloading different sized images over a period of time) was the cause, but it still failed even if the very first image loaded was the large one - at this point immediately after runnng the exe the process memory must still have been pristine, in contrast to the state of the physical memory. (I assume that the memory for the CVI call is being sought from the user memory space, and not the kernel one? That might explain the situation.)

I cannot upgrade the runtime; I could not patch it; there seems to be no other suitable CVI method of handling jpeg files (I need the X/Y sizes and to apply custom algorithms to the pixels) so I think there is no alternative for me other than to change development systems. (Just hope the LabView boys don't get to hear about this.) On the plus side, this could be ammunition for me to persuade the company to upgrade from CVI 7.

JR

[LuisG]

JR,

CVI does use user memory, not kernel memory.

I'm beginning to suspect that the problem you ran into is different from the out-of-memory errors that I had seen before. The ones I've seen before tended to always happen, regardless of the current state of the PC. I'm beginning to wonder if this might not be something more specific to .jpeg images (i.e. the code that reads the image from file, as opposed to the image manipulation code that I've seen fail before).

I'll try testing with the largest jpeg image I can find and see if and where it fails. If I have any insight, I'll post back.

At this point, I wouldn't recommend going through any trouble to upgrade your CVI runtime unless we understand exactly what the problem is, and whether it has been fixed in the later versions.

I'm really sorry for the unpleasant demo experience.

Luis

[cdk52]

JR,

Guillaume Dargaud has produced a CVI instrument (ReadSaveJpeg.fp) which may be of use to you. I have used his equivalent utility for PNG images for some years.

You can get it here.

Colin.