IMAQ write to disk is extremely slow, problems configuring buffer

[PeterI]

Hi all,

I'm trying to use Labview to operate the Hamamatsu Orca Flash 4 camera.  I'm using Hamamatsu's own Labview VIs to collect the images, and then piping them into IMAQ for saving to disk.  The problem I'm encountering now is a precipitous slowdown at the point of writing to disk.  I can acquire the images at 75 FPS full frame -- already a significant decline from what you would expect, 100 FPS -- but this goes down to about 15 FPS when I write to disk, even when I am in fact saving the images to RAM using a RAM disk.

I tried setting up a buffer for the images, but I was having trouble doing this; I think I can't set up a buffer using the IMAQ VIs unless I really use IMAQ for the whole process from first to last, because ConfigureList demands an IMAQ session, which I have to make with IMAQ init, but IMAQ init wants to be connected to the camera interface, and so on.  As far as I know I can't use IMAQ to acquire the images directly from the Flash 4 so I am tied into using the Hamamatsu VIs.

Any thoughts?

Thanks,

Peter

[SamiBenF]

What is your hard disk configuration ? The Orca Flash 4 outputs a huge amount of data (around 800Mbytes/s) and of course you will need a very powerful storage configuration.

Sami.

[PeterI]

Yes, I certainly will; but just to test the speed of the system I am outputting the data directly to RAM, by writing to a RAM disk.  So that should certainly not be the bottleneck, it seems to be the IMAQ write function itself.

Peter

[BruceAmmons]

If you are writing each image to an individual file, there is a lot of overhead involved in creating, opening, and closing each file, even if you are using a RAM disk.  Perhaps try an AVI file or flattening the data to an array/string and writing to a single file (or one file per second, something like that).  IMAQ Write is probably not designed for high speed transfer.

Before you are convinced it is IMAQ Write, I would consider some other tests.  Can you use regular file IO to write the same size file repeatedly to the RAM disk at a faster speed?

Bruce

[PeterI]

Hi,

Sorry for the long delay.  I tried writing equivalent sized files (to wit, the same tiffs, but using the native Hamamatsu software) to the RAMdisk, and while this is somewhat faster than using Labview it's not exactly astonishingly fast (25 FPS as opposed to 15 FPS).  Whatever the issue is it is not a particularity of IMAQ write. Interestingly, there's no difference in speed here between writing to RAM drive and SSD, disk speed seems to not much influence the overall file writing time... so it is probably the CPU that is limiting the process...?

I tried writing the data to text file and to spreadsheet in Labview, and these two options were much slower than writing out individual tiffs.  I'm not sure what else you might have meant by "ordinary file i/o."  I also tried the suggestion of writing out AVI files, and this was appreciably faster than writing out TIFFs but not great (about 22.5 FPS). I had to convert the data to 8 bits in order to write to AVI -- the Flash 4 natively produces 16 bit images -- and this conversion is a significant part of the overhead (it also presents other problems, since I will probably need data values > 255).  Perhaps there is some better way of doing this than using the Labview "to unsigned 8 bit integer" function?

Any other thoughts / ideas on ways for me to get the data to disk?

Thanks,

Peter

[BruceAmmons]

Convert each image to a string using whatever format you like.  Open a binary file, and write the string with header turned on.  Keep writing each image as a string without closing the file.  When you are done, close the file.  You can read the file one string at a time and write each image to a separate image file when you aren't in a hurry.  I did this with one of my programs using JPEG format and it increased the speed by an order of magnitude.

Bruce

[E-mina]

Hi Peterl,

I'm cosindering using Hamamatsu Orca Flash 4 for my project, too.

I am wondering where you've got your camera file.

If you don't mind, can you share it with me?

Emina

[Andy-C]

Hello Emina,

There is more information here on acquiring a camera file. I checked myself, and it doesn't look like there is a camera file specifically for the Flash 4 at this point, although there are camera files for other Orca models. One of these camera files for a similar model may work fine or require some slight modifications. I recommend trying these first.

If none of these files work, you may consider contacting Hamamatsu for their most recent camera file specific to the Orca Flash 4.

I hope this helps!

Best,

Andrew Church

Applications Engineer

National Instruments

http://www.ni.com/support 

[PeterI]

Hi Emina,

Sorry for the long delay.  I don't recall ever having to think about a camera file... perhaps because I'm talking with the camera using Hamamatsu's own Labview VIs?

If you'd like, I can show you the code I'm using to talk with the camera... I have the buffering, but the writing to disk remains extremely slow (none of people's suggestions here made any difference, that I could tell).  My thinking is to eventually have the program write several files to several disks in parallel, I think that would sort out the problem.  Anyway, I can show you what I have.

Best,

Peter

[whatever23]

HI,

I wonder if you still ahve the code you're using to grab images from the hamatsu orcas flash 4.0? I would really love to see it as I can't buffer the images!

Is it possible to upload it on here?

Thanks