What is the best way to store lots of images? Advise needed

[lukasz.kurzepa]

Dear all,

I'm working on a software which store measurement data in database. Recently I connected and camera to the computer and I would like to make an snapshot and store it somehow in the computer and corelate them with data samples saved in database. 

I have three options, but I don't know which one is the best, which one gives the best performance.

1. Store the images as jpeg file on hard drive and save a link to that file in database.

2. Store image as binary file in database

3. Store jpeg files in zip file, so Labview has to add new jpeg file to existing zip file.

I don't want to exeed one second loop time of the program.

I like the second idea the most beacause no one can manipulate any of the image files  but single binary image file has about 500kB and after 1 our of data aquisition I will have 1.7GB of binary data in database which I think it is a lot.

Third idea is nice, bacause all images will the stored in one file, but I don't know how to add files to an existing zip file in LabVIEW.

I would appriciate any advices.

regards

Lukasz 

[toto69]

Hi,

If you store your images as binary files, it will be quicker as there is no compression step. But as you mentionned, the file size will be huge.

If you store your images as JPEG, it's a good solution as the encoding time is not a major problem nowadays with the power of actual CPUs.

But what I would sugest is to write your images in a multitiff file or AVI file. Because thes formats both supports compression (zip or jpg for tiff and MJPEG for avi) This solutions allows you to have compressed images in a single file. For AVI, you have everything available in the VDM, for multitiff there are some dll on the net you can call within LabVIEW.

Hope this helps

Regards

[TiTou]

Is your concern is mainly performance?

What's your hardware and what's the size, bit depth and rate of your image acquisition process?

I mean, if you want to store to disk less than 15-20 Mbytes per second (depending on your computer config of course) then you should be fine with any solution you mentioned. Solution 3 sounds wierd to me, then between solution 1 and 2 it depends how familiar you are with database, I am not at all so I would go for solution 1 except I would store BMP files to not loose any data and have the same size for each image, but that also is debatable depending on your needs ; if you need to do measurement of any kind in your images I think BMP is best if it's only to do visualization then any format you like really...

Hope this helps

[lukasz.kurzepa]

Dear toto69,

thank for very quick reply. I decided to store data in an avi file. Even with a small compression jpeg images have still good quality for my needs. Nice think about it is that all images are in the same file so it is impossible to accidentally delete one of them. Moreover with 10frame/sec frame rate I can replay a video 10 times faster that real process lasted. I was surprised that I cannot open avi file to write a frame on the end of the file, I had to create avi file on whilst start of the program and keep it open.

Many thanks for help  

[lukasz.kurzepa]

Dear Titou,

Thanks for your reply. In my case all images will be recorded only for visualization purposes. However I wanted to keep size of them as low as possible and keep them compact so no one can accidentally delete any frames. CPU I'm using is Intel i7 (I don't remember the clock but for sure it is not the slowest one), 4GB of RAM and 1GB HDD. Images will be stored for years, I don't want to delete any old data to make some free space on HDD, maybe I will be doing backups in the future. I know databases, I'm using one in my current project, but size of binary files will be a bit too big for me. Thanks for your advice as well I already made storing images in avi file.

Many thanks