Has anybody tried creating DAQmx data acquisition and save to file (hdf5)

[spsuamin]

I am familiar with using DAQmx for data acquisition. I have experience in saving with TDMS format and others except for HDF5. I was wondering if anybod has tried to re-create the example "Cont Acq&Graph Voltage - Write Data to File (TDMS).vi" except using HDF5.

Also, I have already looked through the forums pretty carefully about HD5F, and as I continue to look I either get more confused or begin to realize this capability is pretty difficult to implement. I am confused about whether niHWS file format is actually HDF5 format. 

Any insight/advice would be greatly appreciated. 

[DFGray]

NI-HWS is a specific internal data format using HDF5 (version 1.4.4).  It is HDF5.

Learning HDF5 is difficult.  Implementation of an HDF5 wrapper in LabVIEW is tedious, but not difficult.  HDF5 uses a lot of preprocessor macro functions for enum values, and these cannot be directly used by LabVIEW.  A C wrapper must be written to allow LabVIEW to call HDF5 in a consistent manner.  There are somewhere around three or four LabVIEW wrappers for HDF5 that I am aware of, so you don't need to reinvent the wheel if you don't want to.  Most are free.

That said, NI-HWS works well for most streaming DAQ applications.  I have only used it for NI-SCOPE, but the methods are very similar.  For best results, use a producer/consumer loop architecture and pass data with a queue.  This will give you a high-performance, multi-threaded application that should be disk speed limited.  The example you quote is single-loop to make it simple for an example, but will not give best performance.  However, you should be able to replace the TDMS calls with NI-HWS calls fairly easily. This will give you an HDF5 solution.

If you use HDF5 much, you should probably download HDFView, a great browsing utility for HDF5 files.

Good luck.  Let us know if you need more information.

[spsuamin]

Thanks DFGray

Yes I have seen several of the LabVIEW wrappers for HDF5. I am still a little confused though. Why would I need these wrappers if I can use the vis on the NI HWS pallette? 

I agree about implementation. I would definetly setup a parallel architecture. I have actually adopted the daqmx multi-streaming example as my primary architecture in developing DAQ applications. I just used the standard "Cont Acq&Graph Voltage - Write Data to File (TDMS).vi" as an example. 

Do you have any opinions on HDF5 versus TDMS?

[DFGray]

NI-HWS is a wrapper over HDF5 to remove much of the complexity of programming with HDF5, but is designed for waveform storage only.  If you need to do something more general (e.g. 3D arrays or images), you should use an HDF5 wrapper which exposes more of the base API.  For waveform storage, use NI-HWS.

I am a National Instruments employee, so my opinions will be biased.  That said,  HDF5 is a general purpose file format while TDMS is designed for large channel count, high-speed logging/data streaming.  In streaming multiple channels to disk, current versions of TDMS should outperform HDF5 somewhere around 8 or more channels.  HDF5 is more mature, so has a lot more bells and whistles and is supported directly by more third party analysis packages.  TDMS is much easier to use and has support for most major analysis packages. So which you use really depends on what you are trying to do and where the data is going.

Let us know if we can give you any more help.