saving labview data to a database

[falkpl]

I am looking to save data from my labview programs to a database file, I am very familiar with labview, I have the DB tool kit module but have minimal experience with databases.  I was wondering what the least expensive and simplest implementation approach, this application will be distributed on many machines as a built executable and will write to its own copy of a database.  Has anyone worked on such a problem, I am trying to avoid any potential problems.  Currently I am saving each test/run of the Vi in its own custom XML file and the files are building up with no easy search and organization method, so I think I have to learn to save the data in one database file.  Any help is greatly appreciated

-Paul

LV 7.0

[falkpl]

Also the deployment system in Windows XP PRO, I do have access to access but want a solution with no licensing if possible?

[Will.D]

I have done this in a few of the programs I have written...
If you're unfamiliar with databases, you should first familiarize
yourself with SQL. I found this and this to be quite helpful, especially since they differentiated between the different database programs.

I have never had more than about 20 pieces of data to write to a database
(at once), so I always just used "concatenate strings" to build the SQL
string with the appropriate commands (INSERT etc.), and all my data (as
separate strings). I'm not sure what the best way to go about it with
larger amounts of data though, as this way would be quite cumbersome
and confusing.

Hope this helps.


Edit: I saw your last post... I'm not entirely familiar
with licencing, but I know MySQL is a popular low cost
alternative to things like Access...

Message Edited by Novatron on 07-26-2005 02:18 PM

[Dennis_Knutson]

I don't have the DB toolkit so I don't know what if any addtional licenses are required for that. I don't think there are any but you should check the docs that come with the toolkit for info on that. As far as the database, with the toolkit you write directly to a .mdb file and Access is not required. You would only use Access to view and analyze the data. The database file (.mdb) could reside locally or you could have a shared file on the network. I'm not a big fan of Access for multi-user though. A much more robust solution would be SQL Server which is costly or MySQL which is not. The necessary ADO and odbc drivers should be part of windows.

[mikeporter]

The easy part of what you are wanting to do is getting to a database because it all comes built into Windows. The database management system is Jet and the interface capability is through ADO. Neither of which have a licensing requirement. However, Jet (the database management system, or DBMS that Access uses) is a very poor choice for multi-user applications. Although MySQL is very popular, I have always prefered Firebird, which is another Open Source DBMS that is much more robust (and safer for your data) than MySQL. Another option is the MSDE or Microsoft Desktop Engine. This is basically a desktop version of SQL Server that is free for most applications--check their licensing agreement for details. Bottom line: there are a ton of choices for DBMS and the interface capability is built into Windows so you don't have to buy that either.

For more info, see this thread too...

http://forums.ni.com/ni/board/message?board.id=170&message.id=132530

Mike...

[falkpl]

Thanks for all the help, I will try out all the options.  The databasing is fairly simple, I have 2-3 tables and each table has 5-10 fields, I got it working on access and was able to communicate with it through the jet engine on windows XP PRO from my labview application, I am very pleased.  I will try out mySQL and compare it to access which I already own.  It is good to know that I don't need more than the ADO objects and the JET database engine and a .mdb file to communicate with the database.  Again all the input has been very helpful

-Paul

[unclebump]

File size can become a problem with databases. I started out with this approach and ended up saving my DSC array data into Citadel binary memory tags. I am using this on production machinery and saving thousands of data values everyday.

[mikeporter]

Yes Paul;

But as unclebump points out, scalability can become an issue. However the best part of the approach you are taking is that if you learn the ADO part well and implement good clean modular code, you can move your application in the future to "industrial strength" database management systems like SQL Server or Oracle with little or no change in your LV code.

Mike...

[falkpl]

What is considered a large data file?  The application (using LV7.0 + DAQmx 7.2 and some NI DAQ cards) is streaming data from some sensors (typically 1-8) I designed for monitoring radiation exposures.  Although the data is collected with ~5-10000 points/sec/sensor the data is processed and a value is calculated at 2-4 times/second, a real time response.  I want to save the data of 1-4points/second with exposure times or ~1-120 seconds in a database on a per sensor basis with additional information about the exposure settings.  So each treatment will yield a few hundred points of data plus ~5 fields of information regarding the experimental setup.  Additional tables will keep track of sensor calibrations and characteristics.  I will be testing the sensors 10's of times a week.  Currently I generate an xml file for each experimental exposure but unfortunatly this leaves me with hundreds to thousands of files over time with no easy method for searching them or organizing them (the book-keeping is left up to me).  In the long run I want to retrieve data depending on the experimental exposure settings to determine the performance of the fibers.  So is this too much for the database?  I am not saving the millions of raw data points in the database, just the processed data and settings and summary of each experiment.  The raw data is saves in a binary file and the location is referenced in a field in the database table (right now xml file).  So how do know what is too much data for a database?  Each entry should have 5-10 fields, one of which a string of data of 10-1000 points and I will add entries up to maybe 50 times a day a few days a week.  Again any input is appreciated since I am a database neophyte,

Paul

[unclebump]

Around 2GB is the file size limit.

Take a look at this link.

http://zone.ni.com/devzone/conceptd.nsf/webmain/5E8689F60B84288786256F480065722D