Interfacing Labview with snazzier data visualization

[tf]

I have a pile of log files from some automation equipment.  They are simple text files full of various metrics and results from each of the machine's substeps/processes.  I need to do some data mining from them to pull out the answers to questions like "how many cycles has component serial number X been through?,"  "What has its yield been over time?," etc.  Data files tend to be 3~4 MB per day, times many months of running, times multiple machines. 

The log files are on the PCs running each machine (via a LabView .exe), and the PCs are pretty bare bones (no Excel on them) but they do already have the LV runtime engine.  So I'm thinking I can brute force the log parsing and summarizing with Labview, but the data visualization options are pretty limited.  Has anyone tried porting data from Labview to one of the snazzy Java-based packages (like plot.ly)?  I'm not even sure what the steps would be right off, but I'm sure it's doable.  Something like having Labview parse through the files and collect data, then turn it into a formatted CSV or tab-delim file, then stuff it into plot.ly to get a beautiful interactive plot out of it.

If anybody has tried this, I'd appreciate hearing about it.  Or other ways to do it.

Gracias...

[Blokk]

Have a look at the product NI DIAdem. It can do pretty much what you described...

[Sam_Sharp]

As Blokk has said - DIAdem is the NI data aggregation / analysis and reporting tool - as well as support for common data sources (e.g. CSV files, Excel files, TDMS), you can also write scripts to pull in data from a database and extend the functionality etc. - it has lots of analysis/calculation features and even a built in reporting.

Another option would be to put all of your data into a database and then use/build a web-based tool to do the analytics from that - e.g. a series of dashboards etc. that you can use to interrogate the data.

[thoult]

External libraries like matplotlib and gnuplot can be interfaced with to produce high quality plots.

A good example of the former is the Advanced Plotting Toolkit, albeit at a cost:

http://forums.ni.com/t5/LabVIEW/Open-Beta-New-toolkit-for-high-quality-plots/m-p/3111618

http://advancedplotting.github.io/

You can probably do similar with plot.ly - or use the RESTful API.

[GerdW]

Hi thoult,

the AdvancedPlottingToolkit is available for free in the tools network by now…

[thoult]

Thanks Gerd, I didn't realise they had changed tack from their proposed charging structure!

[GerdW]

Hi Thoult,

I didn't realize it by myself, but I heard the news at German LabVIEW Days event… 😄

[Gregory]

I just tried installing that toolkit but I failed twice. When I find it in VIPM I cannot download, probably due to work restrictions. When I download from this link and install, VIPM gives me an invalid checksum error! Does anyone else have a similar experience?

[GerdW]

Hi gregoyj,

at home I could install AdvPlottingToolkit without any problems using VIPM.

You should talk to your admins!