10-17-2017 06:45 PM
Has anyone set up a DIAdem script to run on TDM Server 2017? I'd like to see if TDM Server 2017 offers any performance advantage when converting a large batch of ASCII data into TDM format. This script currently runs in DIAdem, but takes about a week to process one year's worth of test data. If anyone could post some instructions on how to set this up, it would be greatly appreciated.
10-20-2017 11:13 AM
Hello Chris,
TDM Server has one component called the "Data Preprocessor" which allows you to prepare data for further analysis. One of the features is to convert the raw input file (in your case csv) to TDM or TDMS. All you would have to do (assuming you have access to a TDM Server and DIAdem 2017 is :
Open DIAdem SCRIPT
From the menu, select "Settings"=>"TDM-Server"=> "Data Preprocessor"
Create a new package
Save it => This will create a package which converts incoming data to the TDM file format. No other processing will be done.
Upload this package to the TDM Server where you assign it to a preprocessor and point this preprocessor to the directory containing the files to convert. Please take a few minutes to become familiar with the configuration details on the server side.
Now what makes the process faster is that the server can process files in parallel which is different from how you do it in DIAdem
That might already give you the performance boost you need.
That said, we could also take a look at you files and the way you read them to see whether we can do anything to accelerate the process. Which would then also help to make the process on the server faster. To do so, we would need an example file.
Let me know if you want me to take a look.
Andreas