Diadem script to compute the moving average of a channel

[ebenellis]

I'm trying to calculate a moving average of a given channel of a tdms data file in DIAdem.

The smoothing function won't work for me and I need a way to calculate the moving average of a given channel.

Can someone recommend the best way to do this?

[Brad_Turpin]

Hi josh,

The Smooth function in DIAdem is one of three functions that calculate the moving average.  Please explain what you mean by moving average that does not correspond with the Smooth calculation.  Do you want to only calculate based on leading data points or only from trailing data points?  The Smooth calculation always uses data points on either X side of the given point in its moving calculation.  Or do you want something different from the boundary conditions available in the Smooth function?  What are you after that's different from what the Smooth function and its two cousins do?

Brad Turpin

Principal Technical Support Engineer

NI

[ebenellis]

Hi Brad,

I want to calculate the moving average based on the leading data points. Please find below an example where the moving average is calculated based on N=3 leading data points. Could you please advise how we can do this in DIAdem?

Samples	Moving Average, N = 3
1	1
2	1.5
3	2
4	3
5	4
6	5
7	6
8	7
9	8

[Brad_Turpin]

Hi josh,

When I wrote a quick script to reproduce your table above and try out some things, I found in the DIAdem 2021 help system that there is a "leftOnly" mode for the Smooth() function in DIAdem that appears to do exactly what you're requesting.

What version of DIAdem are you using again?  Do you see this option in your DIAdem help system under the Smooth() command?

Call Data.Root.Clear()
Set Group = Data.Root.ChannelGroups.Add("Test")
Set ValChannel = Group.Channels.Add("Values", DataTypeChnFloat64)
Set AveChannel = Group.Channels.Add("Averages", DataTypeChnFloat64)
Call ChnLinGen(ValChannel, 1, 9, 9)
Call ChnSmooth(ValChannel, AveChannel, 2, "leftOnly")

Brad Turpin

Principal Technical Support Engineer

NI

[ebenellis]

Thanks Brad I am using DIAdem 2021 and it has the "leftOnly" option. I tried the below script and ran into the error:

Could you please suggest a way to directly read the channel from .tdms file?

'Error message from the DIAdem command kernel:\r\n\r\nInvalid channel in command "ChnLinGen" (Parameter: 1).', None, 0, 0), None)

import sys
if 'DIAdem' in sys.modules:
    from DIAdem import Application as dd

    if False:
        import DIAdem_CodeCompletion as dd

#    if dd.AppEnableScriptDebugger:
#        import debugpy
#        debugpy.configure(python = sys.prefix + '\\python.exe')
#        if not debugpy.is_client_connected():
#            try:
#                debugpy.listen(5678)
#            except:
#                pass
#            debugpy.wait_for_client()

# --------------------------------------------------------------------
# -- Beginning of user code --
dd.Data.Root.Clear()
Group = dd.Data.Root.ChannelGroups.Add("Test")
ValChannel = dd.Data.Root.ChannelGroups.Add("Values")
AveChannel = dd.Data.Root.ChannelGroups.Add("Averages")
dd.ChnLinGen(ValChannel, 1, 9, 9)
dd.ChnSmooth(ValChannel, AveChannel, 2, "leftOnly")

[Brad_Turpin · Accepted solution]

Hi josh,

When you create new channels, you have to tell DIAdem what datatype that channel will be and what group it should show up in.  Your code creates 3 groups and 0 channels, but then tries to use two of the group variables as channel variables.  Here's the python equivalent of what I sent you earlier:

dd.Data.Root.Clear()
Group = dd.Data.Root.ChannelGroups.Add("Test")
ValChannel = Group.Channels.Add("Values", dd.DataTypeChnFloat64)
AveChannel = Group.Channels.Add("Averages", dd.DataTypeChnFloat64)
dd.ChnLinGen(ValChannel, 1, 9, 9)
dd.ChnSmooth(ValChannel, AveChannel, 2, "leftOnly")

The only way I know of to read data arrays directly into python variables from a TDMS channel is to use the nptdms module, but NI does not manage that code and can not vouch for its accuracy or robustness.

If you want to use the DIAdem commands, though, you HAVE to load the data into the Data Portal, because all DIAdem commands operate on channels there by reference.  You can always retrieve the array from a channel you've loaded with the GetValuesBlock() command:

AveArray = AveChannel.GetValuesBlock()

Brad Turpin

Principal Technical Support Engineer

NI

[ebenellis]

Hi Brad,

Thanks for the solution!

As an extension to the solution is it possible to edit this script to run this on multiple tdms files to achieve the below task:

1. Add moving average channel AveChannel

2. Calculate the peak & average of the below channel

AveChannel 

3. Store the value in a file

4. Repeat the same for each file

[Brad_Turpin]

Hi josh,

The short answer is "yes".  I recommend that you create a working group to hold your overall results, with a "Peak" channel and an "Average" channel and any other channels to identify each row's source such as a "File Name" channel.  Then at the end of the script you can export that working group to a CSV or Excel file or whatever.

So, what part of this is challenging you?

A) Multiple file selection (all in one directory, file dialog, etc.)

B) Looping over each file

C) Calculating Peak and Average of each moving average channel

D) Exporting the working group to a CSV or Excel or other file format

Brad Turpin

Principal Technical Support Engineer

NI

[ebenellis]

Hi Brad,

The challenging part is exporting the peaks(PeakXp, PeakYp) generated to a CSV/excel

I am using the below formula to determine the peak, could you please suggest a method to store these values into a csv/excel?

dd.ChnResult = dd.ChnPeakFind("[1]/Values", "[1]/Averages", "/PeakX", "/PeakY", 1, "Max.Peaks", "Amplitude")

[Brad_Turpin]

Hi josh,

I vastly prefer to use object variables to keep track of the groups and channels I use in a script, so please humor me when I include that approach in answer to your question below.  DIAdem has multiple output methods to create an ASCII or Excel file from channels in the Data Portal.  Here are the simplest two.  Let me know if you want to tailor this some:

DataFilePath = dd.ProgramDrv + r"Examples\Data\Example_data.tdm"
PeakFilePath = dd.CurrentScriptPath + "Peak Output"
dd.Data.Root.Clear()
dd.DataFileLoadSel(DataFilePath, "TDM", "[1]/*")
DataGroup = dd.Data.Root.ChannelGroups(1)
PeakGroup = dd.Data.Root.ChannelGroups.Add("Peaks Found")
XDataChannel = DataGroup.Channels("Time")
YDataChannel = DataGroup.Channels("Speed")
XPeakChannel = PeakGroup.Channels.Add("PeakX", dd.DataTypeChnFloat64)
YPeakChannel = PeakGroup.Channels.Add("PeakY", dd.DataTypeChnFloat64)
dd.ChnPeakFind(XDataChannel, YDataChannel, XPeakChannel, YPeakChannel, 1, "Max.Peaks", "Amplitude")

#dd.DataFileSaveSel(PeakFilePath + ".csv", "CSV", PeakGroup.Channels)
dd.DataFileSaveSel(PeakFilePath + ".xlsx", "ExcelTdmExport", PeakGroup.Channels)

Brad Turpin

Principal Technical Support Engineer

NI