Continuous filtering

[kaba2005]

Hmmm... I still don't know what's going on...

Your code is not a filter. It's just cutting off a number of samples resp. setting them to zero.

And a delay ....?

[Gigi_31]

Yes i'am still stuck into it.

Using feedback node was a good idea at first but as you understood, i have to filter with a median filter afterwards and i got my first samples destroyed (i mean remplaced by 0). I did a workaround in order to save my last previous sample to replace all 0 by my last samples.

Feedback node is the closest solution so far to get what i want. I am not happy about this solution so if you have another idea, please share it 🙂

[Bob_Schor]

Hello, Gigi.  I'm going to wear my "Professor" hat and make two comments that address two of your recent posts.

First, you talked about acquiring data as an Array of Waveforms, and not finding a Filter function that takes a Waveform as an input.

A Waveform is simply a Cluster consisting of an Array of Data and two "timing" parameters that specify the Start Time (t0) of the data and the time interval (dt) between the samples (the third component, Y, is the Array of Data itself).  When you acquire data, you usually do it in "chunks", i.e. "Sample 1000 points at 1 kHz", which gives you an Array of data "every so often".

If you want to filter a Waveform, you need to extract the Data (Y) and filter it.  You can put the filtered data back into the Waveform (or build another one, called "Filtered Waveform" using the filtered Y array), but you may want to think about how you make the Filter "cross the boundaries" of the chopped-up "Array of Array of samples" (each Waveform in the Array is an Array of Samples).  This isn't too difficult, but you may want to take pencil and paper and work out how you'd like to do this.

The second thing is implementing a "Delay Line".  Your method certainly works, but it is a little unwieldy, possibly not that clear at first glance (particularly to someone new to LabVIEW), and would be really tedious if you wanted a delay of 100.  There's something that Electrical Engineers call a "Delay Line", where each "tick" data move from one position to the next.  We can implement a Delay Line of size N as an N-element Array.  Consider putting a new Point in Position 0, shifting everything else to the right by 1, and taking the element that "falls of the end", i.e. the element at position N-1, as the output.  What a lot of work, right?  So we do a trick -- we leave the data in place, and shift the insertion point.  Suppose we are starting out, with nothing in the Array, and our "Insertion Point" (IP) at 0.  We get a new point, we output the point at the IP, and put our new point into the Array at the IP.  We then increase the IP by one, so the next time, we use the next point.  So we'll start taking out the points at 0, at 1, at 2, and putting new points in 0, 1, 2.  What happens when we've inserted N points (into positions 0 to N-1) -- what is the "next" point?  I hope it's obvious to you that it is Point 0 again.  Is there a general "rule" for incrementing IP?  Yes, IP = (IP + 1) mod N.

So you can implement this very easily and it will run very quickly, no matter how long a delay you need.  It is an interesting exercise to figure out how to build this.  You need an Initialization Step (look up the First Call function to help with this) where you create the N-element Array of zeros and initialize IP to 0, and need to "remember" the Array and IP from call to call (I recommend writing this as a sub-VI called something like "Delay Line").

Bob Schor

[Gigi_31]

Hello Bob,

Thank you for your answer.

I am not sure about the benefit of the description of how to filter a waveform but that's allright, it might help some people i guess.

Yes i talked about acquiring data as an array of waveforms and that's what my application does. (not the issue at all)

My problem is not how to extract Data (Y) from a waveform either.

I want to filter my channels individually (with a median filter for instance) but i want this filter to incorporate a delay line set up by a number of samples.

I posted here because I have a lack of knowledge about filter and signal processing and i don't know which filter will suit me.

Best Regards

[Bob_Schor]

There are two issues that are "different, but related".  One is the notion of a Filter, which can be considered (in the digital, discrete world) as a function that operates on an array of data, and the other is dealing with "chopped up" data, i.e. how to deal with a lot of "continuous" data broken up into smaller pieces, i.e. the representation of an unspecified amount of data as an array of a representation that includes a fixed array of data (I'm describing your Array of Waveforms).

First, I hope you realize that the Delay Line is just another Filter -- it takes all of your data and shifts it N "positions" later (presumably filling the first N spaces with 0 or some other initial value).  So at some point, even for a simple Delay Line, you need to confront the "Array-of-Arrays" problem.

Example with Delays -- Let's make a Delay of 10 samples, and assume you have an Array of Waveforms where the Waveform is an Array of 1000 samples.  You initialize the Delay Line to 0, then start feeding it the first 1000 samples.  You'll get 10 zeros, the Sample 0, Sample 1, … Sample 989 (with Samples 990 .. 999 still "inside" the Delay Line).  You now start feeding in the second Array of 1000 samples -- works just fine.  The point is your filter works with a finite sample (here 10) and you are putting points in, applying the filter, then putting more points in.

All filters have a "width" (which, indeed, might be infinite, but rarely are …).  You are talking about a Median filter -- what do you mean by that?  Suppose you have a sample of 1000 points -- are you replacing all 1000 points by a single point, the median of the sample?  Are you replacing it with 1000 points all being the median?  Are you taking the median of 10 points and "sliding" this median down the 1000 points?  It's unclear what you mean, but I hope you see the notion of potentially "sliding" your filter down the data (as we did with the Delay Line, and in the final discussion of the Median Filter).

Sometimes Filters can be used independently.  Depending how you implement your Median Filter (you might look at, or think about, how Point-by-Point functions are written), you should be able to pass your data through a Delay Line and a Median Filter sequentially, either one coming first.

If you decide to take a crack at developing these functions, I strongly suggest that you develop them as "stand-alone" functions, and use the Waveform Generation functions to manufacture "known" data that you can feed in to your filter attempt and see how it all works out.  I might try this exercise for myself ...

Bob Schor

[Gigi_31]

Hello Bob,

Thank you for your comments.

I did some code in order to show you the behaviour of my filter so far. You'll find it attached.

I am not sure this is the best way to do it but this is it so far.

Best Regards.

[Bob_Schor]

Not bad!  The one "gotcha" is adding additional Arrays (you never need 2D arrays, and can also "get rid of" the 2D "auxilliary" Dbl Array) which I think you are using to initialize your home-made "Point-by-Point" Delay Line.

Do you remember my talking about the Delay Line as "just another Point-by-Point Filter" that could be put "in series" with the Point-by-Point Median Filter?  Here is a re-worked TEST Filter Chan function that does that (the code, and the other changes to your code, is including in the attached ZIP file).

Filter Chan (BS)

Instead of an Array of Waveforms (which you seem to use for "initialization" purposes, not needed here), I bring the un-arrayisized Waveform In and Out.  I renamed Nb samples to Delay (samples), as it is not related to the parameter "10" for the Median Filter.  The funny-looking Structure inside the Error Case Statement is an "In-place Element", in this case a Waveform Unbundle/Bundle that conveniently gives you access to the Y array (and bundles it up at the end).  Inside the For Loop, I pass each point through the Delay Line (which delays it by the amount you specify when you first call it), then passes the delayed waveform to the Median Filter.  Note you could Median Filter first, then Delay -- I'm pretty sure these two operations are commutative.

Look at the way the Delay Line works.  It uses a Functional Global Variable design, as does NI's Point-by-Point Median Filter (though it may be harder to see this -- try opening the Median Filter and see if you can understand why and how it works).

Hope this is useful for you.

Bob Schor

[Gigi_31]

Thank you Bob for this answer.

Can you share what you just send with LabVIEW 2016 version ?

I am going to dig in Median filter FGV while waiting for 2016 version.

Best Regards

[Bob_Schor]

Oops!  My bad, I thought I was doing this in LabVIEW 2016, but LabVIEW 2017 was also open, and "grabbed" it.  Here is everything in LabVIEW 2016 ...

Bob Schor

[Gigi_31]

Thanks sharing this code, it was instructive.

In my main application i got an array of waveforms that i have to filter each one with a delay parameter.(1-50) So input/outputs were OK.

I think your method is really interesting but i will modify your FGV in order to add a RAZ state which will clear out FGV memory when starting a new array index of waveform.

However there is something i cannot understand, Over 30 Delay (samples), i got weird filtering which is not corresponding to data. (Sware wave mode). See attached picture.