Best practice: multiple frequency events

[ehudson]

Hi,

I am reading data at a set rate (e.g. 100 Hz) and want to do something with it at different rates.  For example, I might want to plot it at 10 Hz, write it to one file at 1 Hz and to a different file at 0.1 Hz.  What is the best way to do this?  Right now I have the read in a while loop and then calculate the iterator (i) modulo the ratio of the frequencies, only taking an action if the modulo is 0.  This works (running every N, M... cycles), but seems a little clunky and begins to fill up real estate if I have a lot of different frequenices that I want to do different things at. 

Ideally I think I'd like to have some event handler trigger (or some occurrence fire) every N, M, ... cycles but I don't see a clean way to make this happen.  I also thought of using multiple timing loops, each with their own time base, but don't know if this has drawbacks.  Either way I guess I'd have to get the data through a local variable which seems broadly discouraged. 

Thanks for any advice,
Eric

[falkpl]

You might consider a queued state machine with a common case to determine timeout conditions or you can do a producer (DAQ read) multiple consumer loops. The first architecture is easy to implement and scale but can cause buffer overruns in the DAQ state) the second will let you balance the processor demand and have better timing through multithreading but is a little more difficult to grasp conceptually.

Paul F.

[ehudson]

Hi Paul,

Thanks for your comments.  I'm not sure how the queued state machine solves the problem.  Where does the timing come in that tells the consumers "OK it's your turn?" 

For the single producer/multiple consumer I guess this is what I meant by having multiple timed loops (with the producer running at its rate and each consumer running at its own, different, rate).  The question then arises how to get data between the two.  It seems that a queue isn't appropriate because the consumers couldn't be responsible for dequeuing (if one did, the others might not have the data they needed).  I guess that I could have the producer both queue and dequeue, leaving a constant amount of data in the queue (the last N seconds).  Or I could do the same thing in a local or global variable, an array that the producer is constantly popping the front off of and appending a new end to.  Any idea which is the most efficient?

Thanks again,
Eric

[Ben]

I would lean toward the single producer/multiple consumer.

Use a seperate queue for each consumer.

Ben

[ehudson]

Hi Ben,

I meant to ask about that.  The thing is, each consumer is doing something with the same data.  Different amounts of it at different times, but all from the same source.  So when you say a separate queue for each consumer, wouldn't I be duplicating the data needlessly?

Thanks,
Eric

[Ben]

"wouldn't I be duplicating the data needlessly?" (emphesis added)

The answer depends on how you define "needlessly".

I once wrote dual ported round robin buffer that used a single buffer to store data going to multiple consumers. Worked great and I re-used it multiple time.

Boy was that baby complicated!

It was only dual ported.

In the end I still had to move tht data out of the buffer into wires in the consumers.

But that was before polymorhic queues were available. Queues are not only very fast but efficient as well. Dpending on how you code your VI, LV can actually use the buffer that stores the queue element when you use the data. It can do this because when you dequeue an element, LV knows the queue will not use that buffer anymore.

So yes if you are counting bytes, the queue approach is "wasteful".

If you are counting how long it takes for you to move onto the next feature, then worrying about bytes may be a needless concern.

Ben

Message Edited by Ben on 01-02-2007 03:42 PM

[Ben]

Other thoughts;

Another advantage of the multiple queue approach helps when the dat set becomes very large. LV want to use contiguous blocks of memory for its buffers. This will be come an issue if I use a single buffer to store the data for the slowest process since the slowest process will dicatate the size of the buffer.

On the other hand...

Using queues the update each get stored in multiple smaller buffers so the demand for large contiguous block of memory is reduced.

Now the other ideas:

The Queued state machine may not work due to the fact that the slowest process will effectively hang the other processess if it take more time to complete than is allowed for the fastest process.

Furthermore:

If after all of the above you do not want to use the queues and want to develop a multiported round-robbin buffer...

Then I'll be happy to offer what ever help I can.

Ben

[falkpl]

I agree with ben the seperate thread is a nice solution but will use a moderate amount of RAM, one copy for each consumer.  This is not a problem when sampling at 100 Hz where you will only be using ~4KB/sec pwe each thread.  The other solution is to use a single array to store the data in the producer loop and a write index.  These are within the local scope of the consumer loops, each consumer will just need their own read index.  You will however need to lock the local during read/write acess (semaphore) and make sure that you have a test to meke sure that you do now read past the write or write past the end of the buffer.  As you see you MIGHT save some Memory but I doubt it is worth the effort since the Labview queues are very well programmed (one of my favorate VIs to use).  The queue does all of the hard work for concurent programming, it will simplify your architecture to many single producer/consumer subproblems and allows for great scalability in your code. That is add a new consumer, just add a loop and a queue (an array of named queues will help).  Good luck.

Paul F

[falkpl]

This is a great question since this is a very common design pattern for labview (DAQ systems).