CounterOutput callback causes high CPU usage around every 3 seconds... bug?

[pgruebele]

Hi,

I am using the NIDaq .Net library (latest 871 version) to implement the following callback on a 6602 PCI timer board. The method taskTriggerSpin_CounterOutput simply returns (it does nothing):

    taskTriggerSpin = new Task();
    taskTriggerSpin.COChannels.CreatePulseChannelFrequency("/Dev1/ctr3", "CameraSpinTrigger", COPulseFrequencyUnits.Hertz, COPulseIdleState.Low, 0.0, 1000, .5);
    taskTriggerSpin.COChannels[0].CounterTimebaseSource = "/Dev1/20MHzTimebase";
    taskTriggerSpin.Timing.ConfigureImplicit(SampleQuantityMode.ContinuousSamples);
    // call back will happen on NI task thread on every trigger edge (rising and falling) so that we can set new ROI when necessary. 
    taskTriggerSpin.SynchronizeCallbacks = false;
    taskTriggerSpin.CounterOutput += new CounterOutputEventHandler(taskTriggerSpin_CounterOutput);

So, all I am doing is generating a 1Khz pulse and calling the callback at a frequency of 2000Hz.  The problem lies with CPU usage.  CPU usage is very low (2-8%) except for short bursts of around 80% every 5 seconds (I am talking about CPU usage of the single core that is running these threads, processor has 3 more cores with ~0% usage).  Since these callbacks are happening continuously and CPU usage is 3-8% most fo the time, the NI libraries must do something weird every few seconds to cause this spike in CPU usage.  Further analysis shows that there are 2 threads originating from nipalut.dll which are causing these spikes in CPU usage.

It appears to me that this CPU spike is a bug since the callbacks themselves appear to be only causing 3-8% cpu usage.  Also, why are there 2 threads causing this spike when I only allocate one callback?  The number of hardware interrupts also appear to be spiking at the same time as CPU usage spikes.  This also does not make sense.

Is there any way to reduce the callbacks to only happen on every rising edge of the trigger signal (at 1000Hz)?

On another note: I am measuring callback jitter and find that exactly EVERY 1 second, the callback is delayed by around 2ms, followed by several faster callbacks to make up for the long delay.  This happens every second although my process is realtime and nothing else in the process is consuming CPU.  I need these callbacks to be more precise for my application and the timer board + NIDaq software is performing well below expectation regarding this and the CPU issue.  These callbacks should be happening as a result of hardware interrupts and should therefore be more precise.....

Would this jitter issue be resolved by using the C interface to perform these callbacks?  Can I register callbacks using C (native) but continue to allocate the task from .net?  How would I do this?

Sorry for all the questions, but these issues are causing the timer board and NIDaq software to be nonusable for my application....

Please advise.

Thanks

Philip Gruebele

[pgruebele]

Update:

I rewrote the code for this timer and callback using NI-DAQmx C library.  Now the jitter problem is completely gone (great!), but the CPU issue is even worse.  The following code utilizes 100% CPU even with the 10Hz counter!!!!  Am I missing something here?  I tried all the options for interrupt based processing instead of busy polling to no avail.  This is just unacceptable.  These days one can't even deliver an app that is permanently running at 100% cpu without raising some eyebrows...

I must be missing something...

Please help,

Philip

static int StaticTriggerCallback(TaskHandle taskHandle, int32 signalID, void *callbackData)
{
return 0;
}

int TTCameras::StartTimer(void)
{
 int         error=0;

CTTEyes::Log("Starting Spin Trigger", false, 1);

 spintotalTriggerPulsesCallbacks = 0;
 spinROITimeStart = 1;
 spinROITimeEnd = 0;
 threadHandles[0] = threadHandles[1] = threadHandles[2] = threadHandles[3] = 0;

 // Configure DAQmx ct3 as our spin camera trigger.  1000Hz
 DAQmxErrChk (DAQmxCreateTask("", &taskHandle));
 DAQmxErrChk (DAQmxCreateCOPulseChanFreq(taskHandle, "/Dev1/ctr3", "CameraSpinTrigger", DAQmx_Val_Hz, DAQmx_Val_Low, 0.0, 10.0, 0.5));
 // /Dev1/PFI1 rising edge will enable this trigger so that it will start at the same time as /Dev1/ctr0
// DAQmxErrChk (DAQmxCfgDigEdgeStartTrig(taskHandle,"/Dev1/PFI1", DAQmx_Val_Rising));
 // continuous samples
 DAQmxErrChk (DAQmxCfgImplicitTiming(taskHandle, DAQmx_Val_ContSamps, 1000));
 DAQmxErrChk (DAQmxSetReadWaitMode(taskHandle, DAQmx_Val_WaitForInterrupt));
 DAQmxErrChk (DAQmxSetRealTimeWaitForNextSampClkWaitMode(taskHandle, DAQmx_Val_WaitForInterrupt));

 // register callback which will occur each time counter reaches zero and toggles output trigger signal.
 DAQmxErrChk (DAQmxRegisterSignalEvent(taskHandle, DAQmx_Val_CounterOutputEvent, 0, (DAQmxSignalEventCallbackPtr)StaticTriggerCallback, NULL));

 DAQmxErrChk (DAQmxTaskControl(taskHandle, DAQmx_Val_Task_Commit));

 double freqSpin;
 DAQmxErrChk (DAQmxGetCOPulseFreq(taskHandle, "CameraSpinTrigger", &freqSpin));
 TTCameras::periodSpin = 1.0 / freqSpin;

 DAQmxErrChk (DAQmxStartTask(taskHandle));

Error:
 if (DAQmxFailed(error))
 {
  char str[1000];
  int l = sprintf_s(str, 1000, "Error: StartTimer(), DAQmxFailed=.");
  DAQmxGetExtendedErrorInfo(&str[l], 1000 - l);
  CTTEyes::Log(str, true, 10);

  StopTimer();
 }

 return 0;
}

[PBear]

Hi pgruebele,

Unless using interrupts is required by your program, I would recommend removing the lines:

 DAQmxErrChk (DAQmxSetReadWaitMode(taskHandle, DAQmx_Val_WaitForInterrupt));
 DAQmxErrChk (DAQmxSetRealTimeWaitForNextSampClkWaitMode(taskHandle, DAQmx_Val_WaitForInterrupt));

Also, it's not apparent that you're reading the counter registers (except for the event the callback is monitoring) during your Counter Output task. Therefore setting the ReadWaitMode propery to an interrupt is going to hinder your performance as you are polling your board even though you're not reading from it. The perfomance would be the same if you read or not but since you're not then there is no reason to set DAQmx_Val_WaitForInterrupt.

Try removing those two lines in order to limit your CPU  usage.

[pgruebele]

Thanks for the response PBear.

I removed those 2 lines of code and NIDaq still takes up 100% CPU, so your suggestion does not work.  The only reason I put them there in the first place is in order to try to get NIDaq to use interrupt based processing rather than busy polling.

My application requires that I generate TTL level trigger pulses and at the rising edge of each trigger pulse, I also need to execute some very short C code. This code should run within .5ms of the trigger edge.  This should be no problem for a modern CPU. This is why I am using DAQmx_Val_CounterOutputEvent.  However, I cannot have NIDaq implement this using busy polling.  It is simply not acceptable to be running at 100% CPU for several reasons.  The documentation and your support site talk about interrupt based processing, and the hardware is obviously capable of it.

So, the 2 questions I have are:

1. What do I have to do in order to have NIDaq use interrupt based processing instead of busy polling (for the DAQmx_Val_CounterOutputEvent callbacks in my application)?

2. In addition to (1), can I make it so that my callback only gets called on the rising edge of the trigger signal instead of both the rising and falling edges?

Thanks in advance

Philip

[Chris_D]

Hey Philip,

1) I don't think switching to interrupts is going to help your problem. I set up an application very similiar to yours using DAQmxRegisterSignalEvent(taskHandle, DAQmx_Val_CounterOutputEvent, 0, (DAQmxSignalEventCallbackPtr)StaticTriggerCallback, NULL); and it did not take up anywhere near 100% of my CPU usage. When you comment out the DAQmxRegisterSignalEvent line does it still rail to 100%? Also, I would suggest running an example from the NI examples. I used the DigPulseTrain-Cont.c example. Try running this first to see if it rails to a 100%. If not, add in the DAQmxRegisterSignalEvent and callback function to see if it still does the same. This will help us determine if some other part of your code is causing the problem.

2) I am not sure why the event occurs on both the rising and falling edge. According to the NI-DAQ help, the Counter Output Event should occur when the counter reaches terminal count. While you are doing the tests I posted above I will do some more research on this.


Message Edited by Chris_D on 07-07-2008 03:21 PM

[pgruebele]

Hi Chris,

1) I checked out the sample you mention and you're right - it does not consume 100% cpu - even when I add DAQmxRegisterSignalEvent to the code and make it otherwise identical to my code.  When I removed DAQmxRegisterSignalEvent  from my code, CPU went back to normal, but with it there, it was 100%.  Very strange.  Anyway, I have some tests I want to run but am in the middle of something else.  Hopefully tomorrow I can post the results of my tests.

2) My understanding is that the counters toggle their output when they reach terminal count.  Therefore, if you are using the counter to create a signal with a frequency of 10Hz, the counter will reach terminal count every 1/20th second in order to toggle the output and therefore the callback will be called at 20Hz.  Please correct me if I am wrong or if there is a way of setting up the counter and callback so that this does not happen.  I know for a fact that the callback is getting called at 20Hz since I measured it.

Thanks for your help.

Philip Gruebele

[pgruebele]

Also sorry for the duplicate posts. I thought since I am now doing this using c++ I should post to the appropriate forum.    Anyway, I am happy to pursue this issue with you in this forum 🙂

Philip

[Chris_D]

Hey Philip,

That's good to hear that we are making some progress into finding the problem. It does sound like there may be something else going on with your code. Is there anything else that is calling the callback? Just let me know the results of your tests.

I looked more into the Counter Output Event and you are correct about the toggle. When generating a pulse train, terminal count is reached when the pulse goes both high and low. It loads one register into the counter and counts down to terminal count, toggles low, and then loads the second register into the counter, counts down to terminal count, and then toggles high. The event will be fired on every toggle and there is no way to avoid this. However, in software you could set it up so that you only execute a piece of code on every other iteration.

[pgruebele]

Hi Chris.

This is very strange but now my application no longer has this problem either.  I did change some other third party software, so that must somehow have done it.  However, removing DAQmxRegisterSignalEvent previouly got rid of the 100% CPU even though the callback did nothing, so it was definitely enabling the callback that caused 100% CPU. Very strange.

Anyway, now it consumes reasonable amounts of CPU so I'll leave it at that.  The only thing for me to test now is the accuracy and jitter with which the callback gets called.  I can't test this right now but will post results as soon as I do.

BTW, you said that using interrupts would not help my situation (in terms of CPU usage).  Can you tell me what mechanism NIDaq is using to call this callback?  The output frequency of this counter will have to be around 1000Hz, meaning that the callback will be called at 2000Hz.  Since it is no longer using 100%, doesn't this mean that this must be done via interrupt based processing?  Busy polling would incur a lot of CPU usage no?  And sleep mode would not be possible due to the high frequency, no?

Thanks for your help

Philip Gruebele

[Chris_D]

Hey Philip,

The post below explains it very well. Let me know if you have anymore questions!

DAQmx does not use Windows interrupts???
http://forums.ni.com/ni/board/message?board.id=250&message.id=34907&requireLogin=False