Intermittent 2940R Startup Issues

[emonk]

I'm working on a USRP-2940R project with LV 2015 in Windows and I'm having an intermittent gain problem.  I'm attempting to read the frequency of ~1.25 GHZ RF bursts and triggering the receiver with coincident TTL pulses that I'm applying to the PPS input.  Most of the time it works well, but every 10th time or so that I start the USRP and run the project, there's a severe lack of gain in the receiver.  When this happens, only the input signal is affected; the LO spikes in the spectrum are unchanged.  Also, adjusting the DSP gain on the fly simply raises the noise level of the spectrum.  The only way I've found to fix it is to exit the project, power down the USRP and restart Windows, then all is well.  Until the next time it happens.

This issue affects both Rx inputs, happens with multiple 2940Rs and can be reproduced using the Rx Streaming example that ships with NI-USRP 15 (after recompiling the FPGA to use the PPS trigger).  So far I've only seen this while receiving pulses with a PPS trigger, never while receiving CW, so I'm thinking it may be a data synchronization issue on startup.  But I can't figure out how to prove or disprove that notion.  Or how to fix it. 

Has anyone else seen this?  Can anyone suggest something else I can look at to try to pinpoint the problem? 

Thanks and regards.

[Libbin_la_vida_loca]

Hi emonk,

It's strange that you need to restart your computer to get this to work. I understand that you've reproduced the issue on multiple USRPs, but is it also reproducible using another computer?

[emonk]

Thanks for the reply.

---

is it also reproducible using another computer?

---

Yes.  It happens on my development system (an i7) and my runtime system (a slightly slower i7).  It also happens both in development mode and runtime mode.  I'm communicating with the USRP via the PCIe bus, BTW.

Regards.

[Kimchi-Taco]

Hello,

To make sure that we don't have a synchronization issue have you tried using the 'niUSRP Ex Tx Multiple Synchronized Output (PPS Trig).vi'?

Below is a valuable resource regarding synchronization of multiple USRP 294x devices:

https://forums.ni.com/t5/USRP-Software-Radio/how-to-synchronize-multiple-usrp-rio-294x-devices/m-p/3...

Hope this helps.

Thanks,

[emonk]

---

@Kimchi-Taco wrote:

Below is a valuable resource regarding synchronization of multiple USRP 294x devices:

https://forums.ni.com/t5/USRP-Software-Radio/how-to-synchronize-multiple-usrp-rio-294x-devices/m-p/3...

---

Well, I'm only using one USRP, so it's not that.

The possible sync issue I was referring to was between the USRP and the PC.  I'm not at all sure how the PCIe transport mechanism works or how to tell if it isn't.

The proper IQ data is making it to the receive loop, it's just 30-40 dB down from where it should be.  Sometimes.

Thanks.

[Libbin_la_vida_loca]

Hi emonk,

A few troubleshooting questions:

- Is there anything that might be happening in the software that is creating a race condition and preventing the FPGA from executing the trigger response?

- After you receive a failure and before you reset it, does anything work on the device? Try repeating your code until you see the unexpected behavior. Once this happens, try running a basic example. Does it run as expected, or are there still gain problems? If a basic example runs as expected, go back to your code and try to run it again. What is the behavior in this case?

- How did you get the 30-40 db metric?  How much is the raw IQ data scaled down?

[emonk]

Hi, Libbin,

I assembled another system for testing and it does the same thing, only less often (my original prototype is now several states away undergoing evaluation).  Also, it appears that on this system restarting the USRP and host is not necessary to make it work right, merely restarting the runtime application (sometimes multiple times) fixes it. 

Here's what I've learned from playing with it:

- Upon failure, all FPGA functions not associated with reception work fine.  The TX channels are not affected but they are fixed and there's no TX streaming.

- I haven't had a chance to tweak and compile the basic examples for runtime testing but I will when my other projects ease up a bit.

- The received data I'm looking at is coming out out the FFT VI.  When all is well, the peak of the input burst is at -37 dBm with -15 dBm in.  This is well above the LO spike which is at about -48 dBm.  When the gain fails, the signal drops to -67 dBm and my sweep routine can't find it.

-Here's the only concrete symptom I've come up with:  I'm looking at the rx.status data coming out of the FPGA Rx Core VI.  When all is well, the "state" field stays at "Idle" (presumably because the register bus is too slow to reflect the changes in real time?).  When things are messed up, it toggles randomly between "Idle" and "Active", as if the PCIe bus has slowed or is glitching.  I'd like to hang a scope probe on the PCIe lanes to see what's different, but that's not as easy as it sounds...

I've tried adding delays in strategic places, patching the RTE, playing with different sample rates and sizes, changing FIFO sizes, coercing the gain, changing Rx channels and a lot of other stuff.  The issue remains.

One thing I haven't yet tried is using a DIO input for the trigger rather than the PPS input.  On the schematic they look identical, but are there any firmware differences in the way their inputs are handled?

Thanks.