how to synchronize multiple usrp rio (294x) devices

[rkossler1]

Hi,

I am trying to receive 4 channels synchronously using two USRP-294xR devices  (USRP RIO) and running the example program for multi-device receiving.   I frequently receive the error message "Synchronization failed".  If I run the example many times in a row, this error message occurs more often than not, but for those times that it does not occur, I am able to collect data which appears to be synchronous across the receive  channels.  So, in a way it is working for me, but with the annoying feature that I need to run the program several times in a row until I am able to avoid the "Synchronization failed" error.

My setup includes an Octoclock connected to the Ref In and PPS In connections of both USRPs.  In the example VI, I have selected  Ref In, but it is unclear to me what selection I should use for Trigger.  Possible options are "Software", "Immediate", "Digital Edge", "Rx Start Trigger", and "Tx Start Trigger".  I had assumed from my previous experience using these devices with the Ettus FPGA that I would select some option indicating that I wanted to  synchronize to the external PPS signal.  Since "External PPS" is not one of  the triggering options, I am confused as to how the synchronization works.

I reviewed the article below, but it is not clear to me whether or not I should be able to accomplish my goal without making any FPGA changes. Is it possible to synchronize multiple devices using the default FPGA and example program (i.e., RIO example, not the host VIs that use the Ettus FPGA image)?  If so, any suggestions to help me avoid this "Synchronization failed" error?

http://zone.ni.com/reference/en-XX/help/373380D-01/usrphelp/synchronization/

Thank you.

Rob

[Gato-Ray-d]

Have you checked out the example VI in the USRP palette called "niUSRP Ex Tx Multiple Synchronized Output (PPS Trig).vi"?

It connects PPS TRIG OUT to the USRP Export Signal function.

[brooksprumo]

A few system requirements:

• All USRPs must have a common reference clock
• All USRPs' PPS TRIG IN must be connected to a single 'master' USRP PPS TRIG OUT
• This includes the master USRP too

Common system implementation:

• Provide a common reference clock to all the USRPs
  • Set the OctoClock's PRIMARY REF to EXTERNAL
  • Connect a 10 MHz reference to the OctoClock's EXT 10 MHz INPUT
  • Connect the OctoClock's 10 MHz OUT to all the USRPs' REF INs
• Setup the trigger distribution network for the USRPs
  • Connect the OctoClock's PPS OUT to all the USRPs' PPS TRIG INs
  • Connect USRP1's PPS TRIG OUT to one of the 6674T's PFI ports
  • Connect a different 6674T PFI port to the OctoClock's EXT PPS INPUT
  • Use the 6674T's test panel to buffer/route the first PFI port (from the USRP) to the second PFI port (to the OctoClock)
  • Make sure the 6674T's threshold is lower than 1V (I recommend .7V) for the USRP1 connection

You may be asking about the 6674T. The output voltage of the USRP RIO is not strong enough to drive the OctoClock, so the 6674T is effectively being used as a schmitt trigger--feel free to replace the 6674T with another buffer/schmitt trigger of your choice.

Synchronization theory of operation:

Signal-based synchronization's goal is to make it to where the master USRP's trigger is "seen" by all devices at the exact same time (nearest Data Clock cycle). So, the Start Trigger you select for the Master is sent to all the other USRPs automatically; there's no special/new Start Trigger to select. The Master USRP also has an output for the number of clock cycles that elapse between when the Master USRP sees its trigger, and when all the USRPs (including the Master) actually realize the trigger.

Hopefully this helps some! Synchronization is super tough, so let me know what other questions you have.

[rkossler1]

I just looked at this example, but it is one that uses the "USRP" (Ettus based FPGA, Ethernet based communication) architecture rather than the "USRP RIO" (LabVIEW FPGA, PCIe based communication).  Is it possible to synchronize multiple 294x devices using the USRP RIO drivers?  If so, is it possible to do so without needing to make FPGA changes?

[rkossler1]

Thank you.

Regarding "A few system requirements" from your post, these are no problem.  We have both USRPs connected to an Octoclock using 10MHz and PPS.

Regarding "Common system implementation", it seems from your description that the "master" USRP sends a PPS output to the OctoClock which in turn forwards that PPS to the input of all USRPs (including "master").  I assume then that all USRPs will begin streaming upon this common trigger.  But, there are several things I don't understand...

1) Why do we need the master USRP to send a trigger to the Octoclock which is then forwarded as a common trigger to all USRPs?  Why can't the Octoclock's internal PPS signal be sent to the inputs of all USRPs to act as the common trigger?

2) Why does the example multi-device rx streaming VI "sometimes" work?  If I start this VI ten times in a row, it will show a synchronization error most times, but probably 1 or 2 of the times it will begin running and I have confirmed that the data is indeed sample aligned across all four channels.

3) What is the meaning of the "Start Trigger" options of "Software", "Immediate", "Digital Edge", "Rx Start Trigger", and "Tx Start Trigger"?  I have used the first three options but it does not seem to affect the operation of the program in any way that I can tell.  In all cases, I get the intermittent "Synchronization failed" error, but I believe that when it actually runs, the data is truly sample synchronous.

Rob

[brooksprumo]

---

@rkossler1 wrote:

Is it possible to synchronize multiple 294x devices using the USRP RIO drivers?

---

Yes. The link you posted earlier (http://zone.ni.com/reference/en-XX/help/373380D-01/usrphelp/synchronization/) is for USRP RIO in LabVIEW FPGA.

---

@rkossler1 wrote:

I just looked at this example, but it is one that uses the "USRP" (Ettus based FPGA, Ethernet based communication) architecture rather than the "USRP RIO" (LabVIEW FPGA, PCIe based communication).  Is it possible to synchronize multiple 294x devices using the USRP RIO drivers?  If so, is it possible to do so without needing to make FPGA changes?

---

Yes. The default Sample Project FPGA VI and bitfile includes synchronization support, so FPGA changes are *not* needed.

[brooksprumo]

---

@rkossler1 wrote:

Thank you.

Regarding "A few system requirements" from your post, these are no problem.  We have both USRPs connected to an Octoclock using 10MHz and PPS.

Regarding "Common system implementation", it seems from your description that the "master" USRP sends a PPS output to the OctoClock which in turn forwards that PPS to the input of all USRPs (including "master").  I assume then that all USRPs will begin streaming upon this common trigger.  But, there are several things I don't understand...

1) Why do we need the master USRP to send a trigger to the Octoclock which is then forwarded as a common trigger to all USRPs?  Why can't the Octoclock's internal PPS signal be sent to the inputs of all USRPs to act as the common trigger?

---

The USRP RIO synchronization in LabVIEW FPGA is fundamentally different from the NI-USRP host API with Ettus/UHD. USRP RIO is signal-based, and NI-USRP is time-based. With time-based synchronization, all the devices have 'time' programmed on the device, and then all the devices are told to 'go' at a specific time. With signal-based synchronization, there is no time. A single master device instructs all the devices to start, so the host only triggers the master, and then the master triggers all the devices.

With time-based synchronization, a PPS is provided to all the USRPs into their PPS TRIG IN port. The OctoClock generates a PPS and distributes it to all the devices.

With signal-based synchronization, the PPS TRIG ports are not used for PPS, but instead for general-purpose triggering (hence the unforunately-confusing PPS TRIG port name). The master must be connected to all the slaves, so the master has a trigger output to all the devices' trigger inputs. The OctoClock no longer generates a PPS. Instead, the OctoClock is a trigger distribution network that handles forwarding the master's trigger out to all the devices' trigger ins.

---

rkossler1 wrote:

2) Why does the example multi-device rx streaming VI "sometimes" work?  If I start this VI ten times in a row, it will show a synchronization error most times, but probably 1 or 2 of the times it will begin running and I have confirmed that the data is indeed sample aligned across all four channels.

---

For the NI-USRP host API, I'm not sure. If this is USRP RIO in LabVIEW FPGA, then it's likely due to the system configuration with wiring and the OctoClock's configuration.

---

rkossler1 wrote:

3) What is the meaning of the "Start Trigger" options of "Software", "Immediate", "Digital Edge", "Rx Start Trigger", and "Tx Start Trigger"?

---

The Start Triggers are all somewhat different:

Immediate: trigger as soon as the host run Initiate Stream

Software: trigger onces the host sends a start trigger after running Initiate Stream

Digital Edge: trigger based on the input AUX I/O 0 (by default)

Rx Start Trigger: This is only applicable for the Tx chain: it allows the Tx chain to use the same Start Trigger as the Rx chain, so that Rx and Tx start together.

Tx Start Trrigger: This is only applicable for the Rx chain: it allows the Rx chain to use the same Start Trigger as the Tx chain, so that Tx and Rx start together.

Great questions!

[rkossler1]

Thanks for the explanation.  I understand it quite a bit better now.

Since we didn't have a 6674T, I figured I might as well attempt to connect PPS out from one USRP RIO to the PPS in on both USRP RIOs.  I simply used a BNC-T to "distribute" the trigger.  This did not work - perhaps for the same reason you mentioned about the USRP PPS out not being strong enough to drive the Octoclock.  But, I also had another thought....  How do I know which USRP RIO is the "master"?  Is this something I need to designate in the software?  I arbitrarily chose to use "RIO0" as the unit for which I connected the PPS out to the BNC-T, but I'm not sure how the SW would know which USRP I chose to connect the cable to.

Rob

[brooksprumo · Accepted solution]

---

@rkossler1 wrote:

Since we didn't have a 6674T, I figured I might as well attempt to connect PPS out from one USRP RIO to the PPS in on both USRP RIOs.  I simply used a BNC-T to "distribute" the trigger.  This did not work - perhaps for the same reason you mentioned about the USRP PPS out not being strong enough to drive the Octoclock.  

---

Yep, that's exactly it. The USRP RIO doesn't output enough power on the PPS TRIG OUT line to drive two USRP RIOs' PPS TRIG IN lines, unfortunately 

---

@rkossler1 wrote:

How do I know which USRP RIO is the "master"?  Is this something I need to designate in the software?  I arbitrarily chose to use "RIO0" as the unit for which I connected the PPS out to the BNC-T, but I'm not sure how the SW would know which USRP I chose to connect the cable to.

---

The master USRP is the one outputting from its PPS TRIG OUT to all the USRPs' PPS TRIG IN. You then set this device as the *first* device in the RIO devices array for the multi-device synchronization VIs' front panel.

[Mikasa12111]

Hello,

I have read your response to the meaning of different start triggers.  Now I want to let devices start at the time I set using example VI ,'Tx and Rx streaming time(host)' . There is another ‘future time event’ trigger in this VI. What does this trigger mean? Can I use this trigger to  achieve my goal, or can I use other triggers? 

I hope you can give me some advice. Thank you.