UHD 4.1.0.5 X310 DMA FIFO modify size/mask

joshjoshjosh · ‎03-03-2022

Reference UHD version git commit hashes:

commit 7c0291adae76e026733cf1939f5920c9fe64967b (HEAD, tag: v3.14.1.1.L, origin/UHD-3.14.L)
commit 6bd0be9cda5db97081e4f3ee3127c45eed21239c (HEAD, tag: v4.1.0.5, origin/UHD-4.1)

The default UHD 3.14.1.1 X310 HG FPGA image contains a DMA FIFO RFNoC block. This FIFO size can be configured via the device3 interface:

uhd::rfnoc::dma_fifo_block_ctrl::resize

The default UHD 4.1.0.5 X310 HG FPGA image does not contain the DMA FIFO RFNoC block. We are able to add that block back into the FPGA image. The interface for that RFNoC block (in UHD 4.1.0.5):

uhd::rfnoc::dmafifo_block_contro

Does not have that same interface to resize the FIFO size. The dmafifo_block_control uses dma_fifo_core_3000 class to make the calls to peek/poke FPGA registers. The dma_fifo_core_3000 class has functions: set_addr_base, set_addr_mask

I added a new public function to dma_fifo_core_3000 that provides the "resize" feature similar to what was provided in UHD 3.14:

void resize(const uint64_t base_addr, const uint64_t size)
{
    // This error handling was pulled from UHD 3.14.1.1 DMA FIFO block code.
    if (size < 4096) throw uhd::runtime_error("DMA FIFO must be larger than 4KiB");
    uint64_t size_mask = size - 1;
    if (size & size_mask)
    {
        std::stringstream ss;
        ss << "DMA FIFO size must be a power of 2 - "
           << "base_addr: " << base_addr << ", size: " << size;
        throw uhd::runtime_error(ss.str());
    }
    set_addr_base(base_addr);
    set_addr_mask(~size_mask);
}

In order to expose that "resize" function, I created a property in the class uhd::rfnoc::dmafifo_block_control_impl:

RFNOC_BLOCK_CONSTRUCTOR(dmafifo_block_control)
{
    ...
    for (size_t i = 0; i < get_num_input_ports(); i++) {
        ...
        uhd::fs_path base_addr_path;
        uhd::fs_path depth_path;

        get_tree()->create<uint64_t>(base_addr_path / "args" / i / "base_addr/value")
            .add_coerced_subscriber([this, i](const uint64_t addr_base) {
                    uint64_t size = get_depth(i);
                    RFNOC_LOG_INFO("Setting addr_base - fifo core[" << i << "] - "
                        "addr base: " << addr_base << ", "
                        "size: " << size);
                    resize(addr_base, size, i);
                    RFNOC_LOG_INFO("Get fifo core[" << i << "] - "
                        "addr base: " << get_base_addr(i) << ", "
                        "size: " << get_depth(i));
                });

        get_tree()->create<uint64_t>(depth_path / "args" / i / "depth/value")
            .add_coerced_subscriber([this, i](const uint64_t size) {
                    uint64_t addr_base = get_base_addr(i);
                    RFNOC_LOG_INFO("Setting size - fifo core[" << i << "] - "
                        "addr base: " << addr_base << ", "
                        "size: " << size);
                    resize(addr_base, size, i);
                    RFNOC_LOG_INFO("Get fifo core[" << i << "] - "
                        "addr base: " << get_base_addr(i) << ", "
                        "size: " << get_depth(i));
                });
    }
}
...
void resize(const uint64_t addr_base, const uint64_t size, const size_t chan)
{
    UHD_ASSERT_THROW(chan < _fifo_cores.size());

    boost::lock_guard<boost::mutex> lock(_config_mutex);
    if (chan < _fifo_cores.size()) {
        _fifo_cores[chan]->resize(addr_base, size);
    }
}

In my application I can successfully access that property with something like:

// multi_usrp - uhd::usrp::multi_usrp::sptr multi_usrp = uhd::usrp::multi_usrp::make(args);

uhd::property_tree::sptr tree = multi_usrp->get_tree();
uhd::fs_path depthValuePath =
        uhd::fs_path("/blocks/0/DmaFIFO#0/args") / 0 / "depth/value";

uhd::property<uint64_t>& depthValueProperty =
        tree->access<uint64_t>(depthValuePath);

// 32 k bytes -> 0x8000 -> (0x8000 - 1) = 0x7fff or '0b111111111111111'
depthValueProperty.set(1024 * 32);

Finally to the problem. When I set that property, I am not able to pass IQ data through from the SEP to the radio block. Here is a paired down version of the X310 FPGA rfnoc image core .yml:

...
noc_blocks:
    ...
    fifo0:
      block_desc: 'axi_ram_fifo_2x64.yml'
      parameters:
        NUM_PORTS: 2
        MEM_DATA_W: 64
        MEM_ADDR_W: 30
        FIFO_ADDR_BASE: "{30'h00200000, 30'h00000000}"
        FIFO_ADDR_MASK: "{30'h001fffff, 30'h001fffff}"
        MEM_CLK_RATE: "300e6"
connections:
    ...
    - { srcblk: ep0,    srcport: out0,  dstblk: fifo0,  dstport: in_0 }
    - { srcblk: fifo0,  srcport: out_0, dstblk: duc0,   dstport: in_0 }
    - { srcblk: duc0,   srcport: out_0, dstblk: radio0, dstport: in_0 }
    ...
...

I wrote a python script that reads that

/blocks/0/DmaFIFO#0/args/0/depth/value

Property which returns 2 MB (0x200000) as I would expect. When I write something like 1 MB (0x100000) to that property and read it back again I get the value 0x3fe01000. Since the code accessing this property adds one, if I subtract one from that value I get 0x3fe00fff.

Are you able to:

See what I may have overlooked, missed, got wrong
Should MEM_ADDR_W be 32 and not 30?

I thought I read somewhere the 64 bit register that controls the DMA FIFO offset and mask (size), the lower and upper 32 bits need to be written in a particular order but I assumed the UHD peek32/poke32 functions were correct.

USRP Software Radio

UHD 4.1.0.5 X310 DMA FIFO modify size/mask

UHD 4.1.0.5 X310 DMA FIFO modify size/mask