DaqMXWrite error -200018 using nidaqmx python library

[JaredBasis]

Setup is using an NI9263 Analog Output card in a cDaq9178 chassis.

I am trying to output 4 channels of analog output data each at 100kS/sec to this card using the python nidaqmx library. Sample code:

       self.output_channels = np.zeros(
            (len(self.device.ao_physical_chans), self.SAMPLE_COUNT),
            np.float64,
        )

        self.task_name = f"{self.device_name}_analog_output_task"
        self.create_task(self.task_name)
       
        # Add analog output channels to the task for each channel
        for channel in self.device.ao_physical_chans:
            self.task.ao_channels.add_ao_voltage_chan(
                channel.name, min_val=self.MINIMUM_VOLTAGE, max_val=self.MAXIMUM_VOLTAGE
            )

        self.task.timing.cfg_samp_clk_timing(self.SAMPLING_RATE, sample_mode=AcquisitionType.CONTINUOUS, samps_per_chan=self.SAMPLE_COUNT)
        self.task.channels.ao_data_xfer_req_cond = OutputDataTransferCondition.ON_BOARD_MEMORY_LESS_THAN_FULL
        self.task.channels.ao_data_xfer_mech = DataTransferActiveTransferMode.DMA
       
        self.task.simultaneous_ao_enable = True
        self.task.out_stream.regen_mode = RegenerationMode.DONT_ALLOW_REGENERATION
       
        # Initialize StreamWriter
        self.stream_writer = AnalogMultiChannelWriter(self.task.out_stream, auto_start=False)

        # Initial write, defines the buffer size - this must be done before registering the callback
        self.chunk_size = 1000
        self.num_chunks = int(self.SAMPLE_COUNT / self.chunk_size)  # this must be a whole number
        self.current_chunk = 0
        a = self.output_channels[:,self.current_chunk*self.chunk_size:(self.current_chunk+1)*self.chunk_size]
        self.stream_writer.write_many_sample(np.array(a))

        def write_callback(task_handle, every_n_samples_event_type,
                           number_of_samples, callback_data):
      
            a = self.output_channels[:,self.current_chunk*self.chunk_size:(self.current_chunk+1)*self.chunk_size]
            self.stream_writer.write_many_sample(np.array(a))
            self.current_chunk += 1
            if self.current_chunk >= self.num_chunks:
                self.current_chunk = 0
 
            return 0  # callback must return an integer, otherwise callback throws a bunch of errors

        self.task.register_every_n_samples_transferred_from_buffer_event(self.chunk_size, write_callback)
        self.task.start()

This works well except for intermittent errors as follows:

nidaqmx.errors.DaqWriteError: DAC conversion attempted before data to be converted was available.
Decrease the output frequency to increase the period between DAC conversions, or reduce the size of your output buffer in order to write data more often. If you are using an external clock, check  your signal for the presence of noise or glitches.

Task Name: cDAQ1Mod4_analog_output_task

Status Code: -200018

This error happens after anywhere from 10 seconds to 15 minutes. Is there a setting or configuration that I can use to stop this from happening. The system seems sufficiently fast to stream this amount of data, the CPU running this is only at 33% utilisation.

I tried setting the underflow behaviour but that doesn't seem to be supported by this equipment.

I have tried to set variously smaller buffer sizes as per the error message (see the chunking of data above) but haven't seen any improvement. It would also be great if the error message told me how many samples were ready instead of just not enough. This is difficult to find in the nidaqmx library.

I am also running this in an independent python process to limit interference with the callback but I'm unsure if the issue is there or in the NI library itself. I have put in various timing checks sometimes there is a longer delay than usual on the callback but oftentimes there isn't.

[ZYOng]

What is the execution time of self.stream_writer.write_many_sample?

How many sample points are you writing for each write call?

From my experience, DAQmx python API is pretty slow and usually takes more than 10 ms to execute. Usually, we would write the data at around 10Hz, even using LabVIEW, C or C# which execute way faster than Python. In that case, you should write at least 10k samples every DAQmx Write API to prevent data underflow.

[JaredBasis]

Getting the delta of time.time() calls before and after the write samples call is showing between 1-2ms call time but I don't think that's actually the write time as write_many_samples isn't blocking in Buffered mode. I'll do some digging and see if I can get a better idea on that.

The sample points are self.chunk_size * 4 channels. I've tried values from 250 through to 2000 so 1000-8000 samples. My use case is generating simulated mains power samples so 50Hz and was hoping that I could stream that fast (I'm updating the output_buffer via a socket that is rate limited to updating the buffer every 20ms). Perhaps I need to buffer the calls myself and accept a slightly worse latency.

The rest of the test infrastructure that I am integrating this into is in Python so I'm very keen not to need to migrate away from that.

Thanks you very much for the thoughts that's given me some more lines of attack to get this resolved.