Photon Counting, Raster Control, DAQmx

[bcborders]

Hi all,

Advance thank you for anyone who takes the time to help me on this! I've done a decent bit of LabView programming working on this project, but I feel I am still a novice when it comes to the nuances of data acquisition and hardware timing. Information has been difficult to come by, though the documentation and the forums have been helpful. I'm hoping someone can give me a bit of clarity on how make all the components synchronous going forward.

The project: a novel acousto-optical microscope to track proteins and measure fluorescence. Ideally, the sampling rate will be at least 10MHz.

The components:

     - NI sbRIO 9628 w/ RMC breakout board

     - PicoQuant PicoHarp 300 

     - Variable frequency driver to control laser

     - PI MicroMove stage controllers

     - Python script with lightweight Kalman filter to track proteins

Ideal functionality:

     1) Raster scan VI performs knight's tour while photon counter is running

     2) Upon detecting fluorescence (threshold is set experimentally), raster halts and passes last position to Kalman Filter python node. This conversion is done mathematically from Hz (the frequency driver input) to millimeters (micromove stage controllers)

     3) Photon counter continues to read fluorescence while python node predicts next protein position, moving stage controllers

     4) Once protein moves out of target window, kalman filter is halted and raster resumes 

Current working solution:

So this is a lot of moving parts. I have a working VI I created with a software based timing solution. Unfortunately, this has limited resolution (windows can only resolve ms). I will attach the VI for reference. This was previously built on a a PCEI board that does not support DAQmx or hardware timing

What I want to do next:
We procured a new sbRIO board to implement a hardware timed solution. I refactored the knight's tour and it is working well on the board. Now, I need to create a schema of DAQmx tasks to implement the same synchronization I roughly approximated using the software-timed VI.

There are a lot of moving parts, and I would greatly appreciate the expertise and knowledge of anyone who can guide/suggest ways to implement this as efficiently as humanly possible.

I have followed the documentation on how to create and modulate tasks, but then I found this thread:

https://forums.ni.com/t5/Counter-Timer/Photon-Counting-Time-Stamping-and-basic-help-with-how-DAQmx/t...

This person was trying to do something similar, albeit with more counters, and needed timestamps (which I do not). Nevertheless, this post made me realize my current understanding was inadequate for the task at hand.

Thanks for anyone who has read this far, and I greatly appreciate any and all advice and guidance

Dan

[Kevin_Price]

Yes, it IS a lot of moving parts.  I was very involved in the old thread you found, so I wanted to give you at least a token reply.  However, I'm fairly doubtful I'll be able to help a lot with several key elements that constitute those "moving parts."

I'm not versed in Python or FPGA or the dll calls you have for your PicoHarp instrument.  My "thing" is DAQmx, particularly counters and sync schemes.

Can you describe what you're trying to do another way?   What exactly needs hardware timing for real-time sync purposes and what merely needs hardware timing for sampling purposes?   What's your intent for coordinating the various parallel loops I see in your code?  I couldn't seem to decipher any scheme for what would make them work together.

-Kevin P

[bcborders]

Kevin,

Thank you very much for taking the time to respond to me post!

In regards to the python node, that is already set up in the VI I attached to my original post. The functions are set up in the python script, and the parameters (matrices) are passed using labview matrices.

Here is a research project that undergoes a similar experiment, albeit without the stage control and Kalman Filter:

https://core.ac.uk/download/pdf/43551416.pdf

Specifically, the "Control System Development" section is a great overview.

I will try to describe this another way.

- The apparatus begins the raster scan (knight's tour, present at the top of the VI I attached)

- The photon counter runs at the same time, taking a set amount of readings per position of the knights tour. Herein lies the fine details of the timing. Every time the knight's tour moves to a different position, I need to take multiple readings to integrate and produce a curve.

     - Example: If the knight's tour switches position at a rate of 1MHz, and the photon counter is taking samples at 10MHz, we will have 10 samples per position (assuming near instantaneous transitions)

     - This is why the software timing will not work. Windows has a maximum resolution of 1 ms, which means the photon count will be the rate limiting step. Sampling the photon counter at 100Hz (1sample/1ms) would require a knight's tour movement of 10Hz (1 move per 10ms). The ratio of photon counter samples to knight's tour does not have to be 10:1, but it's important to know that there are 36 knight's tour steps (6x6 sample grid)

- When the photon counter finds a knight's tour position with an in integrated photon count greater than our experimentally set threshold (currently 12000, and annotated in the previously attached VI), the positions are then sent to the queue

- The queue is a FIFO setup that feeds the kalman filter, which makes fast predictions to control the microscope stages. These are what will move the entire stage to keep the protein under the laser

- When the fluorescence fades, or the particle exits the sample window, normal functioning resumes

In more amusing terms, the raster is the prison guard while the spotlight is the photon counter. When it finds an inmate (protein), it sends the location to the warden (queue) who quickly radios it to the guards (kalman filter). When the prisoner gets away, the spotlight continues to search for other inmates

Fortunately, the PicoHarp came with a fantastic set of VI's, so not much has to be done there. It is mostly copy pasted from their provided DLL's. I created the python node, and it works very well (tested independently on JupyterLab).

If the VI I originally attached does not seem to make sense with my description, please chalk that up to my being "slightly" out of my depth in this project, and do not read too much into it 🙂

Thank you again Kevin,

Dan

[Kevin_Price]

The only area where I might be able to offer some help is with DAQmx functions.  Others may be able to weigh in on how to sync host-side DAQmx tasks with FPGA-side code, but I can't.  Probably no one will be able to say much about the specialty PicoHarp device or your custom Python code.

It *is* an interesting-sounding system though so I did read the "Control System Development" section of the linked doc.  My main impression was that the folks who wrote that up seemed to have a rudimentary at best understanding of DAQ under LabVIEW.  The way things were described didn't have the air of well-established and confident knowledge.

But getting back to your original post, you said you "need to create a schema of DAQmx tasks to implement the same synchronization I roughly approximated using the software-timed VI".

What synchronization are you trying to accomplish?  Where is the software-timed VI that roughly approximates it?  Do you mean the apparent interaction with FPGA nodes in that upper left corner loop that seemingly does the knight's tour?  Then what's left for DAQmx to do?   And how do you expect to coordinate Python scripts and PicoHarp driver calls with FPGA control?   And where does DAQmx fit into this?   That's probably the only piece I have any chance of helping with anyway.

-Kevin P

[bcborders]

Kevin,

Frankly, help with DAQ is exactly what I need! So thank you very much for offering your help. I will try to answer your questions wholly:

What synchronization are you trying to accomplish?

I am trying to use the hardware clock to trigger the functionalities of every single component. For example, I would place the entire PicoHarp VI from the manufacturer into a subVI, then use the DAQmx Read method (I am only partly sure this is the correct use of it, please bear with me) to query the photon counts from it.

Further, I would like to time the knight's tour using DAQmx as well. The Knights tour steps (in Hz) are hard coded, and currently fed into a sub VI in the sample I attached in my original post (please see the two arrays at the top of the VI -- each number corresponds to a Hz value for the frequency driver). Could I perhaps use "DAQmx write" to feed the steps from the arrays into the sub VI?

As for the Python node, I imagine a FIFO queue would be necessary to prevent overflow or data loss. This is what I attempted to implement in the VI I attached. The """intended""" function of the three while loops was to run concurrently, acting as a de facto queue to pass location data to the Python script. If this is not what is happening to your eyes, please disregard the VI 🙂

Where is the software-timed VI that roughly approximates it?  Do you mean the apparent interaction with FPGA nodes in that upper left corner loop that seemingly does the knight's tour?

The VI attached to my original post runs on software-timing. It uses the built in wait and LabView timing functionalities, all of which have a maximum resolution of 1 ms. If my understanding of this aspect of LabView incorrect, please chalk that off to me being the resident computer scientist on a team of physical chemists 

Then what's left for DAQmx to do?

I would like to have everything running on DAQmx. It is not workable to have some components running at thousands of hertz, triggered by the 80MHz hardware clock, while others are working on 1ms intervals. The difference would be too drastic. So this is where the crux of the problem is... I need to use DAQmx as a conductor of sorts for this motley orchestra I have on my hands here.

And how do you expect to coordinate Python scripts and PicoHarp driver calls with FPGA control?   And where does DAQmx fit into this?

I was hoping that if I place each component into a subVI, then import them into the master VI, I could read and write data to them using the DAQmx functions. Is this possible? With my rudimentary (at best) LabView knowledge, this seems to be the way forward to me. Modularize the components into subVI's, then use DAQmx to coordinate data moving amongst them.

Dan

[Kevin_Price]

I have an admittedly sketchy understanding of the whole system, but what I'm hearing still makes me think there's a fundamental barrier here.  And it's mainly about the distinction between precise (and fast) hardware timing vs. variable (and relatively slow) software timing.

DAQmx only applies to NI's own data acq devices, and only a subset of them at that.  (A rather large subset, but a subset nonetheless).  Nothing about DAQmx can directly provide hardware-level sync with any kind of PicoHarp dll driver or Python script execution.  You'd be making DAQmx calls followed by calls to the PicoHarp driver and/or Python nodes, landing you *squarely* in the realm of software timing as far as coordination of these "moving parts."

So I think you are limited in a very fundamental way here.  It's extraordinarily doubtful that there's a PicoHarp driver available to run under a LabVIEW-compatible Real-Time target or FPGA.  Same for Python nodes (to the best of my knowledge).  All those things have to be performed in a higher level OS like Windows, which again keeps you squarely in the realm of software timing.

When you describe those integer arrays as "frequencies".  Are they analog waveform frequencies?  Pulse frequencies?  What's going on at a low level to turn these integers into real-world signals?   What's the overhead and latency involved in that conversion?   Is it consistent?

Maybe a route exists that can get you to the kind of timing and sync you're after, but I'm pretty doubtful it's farther down the path you're on.

-Kevin P

[bcborders]

Kevin,

This process has been so difficult, I was suspicious that there was some underlying incompatibility here.

RE Picoharp: I will reach out to the makers and inquire if there is such a VI, however I strongly doubt it.

When you describe those integer arrays as "frequencies".  Are they analog waveform frequencies?  Pulse frequencies?  What's going on at a low level to turn these integers into real-world signals?   What's the overhead and latency involved in that conversion?   Is it consistent?

I have attached two files to this message. The first is a screenshot from the Opto-Electronic manual for the frequency driver: this shows the mathematical calculation to get from MHz (F) to the frequency code (N). The values in the array are the MHz (F), and we are using the 31 bit version. The second is what this calculation looks like in the subVI that performs the raster scan. In the VI I attached to my first post, it is the implanted subVI at the very top. For good measure, I will attach the entire manual if you prefer.

The provided manual for the frequency driver does not state too much in terms of hard metrics, but a manual on their website says the rise-fall time is 10ns. Here it is -- the model we have is a DDSPA, not the DRFA:

http://www.aaoptoelectronic.com/wp-content/uploads/documents/DDSPAxx-DRFAxx-2014.pdf

I greatly appreciate the consultation Kevin, your knowledge about this has shed some very important light on this problem I'm facing. I suppose the only question I can ask now is if you have any suggestions on other routes to investigate to implement this solution? Otherwise, we may have to refactor the entire apparatus. It won't be the end of the world by any means, but I'd definitely like to investigate any other alternatives before that point.

Very best,

Dan

[santo_13]

I have been following this thread closely since the first post. From the beginning, I did not have a good feeling of things were compatible and the discussions so far stayed in the same realm. There are just too many things to consider and evaluate the feasibility. By default, any HW synchronization across multiple vendors is a no-go for me and it might clear up based on the manufacturer and careful feasibility study.

I would recommend reaching out to a local NI alliance partner to do a feasibility study and I don't think just the help of forum members is sufficient to solve it.

[bcborders]

Santhosh,

Thank you for taking the time to follow the thread. At this point, I would gladly taken any and all speculation that you may have about this problem and how to solve/sidestep it.

I will look into the closest NI alliance partner! Thank you for the suggestion

Please let me know if you have any questions about the state of this (intended) apparatus

Dan

[santo_13]

Typically any instrumentation used for Test and Measurement has trigger and synchronization capabilities that can export/import electrical signals through connectors.

Now, coming to your PicoHarp 300, the DVI connector labelled CTRL gives me some hope that some level of external signals is possible.

This external event marker capability may be useful by making DAQ send periodic signals whenever you raster and record the time events for post-processing.

Beyond, this I don't think the PicoHarp has any kind of synchronization. This means HW timed synchronization is not possible on the fly but rather you can correlate data by some means.