@clicks wrote:
@BlueCheese Thanks again for your comments and motivation, I really appreciate that.
There IS additional copying of the data. I perfectly understand the way to emulate the ring buffer. But the normal grab does make a copy of the data. You can not recode the data without loading it from the memory and writing it back, this is a copy. No matter how efficient the processor can do the job, this is not the problem. The problem is the memory bottleneck. It is about 1GB of data per second, an additional read and write should be avoided.
The processor can do the recoding of the pixels very efficiently. But he FPGA on the camera can do it even better. So i do not understand why the processor should take all the data and do the job again.
The advanced ring buffer acquisition was obviously implemented and provided by NI to avoid copying the data for high performance applications. But it does not work with the Basler USB3 cam!
I did not investigate the mono/grayscale case further since it is not an option for us (as already mentioned). Its like you buy a Ferrari and when the engine is not working well the dealer tells you to push it by hand. Ok, you will arrive from A to B... But this is not the way NI should address challenges.
The Sony CMOS sensor, The Basler FPGA USB3 system, the NI USB3 framegrabber, and the NI IMAQdx driver are all highly tuned components for performance applications. So I expect a perfect interplay.
You react very professional on my provocation. I did not say "not supported", but the Basler cam is not (fully) compatible with the NI IMAQ. Your were suggesting me to buy another cam with the same sensor from a different manufacturer to be compatible with the NI IMAQ. What should I say to my project superviser? "Sorry, we are delayed a month and need to buy a new computer or new cams because my expectation in NI was too high"? I did a careful proof of concept with six logitech 902 cams using the ring buffer and then extrapolating the to the data rate of the Basler cams. Of course it is my fault to not have checked the pixel format compatibility, I was just assuming this (is it explicitly written somewhere?). Now, I hope NI or Basler can do this little job and provide a common pixel format. Let me know when done, I will help advertising it.
@clicks
Believe me, you'll find no stronger advocate for the Ring API than myself---I helped design and implement it. That said, a lot of the benefits are more along the lines of simplifying coding and moving complexity to the background instead of the user's LabVIEW diagram. It has mechanisms like reference counting Extract() calls on each buffer to allow nice abstractions for sending a single image to multiple threads that can all process it independently and the driver manages recycling the buffer once the last thread Release()'s it.
While I also share a deep appreciation for wringing every last bit of performance out of the system and avoiding inefficiencies where possible, I also have had the benefit of seeing lots of internal benchmark/profiling data, along with the real-world impact on application performance. While I can't say for certain that your application wouldn't gain from using the Ring API, my hunch is that it would be a negligible impact. Here's some numbers for reference (back when we added the Ring API in the first place):
These benchmarks were run with a Basler acA2040-90um (2048x2048x8bpp @ 90fps) - 360MB/s on a CVS 1459RT, which is a 1.9Ghz Atom system (which I assume is significantly slower than your system, especially when it comes to memory bandwidth)
| Background Acquisition only (no Extract or GetImage) |
GetImage API (1-copy) (getting every image) |
Ring API (0-copy) (getting every image) |
| 0.6% |
6.0% |
1.1% |
Now, keep in mind these are pure CPU load measurements, and don't capture all the nuances of memory bandwidth utilization. However, keep in mind that with the Ring API benchmark number above, none of the data being acquired is being loaded into the CPU cache, since it is zero-copied into the buffer and the benchmark doesn't do anything with the data. On the GetImage benchmark, since there is a copy of the data, the data all gets read into the CPU cache during the copy and typically remains resident and hence when you access the image in your code, the CPU already has the data resident in the cache. The huge cache on modern CPUs hides a lot of the performance impact of data copies (though I'm not disagreeing that it is still better to avoid them). In the end, the performance impact to applications on using the Grab-vs-Ring is typically much less than the ~6x simply difference the CPU load in the table above implies, both because of the impact having the data already in the cache has, along with most processing algorithms taking far more CPU load than the acquisition itself.
As to whether there will be better compatibility with the Ring API and these particular Basler cameras in the future, I can't say for certain. I will say that there have been some features like programmable line padding that Basler added to better support NI's Ring API and its required alignment needs. That said, things like a wider RGB pixel format (with an added alpha channel) might have huge implications on the processing pipeline inside the camera (such as 33% more internal bandwidth needed on that pipeline) possibly have a more significant impact than you realize.
As to NI supporting a native 24-bit RGB pixel format for zero-copy processing, that is pretty tough to imagine, to be quite honest. When you have a couple decades worth of algorithms and image processing code, adding a new path on everything is pretty hard, especially when you consider that many algorithms have hand-rolled vectorization code paths. Also, keep in mind that color still represents a smaller portion of machine vision applications, and all the Basler cameras support zero-copy Ring just fine in the monochome case (though again, I recognize that your application requires color).
One last note, with the Logitech webcams you were prototyping with, even if you were using a Ring API, there is still a copy going on in the background. Ignoring whatever the DirectShow infrastructure is doing to convert the native MJPEG/H264/YUV data the camera is naively sending to BGRA, when the image hits IMAQdx as BGRA data, it is "bottom-up" DIB-format bitmap that needs to be transposed to the top-down format that our image processing requires, which the driver does as a copy internally. In an ideal world we could also have the option to silently do this kind of background processing for other camera types (to convert say 24-bit RGB to 32-bit BGRA), so that you could get all the nice API benefits of the Ring API as I mentioned above, but there are a bit more nuances there than there are for DirectShow that make it more challenging.
I wish you success in your application.
Eric
