Continuous Acquisition - IMAQdx - calling a past buffer

[arkapal]

Hi, I am trying to automatize a vision acquisition system using IMAQdx, but am stuck at making the buffer acquisition work. Here's a small description of my system. I want to record videos using a Basler camera (by using the IMAQdx VI from the Vision library in Labview). First, with the initiation of the system, it will keep streaming the acquired frames in the user interface. For this, I specified the number of buffers in continuous mode in IMAQdx Configure Acquisition, then using IMAQdx Get Image to get a contonuous stream on the user interface. Next, I want to start saving the frames into an AVI file with an user toggle switch. Now I want it to start saving from a few frames earlier from when the user toggles the save switch. But, if i set the 'buffer number in' to a past buffer than the current one, it does not change the buffer number out. In other words, it starts saving from when the user hits the save switch, and not from a few frames earlier.

For example, when i start the VI, the camera starts continuous acquisition, and starts filling up from buffer no 0 (assume the no of buffers is 1000). the buffer is filled up to 500 when the user toggles switch to start to save. Now I want to start saving the frames into an avi file from buffer no 400 till I stop the VI (but in spite of setting the buffer number in to 400, it keeps saving from buffer number 500). I am using a case structure to perform the two cases (when it only streams and when it saves into an avi).

Also, how to make sure that there are no frames being skipped while writing into an avi, in this setup?  

[Bob_Schor]

We use this technique all the time, and it works for us.  In figuring out how many buffers you need to allocate, there are some calculations you can run, but there's nothing like an "experiment" to be sure you don't miss frames.

You don't provide any quantitative information, such as the size of your Image (in pixels and in format, i.e. 8-bit Grayscale vs 32-bit RGB), the Frame Rate, the transfer rate (from Camera to Buffer), and how many seconds "back in time" you want to record.

The Proof of Concept is to take a video of a Timer or a Watch with a sweep second hand.  If you push the "Acquire" button when the Second hand reads 0 seconds, and specify a 2-second "before" period, you should see a Video with the second hand starting at 58 and moving smoothly without jumps.  If there are jumps, you need more buffers.  If there are no jumps, try decreasing the number of buffers (to save memory) and see when you start losing frames.

In our case, we are using Axis cameras that transmit images over TCP/IP.  Our images are 640x480 (I think), 32-bit RGB, 30fps, and we want 2 seconds of images before the trigger.  The algorithm we use to compute how many buffers to allocate is fps * (Pre-event seconds + 1) * 1.5, or 30 * 3 * 1.5 = 135.

One thing I didn't mention -- this is part of a behavioral monitoring Project, which can handle up to 24 "stations" simultaneously (we typically run 6-12 at a time).  So far, we haven't seemed to have lost any videos.

One more thing -- once we get a signal to start recording a video, we stream frames from the Camera Buffer as fast as they can be written to disk until we get to the current Buffer, then as new frames come in, they are also written to disk.  Thus the "catch-up" to the current image takes place right at the beginning, minimizing the chance of losing any frames.

Bob Schor

[arkapal]

Hi Bob,

Thank for your reply. Sorry for not providing the quantitative details. The size of my image is 1920 x 1200 pixels in 8-bit Grayscale. The frame rate of camera acquisition is 150Hz. I want to start recording from 1 second back (which means 150 frames or buffers back). I am not sure how to obtain/set the camera to buffer transfer rate.  

I will try to work out the calculation as you discussed in your reply. 

But, i am still stuck at how to start recording from a past buffer. The VI that I attach does not work properly as i want it to. I am not sure how to start recording from past. (Setting the 'buffer number in' to a past buffer number in IMAQdx Get Image does not apparently return a past buffer).

Best,

Arka 

[Bob_Schor]

Every time you capture an Image in the Camera, the Buffer number goes up by 1.  Suppose you have 10 buffers in your Camera, and have recorded 5 images in Buffers 0 through 4.  The next 5 images go into Buffers 5-9.  The next Image, Buffer number 10, goes into the Driver's Buffer 0, overwriting what was formerly Buffer Number 0.

So if you've recorded 123 images, your current Buffer number will be (122 Mod 10) = 2, but if you want the image, you need to ask for Buffer 122. 

Here's where the fun comes in.  Let's say you have configured the Driver with 100 buffers, instead of 10.  At some point, you want to start saving Images.  Here's the idea:

• You need to know the current (always-increasing) Buffer Number, let's assume it is 400.  Calculate how many buffers "back" you need to get the "preceding images" -- let's assume it is 50.
• Starting at Current Buffer - "Back" Buffers, or 400 - 50 = 350, in a parallel processing loop, start getting Images from buffers 350 to the current Buffer (initially 400, but probably higher by the time you get the earlier images written) and writing them to the AVI file.  At some point you'll "catch up" to the current Buffer, then can pace the Acquisition rate.  Note that during this initialization of the AVI file (which may take some time), more Images are being acquired -- you want to save the earliest buffers before the Driver overwrites them with newly-acquired images, hence the algorithm to compute how many Buffers you need.

By the algorithm I suggested for computing Buffers, you need 2 seconds worth, or 300, possibly with a 50% "safety factor", or 450 Buffers.  But you need to test it to be sure it is adequate to allow you to save AVIs.

You must use parallel loops and something like the Producer/Consumer Design Pattern to accomplish this, particularly with such large images and such high frame rates.  It would be good to have a modern Windows machine with multiple cores (an I7 would be a good idea) and a fairly fast clock.

Your buffers will take 1920 * 1200 * 450 = 1GB of memory, and you need to be able to write at 350MB/sec to "keep up" with the data coming in (this shouldn't be difficult unless you are dealing with multiple cameras).

I see, looking at your attached VI, that you allocated 20 buffers -- about an order of magnitude too low ...

Bob Schor

[rolfk]

350MB/s is certainly not trivial to get to write to a harddisk even with a moderate SSD installed. The write speed of a high end SSD driver of about 560MB/s is a theoretical maximum. The real value for a real world application is always lower as the video writing will also have to do some intermediate calculations. Compression can lower the amount of data needed to write to the disk, so that even a fast HD might be usable but that will of course require some CPU power to do.

Generally I would say that 1920*1200 at 150 frames per second would still require a carefully selected hardware system in order to allow for more than short periods of video writing to disk.

[Bob_Schor]

Rolf -- thanks for catching my careless error.

BS

[arkapal]

Dear Bob,

I understand the algorithm that you mentioned, but I am having difficulties executing it.

Let me mention that I will be recording animal movement with this setup, and the idea is to start recording when the animal starts moving. In order to make sure that we do not lose any frames at the point when the animal starts, we want to start to record from a few frames before (~2 secs). From previous studies, we are assuming that we need to record a maximum of 3 seconds of video from trigger. So in total, we want to save 5 seconds of video in total (2 secs = 300 frames before the trigger point and 3 secs = 450 frames after the trigger point). I am setting number of buffer as 1000, which I suppose is more than enough. 

Now, if you look at the VI that I have attached, I have constructed a case structure with a start toggle switch to start saving. I am using the IMAQdx Get Image in both cases. When false, it uses the buffer mode, "Every" in order stream every frame. But when the case structure is true, the buffer mode changes to "Buffer Number". and the Buffer Number in is calculated by 'present buffer - 300'. The problem arises here. if the current buffer is less than 300, then it does not do anything, and if the current buffer is more than 300, then it only captures 1 frame (for example, if current buffer is 322, then it only captures frame 22, and stops). 

The goal is to keep streaming the video as usual, but to start saving from 300 frames before. Let me elucidate, I have 1000 buffers. The camera keeps acquiring images at 150 frames/sec. So in the first second, buffer 0-149 is filled. Now if I trigger the save button when the buffer number reaches 322, the goal is to start saving from buffer 22 (=322-300) till 3 more seconds (450 frames) which means till buffer number 772 (=322+450) and then stop. 

It would be great if you can kindly refer to an example VI from where I can get ideas from?

Best,

Arka