Fast data logging in VBAI2014

[PaulFreudenthaler]

I am working with a VBAI.vi which grabs images from a dual gig-e camera and information from a cRIO. It is supposed to grab the information fast and write them to a hard drive.

The procedure looks like this (you can also see them in picture "procedure.JPG"):

1. The picture is grabbed via dual-gig-e and the "acquire image" module.
2. Via the "calculator" module the file name is determined and several variables are read via network from a cRIO.
3. Via the "data logging" module some of the variables are written into a .csv file to the hard drive.
4. Via the "image logging" module the acquired image is written as a .tiff file to the hard rive.

Our camera is capable to deliver up to 19 images per second via dual gig-Ethernet. To avoid a image-bugger overrun, we need to acquire and write the information to the hard drive in 1/19 seconds --> ~50 milliseconds.

As you can see in "measurements.JPG", the procedure takes between 80 to 100 ms, the slowest of the modules seems to be the "data logging" step which takes 44 ms, which seems odd since it appears less elaborate than image logging. You can see the data which is logged in "logging.JPG".

Do you have any idea how I can speed up this procedure? I thought about writing a LabVIEW VI which logs data to spreadsheet instead of using the "data logging" module from VBAI, but I don't know how to run the VI in VBAI.

These processing times do not depend on the hard drive speed, I get similar results on a ramdisk.

Thank you for assistance!

Sincerely,
Paul

[udka]

Not familiar with VBAI(hence cannot comment on the timings)

-What you can do is parallelize the process in LabVIEW

-Check at Queued Message Handler or Producer/Consumer architecture to parallelize the Image acquisition and Processing, so that acquisition need not wait untill the logging/processing is done.

-Refer: http://www.ni.com/white-paper/3023/en/

[Brad · Accepted solution]

Here are a few tips to improve performance:

1. Make sure you uncheck the "Display Result Image for this State" at the top left of the inspection script area. Do this for all your states to improve performance. I think the Datalogging could look like it's taking so long because the Display update is asynchronous and whenever the OS decides to redraw the image, it may happen to be whenthe datalogging step is running.

2. Make sure youare running in inspection mode. Configuration mode adds a lot of delays so you can see the progress of the inspection

3. You could try logging the images/data in the background (this is a setting within the steps). This won't guarntee that every image is logged though if you are acquiring images faster than your HD can log them. If this is the case, you may consider:

   - Getting a faster HD like and SSD.

   - Trying different file formats (BMP or JPG may be faster than tiff)

   - Make sure you aren't including the overlay in the image since this converts it to a color image and can produce much larger images to save to disk.

4. Try running the 64-bit version of VBAI for improved performance.

5. Do you have a fast PC you are running this on?

As a quick sanity check, open the attached inspection. I put a simulate Acq in the Setup State (View>>View Complete Inspection Setup, then click on the Setup State in the left toolbar area). Then Choose Operate>>Benchmark Inspection with 50 iterations and see how long the image logging takes. This will give you an idea of just how fast your hardware can support saving to disk. Try different image formats like PNG, JPG, BMP, etc. since different PC configurations will have different performance.

[PaulFreudenthaler]

Thank you so much Brad! I wasn't expecting such a thorough explanation and guide! I am going to check everything ASAP!

[ChristopheC]

You can also experiment with the compression factor of some of the image file formats in the configuration of the image logging step.

Typically, a higher compression factor will result in smaller images that are faster to write, but it is more CPU intensive to compute.

One important thing about compressed file formats is that some are lossy and some are lossless. Choose appropriately based on your application and your need or not to retrieve the original image without compression artefacts.

Also, are you looking into saving individual images or a stream, like a movie file format?

In some cases. Depending on your hardware, you may be able to install some third party hardware accelerated codecs that will allow for hardware compression of the video stream.

In some cases, you might also get better performance by adjusting the image size to be a multiple of 8.

For the data logging step, try experiencing with logging a single file as opposed to multiple smaller ones.

I hope this helps.

Christophe

[PaulFreudenthaler]

Thank you Brad, inspection mode did the trick! 19 fps are no problem anymore!

[PaulFreudenthaler]

I am not sure if the next problem we ran into is a cRIO or a VBAI problem, therefore I would appreciate it if you mind to take a look at this: http://forums.ni.com/t5/Real-Time-Measurement-and/cRIO-RT-FIFO-for-network-shared-variables-and-VBAI...