Codec issues in recording and playback of compressed video.

[jtrout]

I want to store compressed video and review the video later on a separate Labview station.  H264 provides the needed level of compression, but LabVIEW does not play it fast enough.  Media Player and VLC play the video fine.  However, I want to use LabVIEW for the measurement tools (as opposed to playback in Media Player via ActiveX).  Can anyone recommend an MPEG-4 codec that you have gotten to work with good performance?  I can't seem to get an MPEG-4 codec working right now.  

Details: My camera captures 8-bit grayscale in 1280x960 pixels at 24.5 fps.  Raw video takes about 10 GB per 5min inspection video.  I can compress lossless with Lagarith at about 4 GB per inspection.  We need to archive onto DVD media, and we need to store about 10 inspections per disk (or compression down to about 4% of the raw video).  This level of compression requires a good lossy codec.  H264 can give me the compression I need, but it just plays back too slow in LabVIEW.

[Wee-Bey]

What code are you using to run this test? If it's sample code, which sample code are you using? A screenshot would help. If you're dealing with an 8-bit monochrome image, you should be able to stream it if your processor is fast enough, which is implied if you're able to run it through VLC. Are you seeing a choppy playback or freezing? What indicates to you that the playback is off?

[jtrout]

I am able to capture from the camera, display, and write a compressed avi in a multi-threaded LabVIEW exe without any CPU issues.  However, I am having trouble getting the recorded video (when encoded with good compression) to play back in a separate LabVIEW application.

When playing back H264, the best performance results in approx. normal speed at first and gradually slows down over the next few seconds.  I have tried increasing the I frames, which speeds up the playback overall, but it stutters and flashes (this still plays back fine in VLC or Media Player). 

I'm running on an I7 processor using a virtual machine given half of the cores.  Using a VM may slow it down some, but not enough to account for the slow playback.  The CPU usage is down around 10% when playing the H264 encoded video with VLC.  The CPU usage goes up to 80% when using LabVIEW and gradually climbs higher.  One post suggested that this is due to LabVIEW performing the playback with the CPU, but VLC would keep the processing on the GPU.  It may also have something to do with the inter-frame dependencies used in H264 that LabVIEW has trouble with because it extracts one frame at a time.

I have gotten the playback to work fine with uncompressed video, with a lossless compression (Huffyuv or Lagarith), and with a few other codecs that don't give me the compression and quality that I need.  What I need is a codec that will give me compression performance like H264 that will play back in LabVIEW.

[Kira_T]

Hi jtrout,

Unfortunately LabVIEW does not support H264 compression. Please see this Knowledge Base article for a list of available codecs on your computer and a comparison of the compression types. Do any of these other formats give you the performance you are looking for?

Regards,
Kira T

[jtrout]

I have been working on this problem for a long time.  "IMAQ AVI Get Filter Names" helps by showing which codecs LabVIEW is able to "see."  The strings output from this function generally can be input into the "IMAQ AVI Write Frame" function to test the encoding performance.

Not all codecs installed on the computer actually show up in this list.  Mostly the codecs that LabVIEW finds are the older type codecs that don't perform as well as newer codecs.  For example, LabVIEW seems unable to find xvid. Somewhere else in the forum it has been suggested that LabVIEW does not support xvid.  I'm not sure why LabVIEW can use some codecs and not others, but my suspicion is that LabVIEW has trouble with inter-frame relationships.  I am not familiar with how encoding algorithms do this exactly, but I would assume there has to be some buffering of consecutive frames so that areas of motion and non-motion can be encoded differently.  P and B frames are created using information from future and past frames.  Also, key frames need to be generated as needed.  I also think I have read that LabVIEW cannot use B frames, or maybe it was that the AVI container cannot use B frames.  

LabVIEW can write an AVI with x264.  If I remember correctly, the Zero Latency checkbox needs to be checked.  These videos that have been encoded in the H.264/MPEG-4 AVC standard will play back on any media player that I have installed.  They will not play back correctly with LabVIEW.  One interesting behavior is that the playback slows down linearly as the video gets farther away from the last key frame.  Then, the playback returns to full speed upon receiving a key frame.  This problem is somewhat improved by adding key frames more often (e.g. 1/sec).  It is possible to buffer frames from "IMAQ AVI Read Frame" and display those frames in a separate timed loop to get playback at a consistent speed.  However, each key frame appears garbled in LabVIEW (but not in other media players).

So far, my testing has found Lagarith to be the best lossless codec with compression down to about 10-25% of the original file size.  So far the ffdshow MJPEG compressor gives me the best performance among lossy codecs that can be played back in LabVIEW.  x264 performs better compression with high image quality.

My video is 8-bit grayscale and looks like low-contrast slow-moving horizontal bands that float up and down.  Not very exciting.

I have been unable to encode with DivX, theora, xvid, 3ivx.  LabVIEW does not find these codecs.  One DivX 6.9.2 codec does show up in the list but fails to work.  I have been able to encode with VP70, but the playback is the same as with H.264.  

SDKs are available from Microsoft for VC1 and MainConcepts for a variety of codecs.  I have not tried these yet.

I would like to use an out-of-the-box relatively modern codec (e.g. some variety of MPEG-4).  Has anyone had success with this?

[ChrisReed]

I agree totally.  For years now I have complained about the lack of support for the latest codecs for AVI's as well as the other video formats (see link below) and nothing has changed and yet trying to find third-party tools that can do a similar job also have their problems.

http://http://forums.ni.com/t5/LabVIEW-Idea-Exchange/NI-Vision-Support-for-multiple-Video-Formats/id...

The list of available codecs given above are so old that they are almost useless.

I haven't installed Vision 2011 on my computer yet, but I bet that it doesn't support any new Codecs.

Chris

[zkiss]

Hi jtrout,

I am also trying ti write videos with LabVIEW and x264, but I am unable to do it.

What codec pack did you use and what GUI for the codec settings?

Thanks,

Zoli

[ChrisReed]

Hi zkiss,

In Vision 2010 I have used the TechSmith Codec (TSCC) from Camtasia which can be found here:

http://www.techsmith.com/download.html

It's not as good as H264, but at least the Vision toolkit will play AVI's encoded with this Codec.

Chris

[C-Dison]

Hi jtrout,

Now, do you solve the problem that avi file is compressed with h.264 ? if do it ,can you show me block diagram.I am studying real-time compression tech,but not familiar.

Thanks,

C-Dison

[ChrisReed]

I haven't tried the h.264 codec yet, although I would like to since it is lot better than the Camtasia Codec in terms of compression size, encoding and decoding speeds and quality.

Chris