RT Traces are incomplete

[Daklu]

I've been attempting to collect RT traces on my application; however, I'm running into situations where sometimes the trace that is returned appears to be corrupted and/or empty.  I haven't been able to nail down a repro case, but it happens in clumps--when it happens it tends to happen consistently.  I'm able to create a simple test VI and capture traces from that, so I know the libraries on the target are intact, but capturing a trace in my app chokes.

I've included a screenshot of the invalid trace, the block diagram of an executable that sometimes fails, and the .log file of the failed capture.  Has anyone ever seen this before?

[DonhJoe]

Hi,

When it happens, what do you do in order to repair it? Does it just stop by itself? Do you have any firewall, antivirus or protection software running while getting traces?

regards.

[Daklu]

I haven't found any cause-and-effect relationships to make this problem start or stop, and while I do have anti-virus software running, I can create simple VI's and successfully retrieve traces from them so I don't think that's the issue.  This problem seems to be unique to my code or how I'm using the RETT.

This issue *might* be timing related.  Architecturally, we have a small bootloader rtexe (the block diagram shown above) that loads the main application, which is deployed as a source distribution.  It takes between 60-90 seconds for the open operation to finish and the traces (almost?) always return empty.  If I have the bootloader open a much smaller VI (taking maybe a couple seconds), the trace returns okay.  On the other hand, shorter traces (on the order of a couple seconds) I've added to the main application code also almost always return empty. 

One thing I've noticed is the times the trace returns empty seems to correlate to attempting to trace during periods of high CPU utilization.  Is it possible the high CPU utilization is somehow corrupting the trace record?

Edit:

I've also had the Distributed System Manager running on my desktop to monitor CPU usage while collecting traces.  Could the RT side of that be causing problems?

[DonhJoe]

Hi,

I do not believe monitoring the CPU Usage from the DSM could be causing the problem. Do you get empty traces when you are not monitoring with the DSM?

Is the application confidential? At this point, I think the best troubleshooting and understanding of the situation would be achieve3d by reproducing the behavior on our side. If the actual application is confidential, then an example that uses the same architecture and also outputs empty traces on your side, would do the job.

Regards

[Daklu]

I'll need to get approval from management before sending you the code.  NI has a secure ftp server for these kinds of things, right?  I seem to recall you had set up a server where customers can upload code but nobody can download anything.

I've been trying to create a simplified example that reproduces the behavior, but so far I haven't been successful.

[CataM]

You can protect with a password the archived code and only tell the one you want it to see it.

[Daklu]

It's more than that... the sbRIO is part of a larger system and the software (right now) doesn't execute normally if it can't connect to the other components.  It's going to take me a while to trim the project down to something that can run without the connections.

[DonhJoe]

Hi Daklu,

I believe this kind of problems could be addressed more efficiently on a Service Request. Give us a call at 866-275-6964 and ask to talk to an Application Engineer from the LabVIEW Real-Time team. We will be glad to assist you on this issue and also analyze if there is something we should fix or improve on the RT Traces tools.

Regards.

[Daklu]

Will do.  Thank you.