Why do RS-232 errors arise when I changed from cvirte 8.0.1.356 to 8.1.0.271?

[radula]

I recently received the LabView/CVI 8.1 update (having previously received 8.0 and 8.01).  With the 8.1 update, single instances of my application seem to run fine.  However, when several copies of my application, or several different applications, are simultaneously run on the same PC I get mysterious program crashes that debugging indicates are "nondebuggable protection exceptions" arising in the ComRdTerm function of cvirte.dll.  These errors did not occur with CVI 8.0 or (at least as far as I can recall) with CVI 8.0.1.

Browsing through your web site, I found two documents suggesting modifications.  The first, titled "Why is Serial Data Not Sent Immediately After Calling ComWrt Function in LabWindows/CVI?", suggests doing a ProcessSystemEvents() immediately before the ComRdTerm.  This helped matters somewhat, but not completely.  The second, titled "OpenComConfig Outputs Data Immediately when Single Stepping, but Not at Full Speed", suggests setting the output queue length to -1 in OpenComConfig to force immediate writing of RS-232 data when ComWrt is called.  This also helped quite a bit, but I still see occasional instances when my applications crash, which did not occur with earlier versions of the cvirte.dll.

So, I guess this is a question: what has changed in the RS-232 libraries between the 8.0.0.253 cvirte.dll (or maybe the 8.0.1.356 cvirte.dll) and the current 8.1.0.271 cvirte.dll to cause the need for these previously unnecessary code changes?  What can be done to completely eliminate the remaining occasional crashes?  Obviously I can revert to the earlier cvirte, but it'd be nice for NI to fix these issues going forward.

[Mert_A.]

Hello,

Between CVI 8.0.1 and 8.1, the serial library went through some reworking and refactoring while it was ported to run on Real-Time systems.  As a result, it is possible that subltle differences could result in different behavior in certain cases.  Without knowing the specifics of your code, it is very difficult to make a meaningful diagnosis of the problem.  Would it be possible for you to post a small project or source file that exhibits the crash?  Ideally, this would be some small program that does not require a specific instrument/device that crashes when multiple instances are launched and run. 

As to the articles you found on our website, if you configure your port with an output queue length of -1, this bypasses the "write buffer thread" mentioned in the first article, so there should be no need to call ProcessSystemEvents.  Just to confirm, you notice fewer crashes when you set the output queue length to -1?

If we can find that some bug was introduced that is causing your crashes, rest assured it will be fixed for a future version.

Mert A.
National Instruments

[radula]

Thanks for the information, Mert.  I'll have to see if I can come up with a simple piece of failing code.

The order in which I found the articles, and the order I implemented attempted fixes, was first the ProcessSystemEvents(), then the setting of the write queue length, so I don't yet know whether doing only the latter would suffice.

[menchar]

We're seeing a 8.1 rte Vs. 8.0 rte serial problem also.

We get com failures on a non - NI (built into the MB) com port with the 8.1 rte, but not with the 8.0 rte.  The MB is a S2895 Tyan, a dual-micro/dual core AMD system (4 cores total).

All of the serial i/o on the PCI-8430/8 ports in the system works fine with either rte.

There's some question about the clock rate not being quite right on the device side - it could be our setup is vulnerable to differences that might be tolerated on a more robust serial interface.

So we're going back to the 8.0 rte.

Menchar

[AEDavid]

Hi Menchar,

If you can post a simple application that reproduces the error and will work with a generic serial device, we will try to reproduce the error. 

[menchar]

Dave -

Not sure I'll be able to do this.

We're moving megabytes of data at 57.6 kbps using an ACK / NAK + CRC scheme from a PC to a device, and the error doesn't occur at the same spot every time.   We just don't have time to work this if using the 8.0 rte solves the problem.

It looks like a solid correlation though, 8.0 cvirte.dll works, 8.1 doesn't.

So we're going to run the 8.0 rte, not 8.1

We're going to move the PC side onto an NI serial port (PCI 8430/8) also.

There are 512 byte input/output queues being used.

It may well be that the problem is with the device - and that the 8.1 changes, while within the bounds of the rs-232 spec, are exposing a vulnerability in the device's serial port.

Menchar

[Mert_A.]

Menchar,

Unfortunately, it sounds unlikely that we would be able to root out the cause of the behavior difference without more details and investigation.  I understand that it is often not worth spending that time when there is already a known workaround (i.e. using the 8.0 RTE).  I did want to make a couple comments, though. 

First, if you are going to "move the PC side" to an NI serial board, can't you use the 8.1 RTE?  Your previous post stated that the NI ports seemed to work correctly with both the 8.0 and 8.1 RTEs. 

Also, you mentioned you are using input and output queue sizes of 512.  You may have already heard this suggestion, but I always recommend that users disable the output queue by passing a size of -1.  It removes some complexity (an extra writer thread) and usually improves performance.  I have no reason to believe that it would fix your problems in this case, but it probably couldn't hurt.

Mert A.
National Instruments

[menchar]

Mert -

We might be able to try using rte 8.1 with the 8430/8 to see if it'll work with the device, but I'm not calling the shots, and there's bigger fish to fry.

I do have everyone convinced we should use the 8430 with the device, if at all possible.  That way we don't have to retrench every time the PC built in com port changes (different MB's), and if there is a problem with the com HW/SW we stand a chance of getting support from you guys.

I'll suggest we kill the output queue.

We've always had problems with this device - I think it's vulnerable to whatever variation was induced with the 8.1 rte re-write for the serial i/o.

It sure would help if there were more transparency with these types of issues - how would anyone have known that NI re-wrote the serial library between 8.0 and 8.1, or what type of performance differences might result?

Thanks, Mert.

Menchar

[Mert_A.]

Regarding the last point, I understand the frustration of being blindsided by someone else's implementation changes. As a developer, this is the kind of thing that you're susceptible to whenever you use another company's tools. When we make changes to existing libraries, we try to be very careful to avoid doing anything that might reasonably break existing applications, but sometimes apps still get broken. There's any number of behavioral subtleties that existing applications might be (precariously) relying on, and we can't practically be aware of them all. We just try to stay consistent with our documentation and reasonable expectation, and make sure to document it if we can't.

We would never knowingly change the functionality of a library as old and widely used as the RS-232 Library. That's why we didn't announce the changes (aside from now being supported on RT systems). That's also why it's very important to us to address problems like yours when they come up. If there is something we're not handling very robustly, or some bug, we want to fix it.

I'm sorry for the inconvenience caused by this. If you run into any further snags, or would like to investigate this issue more, let us know.

Mert A.
National Instruments

[menchar]

Mert -

In my case, I continue to think it's a flakey device that's susceptible to frame arrival times or some such.  But, not knowing what might have changed in the rs-232 library, this is a pain to debug.   

It sure would help if NI could provide enough implementation detail so that a developer might be able to anticipate the effects of, say, running on a multi-core Vs. single core PC, or the effects of changing the process's base priority.  Also, performance differences that might exhibit when using the native Windows serial driver Vs. the N.I. kernel mode driver and NI serial hardware. 

I understand a desire to hide implementation detail, but, after all, we're developers and we don't always have a perfect device at the other end of the serial cable.

Menchar