Timeout problems using SwitchExecutive in TestStand Steps with Batch Model

[Vince_CH]

I’m having timeout problems by running a test sequence with switching steps in a batch model.

A short description of my HW architecture:

A simple schematic can be found in the appendix.

We have 3x a Keysight 34980A Switch Main Frame with 4 active Slot units. 2x a Matrix, 1x a GP- card and 1x Multiplexer.

Communication Bus is LXI.

Used SW:

• NI TestStand 2014 SP1      - using the standard NI Batch process model
• NI Switch Executive 2015
• OS: W7 64bit professional

The goal is to test a batch with 3 DUTs.

My first approach was to define only 1 virtual switch executive which includes all 3 HW Switch frames.  Advantage would be that I can connect the matrix card from the 3 HW switch frames with each other over a HW bus realized over the Virginia panel. This possibility is actually not required at the moment but is a design decision for the future. Requirement for the actual DUTs is that each DUT has his dedicated HW- Switch Frame with the corresponding card.

For the test sequence; I have a fix SE- Name for the TestStand Step and a test socket depending route name.  The approach works fine for 1 Socket and also for all 3 sockets as long as the tracing is enabled and not too fast.

By disabling the tracing I receive timeout error.

Error: “Could not perform the switching operation:  Details: An error has occurred while attempting to access device DAU2 VISA error = Zeitüberschreitung, bevor der Vorgang abgeschlossen werden konnte.(Error = -2147483648)”

translation German part of error message: Timeout error bevor process could be terminated

See screenshot in appendix.

It looks like there are deadlocks when multiple switching commands are sent.   See my example sequence.

In a second approach I defined a virtual switch executive unit for each HW Swich Frame: So in this case I have a socket depending switch name and a fix route name. This approach work now with 3 SE- objects in contrast to the first approach with only 1 SE-object.

Using this variant I don’t have the timeout error by running the sequence without tracing enable.

I would like to ask/ discuss why I’m seeing the timeouts in the first approach.  Does I do something wrong or is this some kind of restriction from my SW- Design or in general from SE.

Appendix: -     NI MAX export including the SE definitions -     TestStand example sequence file -     Error Screenshot -     HW architecture diagram

[JL@NI]

Hi Vince_CH,

Could you capture an I/O Trace log of a successful test execution (save it separately) and another I/O Trace log of an unsuccessful test execution? Post those here for comparison.

Regards,

Jeff

[Vince_CH]

Hi Jeff,

Thank you for having a look at my problem.

You will find the IO traces in the zip file.

Some comments to the traces:

I used the subSequences “Check50Hz_mit_Debounce_1SE”(ERROR) and “Check50Hz_mit_Debounce_3SE”(NO ERROR). For each seubsequence I have taken 1 trace with TestStand tracing enabled set to the slowest speed and 1 Trace without TestStand tracing enabled.

Generated Errors:

Trace Name:

“Check50Hz_mit_Debounce_1SE_Tracing_enable_onMaxSlowSpeed.nitrace”  -->No Error

Trace Name:

Check50Hz_mit_Debounce_1SE_Tracing_disable.nitrace

If the IO Trace is recording I receive the following error: An error occurred while performing the switch operation 'Connect' for the following device 'VirtualSwitchFrames_N300'. Could not perform the switching operation:  Details:An error has occurred while attempting to access device DAU3 Agilent34980A: Does not support this class-compliant feature: method GetNextCoercionRecord.(Error = -2147483648)

see screenshot

Error_1SE_noTestStandTracing_IOTracingON.PNG

If IO trace is not recording I receive the error: An error occurred while performing the switch operation 'Connect' for the following device 'VirtualSwitchFrames_N300'. Could not perform the switching operation:  Details:An error has occurred while attempting to access device DAU3 VISA error = Zeitüberschreitung, bevor der Vorgang abgeschlossen werden konnte.(Error = -2147483648) 

see screenshot

Error_1SE_noTestStandTracing_noIOTracing.PNG

Trace Name:

Check50Hz_mit_Debounce_3SE_Tracing_enable_onMaxSlowSpeed.nitrace ---> No Error 

Trace Name:

Check50Hz_mit_Debounce_3SE_Tracing_disable.nitrace---> No Error

[JL@NI]

Hi Vince_CH,

After looking through your I/O trace logs, it appears that there are many differences near the end of the trace; certain commands are simply not being sent to one of your switches. One thing that comes to mind is a race condition. Are there any sections of your code that should be synchronized and due to lack of synchronization, the correct sequence of switching commands are not executing in the correct order/timing? Due to the fact that this issue seems to only occur when tracing is disabled (execution will speed up), race conditions are more likely to occur. I also encourage you to look at the I/O trace logs using a comparison tool (e.g. Notepad++ has a Compare plugin). This may help locate sections of code that indicate problem areas with respect to syncronization.

Another test to rule out TestStand execution tracing: enable tracing and set to the fastest setting; does this error still occur?

Please let us know what you find.

Regards,

Jeff

[JL@NI]

Hi Vince_CH,

Do you have any updates on this issue?

Regards,

Jeff