TestStand sequence files - tests?

[pawhan11]

In my setup I've tests for modules that are called by sequence files + sequence analyzer script. This cannot catch runtime logic problems during execution. 

How do You approach testing of sequence files? Write sequence that tests sequence? How to execute that in automated way?

[kosist90]

That will be already integration/system level test, not unit tests, and it is highly depends on the testing scope - like, what do you exactly want to test.

Imagine, that you have code module which stores test results data from the sequence. So what is needed to be tested? Code module itself - then one does not need to run it from the sequence file, it is possible to place it into some VI, and test it just from LabVIEW. Logic of data retrieving from sequence? Then yes, sequence file is needed, and that code module should be called via TestStand.

Or another example - VIs for communication with some device. But actually what is test scope in this case? Logic of messages parsing? Then sequence is not needed. Logic of communication itself? Then again - possible to test it from LabVIEW. And overall, then real hardware is needed - so developer who does not have such hardware, is not able to execute the test. In that case, more useful - as for me - will be to have nice UI which allows to manually "communicate" with the device. And such UI will be used for testing, and also for manual control of the device - which is highly usable during system setup.

So I would first of all define scope of the tests - e.g. which errors are you going to identify - and then select testing strategy.

[pawhan11]

Thanks for response, I did not clarify my problem. Modules that are used in sequences already have unit tests and we expect they work fine. My problem is validation of logic inside test sequence that is sometimes complex. There is always pain when for example loop indexing is invalid or structure condition is broken in TS sequence logic after 30 minutes of tests.

I guess for TS sequence file logic best option would be to write another sequence file and run it via cli. Next step would be to convert results to junit format

https://forums.ni.com/t5/Example-Code/Converting-TestStand-XML-Reports-to-JUnit/ta-p/3689131

Thinking about system level tests and validation when i would like to run top level sequence, simulate all measurement equipment, abstract operator for manual operations and see if for certain simulated values outcome is expected on CI server - it seems impossible to do...