Generating a Report by just executing the main sequence

[Samz91]

Hello , I am a bit new to TestStand. The project I am working on needs to generate a report(if possible) while executing just the "Main Sequence". I see that the reports are generated when I execute "Single Pass" or "UUT Test" but not when I execute the "Main Sequence" option. I  would like to know whether it is possible to generate a report when I execute only the "Main Sequence". If possible then how should I implemet it?Kinldy help.

[~jiggawax~]

You will need to write your own code to do this.  Report generation is one of the main purposes of a process model.  One thing you can do is strip down the process model dramatically to only include the reporting steps.

What is your purpose behind all of this.  There may be a better way.

Cheers,

[Samz91]

Thank you for the answer. The purpose is to generate a customized report after testing a product(Only the main sequence is executed).

I tired to invoke the "NI_reportgenerator" sequence through a sequence call within the Main Sequence. Is that a good approach?I am confused regarding how to collect the results generated and report it using the "NI_reportgenerator.seq". Does configuring this sequence serve the purpose?

[~jiggawax~]

NI_ReportGenerator.seq is a plugin.  The different sequences inside of it get invoked by the process model at different times.  For instance Model Plugin - Initialize is used to get the plugin options from disk (these options determine what type of report to use and where to store the report).  While Model Plugin - UUT Done is where the report is gernerated (depending on the settings and model).  So it's not as simple as calling one sequence out of there.  You would need to know which ones to explicitly call and then pass them the correct parameters.

I highly recommend you take the model and strip all the stuff out of it that you don't want.

When it comes to report/database generation there are 2 parts: Result Collection and Report Generation

As each step executes it places its results inside of the Locals.ResultList if the step is set to record.  When the sequence completes it returns it's collection back to the caller.  Eventually MainSequence passes this ResultList back to the model.  The model then takes that and does the Report Generation.

If you absolutely insist on running in interactive mode (i.e. no process model) then you can take Locals.ResultList and parse through it and spit it out to a file.  Be carefull though because if you call subsequences it could require recursion to get all the results correctly.

If I understood your need for only running MainSequence I might be able to provide better light on the subject.

Regards,

[Samz91]

I am working on a framework which is already set. As per the framework the testing is done by just running the Main Sequence. I actually don't know the why the framework is so set that the testing is done only by executing the "Main Sequence". Since only Main Sequence is executed,the report is not getting generated. 

I am working on to find whether it is possible to use the TestStand features to generate a Report directly similar to what we get while executing "Single Pass" or "Test UUT".

 1) If I try to copy the values of Locals.ResultList to some other variable/file,will it work? Because the values in Locals.ResultList gets deleted once the          execution of main sequence is done.

2)As you said the report generation happens in the "Model Plugin-UUT Done" step. I guess when just running the main sequence  the execution of

     UUT Done step doesn't happen.So will the report generation be possible if I am able to execute this step explicitly along with Model Plugin-Initialize and Model Plugin-UUT Start(to obtain report file path).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                

[Norbert_B]

---

@Samz91 wrote:

I am working on a framework which is already set. As per the framework the testing is done by just running the Main Sequence.[...]                                                                                                                                                                                                                                                   

---

Who ever designed that framework had the worst idea ever with that approach. "Run MainSequence" is a debugging tool, not an execution entry point for a test system.

The purpose of the process model is to handle all stuff on the test system which is not directly concering the DUT test itself. Material handling, scanning the ID, generating a report and common control of the tester are the most obvious tasks. Not using the process model is a concious decision AGAINST running that stuff!

My recommendation is to rework that framework as it breaks the very basic approach of NI TestStand.

However, you can implement your own reporting into your test sequence (which is highly disadviced!). The simplest approach to this is to copy all report related stuff into your test sequence. You also will have to change some things in the reporting as the standard TS report mechanism expects a single container as result container. If you put the reporting into your sequence, the container is not existent, you only have the content of the container available (Locals.ResultList in SequenceCall "MainSequence"). Therefore, you have to remove the "entry container" of type StepResult all the way and replace it directly with the ResultList.

Norbert

[~jiggawax~]

I agree with Norbert.  This is not much of a framework.  You have more of a framework by using native TestStand with the process model.

If you are confined to a framework then just put a call in there that will run your sequence against the process model.  You can just have a step in there that runs something like this:

RunState.Engine.NewExecution(RunState.Engine.GetSequenceFileEx("path to sequence file", 107, ConflictHandler_Error), "MainSequence", "SequentialModel.seq", False, 0)

Basically think of it as a launcher that launches your sequence file against a process model.

[pwaszkewitz]

As the one with the "worst idea ever" I would like to clarify a few things.

The "MainSequence" approach is just a kick-starter. Our actual framework sequences all reside in a single sequence file. If it happens that we want to run n TestStand processes on the same machine but with different settings, then we use n sequence files with different names, loading the settings according to that name, and then start the actual framework through the "SinglePass" execution entry point of the standard TestStand parallel process model.

One reason for this are e.g. StationGlobals. We need the same StationGlobals in these processes but with different values. So the kick-starter sequence modifies the StationGlobals in-process before starting the "SinglePass" execution. There may be other ways to solve this but it works and we had no problems with it. So we have been using the approach outlined in the other follow-up post for years.

What all this is actually about is not MainSequence but that we do not let our "SinglePass" execution end. We are well aware that TestStand reporting usually requires the execution to end and that the UUT entry point is intended for processing a series of work pieces.

This approach however starts a new socket execution after receiving its start event, handling serial numbers etc. And the process of starting a new execution is - or at least, was, when we designed that framework, I come to that in a moment - time-consuming, in the area of 500ms.

In some of our systems, after subtracting time required for mechanical part handling, this is all the time we have, so it is out of the question to spend it reacting to the start signal. With a continuously running execution, triggered by a notification event, we achieve round-trips from start signal reception to result (with empty test sequences, naturally, to measure the actual framework overhead) in the range of 5-10ms.

So to sum up that part: we are using a process model, we are using a normal execution entry point and we are making extensive use of the - as far as I understood doug9000 - recommended customization method, namely callbacks; the most important ones for us being ProcessSetup, ProcessCleanup, MainSequence and PostStepRuntimeError.

So far, this approach has served us very well; the main drawback is of course that we forego TestStand reporting because that typically requires the execution to end. As there have been considerable changes to the TestStand process models since we implemented our framework, we are just now planning to review this design and to explore other possibilities, with the support of an NI expert.

Regards

Peter

[~jiggawax~]

I think you should consider the new plugin architecture when revamping your framework.  It will handle the reporting in a new thread.  You should be able to easily architect it such that you don't need to stop the execution to get a report.

Regards,

[pwaszkewitz]

That's the plan.

Thanks

Peter