TestStand Report Results as JSON

[cmo29]

Hello!

Using the SequentialModel.seq as a base, I would like to implement a reporting method that outputs results in JSON format. As far as I can tell this doesn't exist natively and as such I'm wondering what the cleanest approach might be.

My proposed methods of attack:

     - Collecting results step by step in JSON format (alongside the SequentialModel--seems like a waste but maybe the most straightforward)

     - Parsing results from a container such as MainSequenceResults in the TestResults callback (using a recursive sequence call?)

     - Parsing the results from one of the native text-based formats (xml, html) maybe via LabVIEW, Python, etc.

I'm using LabVIEW 2022 and TestStand 2022.

Thanks!

[santo_13]

Post processing from XML seems like a easier solution.

[James_Grey · Accepted solution]

There is a hidden feature to convert the values in a property object hierarchy into JSON.

To try it out, put a breakpoint at the end of a sequence and type the following in the watch window:

Locals.ResultList.GetFormattedValue("", 0x1001)

If you find this useful, let us know.... it might lead to this being turned into a documented feature.

Here is the text I got from the results for the Computer Motherboard example, minus the indenting which didn't survive the paste:

[
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Done",
"ReportText" : "",
"Common" : { },
"TS" : {
"StartTime" : 4512.3024697000001,
"TotalTime" : 1.7586387000000001,
"ModuleTime" : 1.7585748999999999,
"Index" : 0,
"StepName" : "Simulation Dialog",
"StepGroup" : "Setup",
"StepId" : "ID#:bWGTqrEX5hGzYpiQlqYQ+D",
"Id" : 5,
"StepType" : "Action",
"BlockLevel" : 0
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Done",
"ReportText" : "",
"Common" : { },
"TS" : {
"StartTime" : 4514.0673774999996,
"TotalTime" : 0.00029829999999999999,
"ModuleTime" : 0.00025680000000000001,
"Index" : 1,
"StepName" : "Turn Vacuum Table On",
"StepGroup" : "Setup",
"StepId" : "ID#:bmGTqrEX5hGzYpiQlqYQ+D",
"Id" : 6,
"StepType" : "Action",
"BlockLevel" : 0
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Passed",
"PassFail" : true,
"ReportText" : "",
"Common" : { },
"TS" : {
"StartTime" : 4514.1431216000001,
"TotalTime" : 0.0002721,
"ModuleTime" : 0.00014420000000000001,
"Index" : 0,
"StepName" : "Powerup Test",
"StepGroup" : "Main",
"StepId" : "ID#:b2GTqrEX5hGzYpiQlqYQ+D",
"Id" : 7,
"StepType" : "PassFailTest",
"BlockLevel" : 0
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Done",
"ReportText" : "{True}",
"Common" : { },
"TS" : {
"StartTime" : 4514.1533974000004,
"TotalTime" : 0.0010437999999999999,
"Index" : 1,
"StepName" : "If",
"StepGroup" : "Main",
"StepId" : "ID#:cGGTqrEX5hGzYpiQlqYQ+D",
"Id" : 8,
"StepType" : "NI_Flow_If",
"BlockLevel" : 0
}
},
{
"TS" : {
"SequenceCall" : {
"ResultList" : [
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Passed",
"PassFail" : true,
"ReportText" : "",
"Common" : { },
"TS" : {
"StartTime" : 4514.2088084999996,
"TotalTime" : 0.00020900000000000001,
"ModuleTime" : 0.0001225,
"Index" : 0,
"StepName" : "Register Test",
"StepGroup" : "Main",
"StepId" : "ID#:gWGTqrEX5hGzYpiQlqYQ+D",
"Id" : 10,
"StepType" : "PassFailTest",
"BlockLevel" : 0
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Passed",
"PassFail" : true,
"ReportText" : "",
"Common" : { },
"TS" : {
"StartTime" : 4514.2479971000002,
"TotalTime" : 0.0001796,
"ModuleTime" : 0.00013129999999999999,
"Index" : 1,
"StepName" : "Instruction Set Test",
"StepGroup" : "Main",
"StepId" : "ID#:gmGTqrEX5hGzYpiQlqYQ+D",
"Id" : 11,
"StepType" : "PassFailTest",
"BlockLevel" : 0
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Passed",
"PassFail" : true,
"ReportText" : "",
"Common" : { },
"TS" : {
"StartTime" : 4514.2609068000002,
"TotalTime" : 0.0001607,
"ModuleTime" : 0.0001187,
"Index" : 2,
"StepName" : "Cache Test",
"StepGroup" : "Main",
"StepId" : "ID#:g2GTqrEX5hGzYpiQlqYQ+D",
"Id" : 12,
"StepType" : "PassFailTest",
"BlockLevel" : 0
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Passed",
"PassFail" : true,
"ReportText" : "",
"Common" : { },
"TS" : {
"StartTime" : 4514.2889754999997,
"TotalTime" : 0.0001493,
"ModuleTime" : 0.0001031,
"Index" : 3,
"StepName" : "FPU Test",
"StepGroup" : "Main",
"StepId" : "ID#:hGGTqrEX5hGzYpiQlqYQ+D",
"Id" : 13,
"StepType" : "PassFailTest",
"BlockLevel" : 0
}
}
],
"SequenceFile" : "C:\\Users\\Public\\Documents\\National Instruments\\TestStand 2022 (64-bit)\\Examples\\Demos\\Computer Motherboard Test\\CVI\\Computer Motherboard Test Sequence.seq",
"SequenceFileVersion" : "22.0.0.49152",
"Sequence" : "CPU Test",
"Status" : "Passed",
"ThreadId" : 3
},
"StartTime" : 4514.1653051000003,
"TotalTime" : 0.12564539999999999,
"ModuleTime" : 0.1255067,
"Index" : 2,
"StepName" : "CPU Test",
"StepGroup" : "Main",
"StepId" : "ID#:cWGTqrEX5hGzYpiQlqYQ+D",
"Id" : 9,
"StepType" : "SequenceCall",
"BlockLevel" : 1
},
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Passed",
"ReportText" : "",
"Common" : { }
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Passed",
"PassFail" : true,
"ReportText" : "",
"Common" : { },
"TS" : {
"StartTime" : 4514.3229064999996,
"TotalTime" : 0.00017420000000000001,
"ModuleTime" : 0.00012740000000000001,
"Index" : 3,
"StepName" : "ROM Test",
"StepGroup" : "Main",
"StepId" : "ID#:cmGTqrEX5hGzYpiQlqYQ+D",
"Id" : 14,
"StepType" : "PassFailTest",
"BlockLevel" : 1
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Passed",
"PassFail" : true,
"ReportText" : "",
"Common" : { },
"TS" : {
"StartTime" : 4514.3238436000001,
"TotalTime" : 0.00012129999999999999,
"ModuleTime" : 8.8700000000000001e-05,
"Index" : 4,
"StepName" : "RAM Test",
"StepGroup" : "Main",
"StepId" : "ID#:c2GTqrEX5hGzYpiQlqYQ+D",
"Id" : 15,
"StepType" : "PassFailTest",
"BlockLevel" : 1
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Passed",
"Numeric" : 5,
"ReportText" : "",
"Common" : { },
"RawLimits" : {
"Low" : 0,
"High" : 10
},
"TS" : {
"StartTime" : 4514.3265643000004,
"TotalTime" : 0.00029,
"ModuleTime" : 0.0001077,
"Index" : 5,
"StepName" : "Video Test",
"StepGroup" : "Main",
"StepId" : "ID#:dGGTqrEX5hGzYpiQlqYQ+D",
"Id" : 16,
"StepType" : "NumericLimitTest",
"BlockLevel" : 1
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Passed",
"Numeric" : 6,
"ReportText" : "",
"Common" : { },
"RawLimits" : {
"Low" : 5
},
"TS" : {
"StartTime" : 4514.3298101,
"TotalTime" : 0.0001329,
"ModuleTime" : 8.7800000000000006e-05,
"Index" : 6,
"StepName" : "Keyboard Test",
"StepGroup" : "Main",
"StepId" : "ID#:dWGTqrEX5hGzYpiQlqYQ+D",
"Id" : 17,
"StepType" : "NumericLimitTest",
"BlockLevel" : 1
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Done",
"ReportText" : "{False}",
"Common" : { },
"TS" : {
"StartTime" : 4514.3317913000001,
"TotalTime" : 0.0014639,
"Index" : 7,
"StepName" : "If",
"StepGroup" : "Main",
"StepId" : "ID#:dmGTqrEX5hGzYpiQlqYQ+D",
"Id" : 18,
"StepType" : "NI_Flow_If",
"BlockLevel" : 1
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Done",
"ReportText" : "(If)",
"Common" : { },
"TS" : {
"StartTime" : 4514.3349170000001,
"TotalTime" : 3.7499999999999997e-05,
"Index" : 13,
"StepName" : "End",
"StepGroup" : "Main",
"StepId" : "ID#:fGGTqrEX5hGzYpiQlqYQ+D",
"Id" : 19,
"StepType" : "NI_Flow_End",
"BlockLevel" : 1
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Done",
"ReportText" : "(If)",
"Common" : { },
"TS" : {
"StartTime" : 4514.3489140000001,
"TotalTime" : 5.1199999999999998e-05,
"Index" : 16,
"StepName" : "End",
"StepGroup" : "Main",
"StepId" : "ID#:f2GTqrEX5hGzYpiQlqYQ+D",
"Id" : 21,
"StepType" : "NI_Flow_End",
"BlockLevel" : 0
}
},
{
"Error" : {
"Code" : 0,
"Msg" : "",
"Occurred" : false
},
"Status" : "Done",
"ReportText" : "",
"Common" : { },
"TS" : {
"StartTime" : 4514.3612233000003,
"TotalTime" : 0.0001674,
"ModuleTime" : 0.00012410000000000001,
"Index" : 0,
"StepName" : "Turn Vacuum Table Off",
"StepGroup" : "Cleanup",
"StepId" : "ID#:gGGTqrEX5hGzYpiQlqYQ+D",
"Id" : 22,
"StepType" : "Action",
"BlockLevel" : 0
}
}
]

[James_Grey]

... also I should mention that feature is somewhat recent. I don't recall what version added it, but it should at least be in the most recent version.

[cmo29]

What a find! Thank you for sharing this is exactly what I was hoping for!

[Oli_Wachno]

James, this is a freaking awesome feature, thanks for sharing 

[Phil_T.]

Yes this is a feature I would be interested in!

[magnux]

This is a great feature, as we are looking to storing our reports to a NoSQL database like MongoDb (currently we are using MariaDB).

Please make it an official documented feature!

Br

/Magnus

[Gerät]

I have built an customized plugin for our customer based on this function. There is a bug here regarding the parsing of a 2D-Array. It is parsed as a 1D-Array.

Locals.TwoDim.AsPropertyObject.GetFormattedValue("",0x1001) ->

[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

Anyone else working with this undocumented feature?