Multipackage Approach to TestStand Deployment

[Oli_Wachno]

Good morning everyone,

in terms of modular distribution, we're thinking of distributing a single test program in different packages, e.g.

Package 1.) TestSequences & Code Modules

Package 2.) Framework / Models

Package 3.) Specific Environment

 Has anybody already gone down this road/ rabbithole?

Using th NIPackage format would be my favourite, thats's why I'd really appreciate answers to the open questions in https://forums.ni.com/t5/UKTAG-UK-Test-Automation-Group/UKTAG-Meeting-6-Agenda/td-p/4097244

(shameless plug)

Cheers

Oli

[~jiggawax~]

My TestStand framework is build using NI Package Manager as the deployment method.  We go even more granular than you are suggesting.  Some of our test systems have up to 40 or 50 packages.  We put them in a feed or several feeds for that matter.  Ever since we switched to this method about 4 or 5 years ago deployment has become a non-issue for the many developers working on my teams.  In fact it is a simple after thought and generally takes less than 2-3 minutes.  Part of this is how we develop as well.  We develop against PPLs, and not against raw VIs.  Basically everything in TS is calling a PPL that has already been tested and deployed in a package.

The other nice thing about doing this is that we can easily "state" machines.  So essentially we can roll the machine back if needs be to a previous date in a matter of minutes.  Makes debugging and testing new features nice because a developer can walk down, install their new package/s, test some things, and then "state" it back to how it was before they started.

I can't imagine going back to the TestStand Deployment Utility with installers and such.  Seems so archaic and cumbersome at this point.  

Hope this helps,

[Oli_Wachno]

H Jigg,

thanks for your answer. What you have been describing is what we are aiming for.

From what I am reading and re-watching https://youtu.be/VEEZNUc2EZc you are no longer using the TestStand Deployment Utility and for building the NI Packages using LabVIEW instead.

I love the concept of NIPackages (as stated multiple times in this group) yet in the meantime I have some reservations to using it due to open issues and questions which have been going unanswered for quite some time now.

Instead, maybe Chocalatey as presented earlier by Abaco is a more flexible way to go.

Rant finished 😉

Are you using TestStand Environments? I see some coupling between an environment and the Testprogram (e.g. StationGlobals) which might may it difficult to seperate them into packages, Not saying it's impossible though (thanks to the blessings of the TestStand API)

Cheers

Oli

[~jiggawax~]

We don't use TestStand environments... although I could see a good case to do so in some situations.  Also, we never write code that uses StationGlobals..... I have my own strong opinions about that.

If you compiled a list of questions/concerns that caused your reservations for using NIPM I'd be happy to address them from my perspective.

I will say this about NIPM:

NI is "all in" on it.  Meaning you have to have it installed anyways to use LabVIEW, TestStand and other NI things.  They are only adding to it and making it easier to use as time goes on.  I've heard rumblings that they will also look at it as more of a developer distribution tool as well.  

It's free, not sure about chocalatey, but NIPM is completely free.

If we have to use it for other NI stuff then we may as well use it for our stuff. Training teams on both types would be a pain.

I think you'll see as it gains traction in the community more tools and better practices, contributed from the community.

SystemLink uses it to manage installations on software.  That's a huge bonus in my opinion, especially if you use SystemLink.

I've only found 1 limitation so far: Conflicts!  Basically the idea that I can only have 1 type of package installed on my machine at once... example: front end callbacks.  I only ever want one of those installed.  I guess environments would fix that but it would be nice to say "uninstall all packages with this string as part of its name before installing my package".  I know the technology exists in Debian but not in NIPM yet.

Anyhow, good luck!

[.aCe.]

---

@Oli_Wachno wrote:

Good morning everyone,

in terms of modular distribution, we're thinking of distributing a single test program in different packages, e.g.

Package 1.) TestSequences & Code Modules

Package 2.) Framework / Models

Package 3.) Specific Environment

 Has anybody already gone down this road/ rabbithole?

Using th NIPackage format would be my favourite, thats's why I'd really appreciate answers to the open questions in https://forums.ni.com/t5/UKTAG-UK-Test-Automation-Group/UKTAG-Meeting-6-Agenda/td-p/4097244

(shameless plug)

Cheers

Oli

---

Hello Oli,

We do use it but not to the granularity of Sam, I am glad he replied to the post.

I don't see the benefit of splitting the environment, test sequence and test sequence specific code modules. You are right about the other components. We use environment quite a bit and it has helped us no end in avoiding type conflicts and to ensure that all developers have the same "environment". We check in the the environment to SCC. You need to choose the files carefully otherwise you end up messing with peoples settings etc.

We stay clear of stationglobals too. However, you can have a stationglobals.ini per environment so that is an option but I still despise StationGlobals, not the concept but the implementation.

If you just need globals (constants / write once globals) between executions / sequences, we tend to use Engine.TemporaryGlobals : https://zone.ni.com/reference/en-XX/help/370052AA-01/tsapiref/reftopics/engine_temporaryglobals_p/

Example: https://forums.ni.com/t5/Example-Code/Using-TestStand-Temporary-Globals-Engine-TemporaryGlobals/ta-p...

[bienieck]

Sam, could you explain the conflictsts you have with callbacks a little more? Don't you have separate packages for front end callbacks/framework?

How to deal with the situation where you need to debug on the target machine (deployed with packages) and then transfer fixed code back to the development machine? Are you using SCC in pair with packages?

[Oli_Wachno]

---

@~jiggawax~ wrote:

[...]

If you compiled a list of questions/concerns that caused your reservations for using NIPM I'd be happy to address them from my perspective.

[...]

Hi Sam,

I'd appreciate your help on this! 

Open questions can be found here

Cheers

Oli

[Oli_Wachno]

Hi Sree,

thanks for the hint about the temporary globals, I wasn't aware of their existence!

We're using TestStand Environments more like "Sandboxes". That's why we see the neccessity to distribute them along the actual testprogram.

We're evalutating our architecture right now and try to find out the next neccessary steps. That's where all those questions originate from.

Cheers

Oli

[bienieck]

Sam,

Regarding Front End Callbacks how about this:

Create "infrastructure" empty package called FrontEndCallbacks and then create seperate packages for your different front end callbacks e.g. a-pack and b-pack. Than specify that a-pack and b-pack depends on your infra pack FrontEndCallbacks (see image below).

Before you install your a-pack or b-pack (your actual front end callback package) you would need to call a script (temporarily stored on a target machine) using "Custom executes" "Pre Install". This script will first check if you have FrontEndCallbacks  pack installed and then if it is installed then look for packs with FrontEndCallbacks pack as dependency.

So you need:

"nipkg.exe list-installed" to check if you have FrontEndCallbacks pack installed

If installed, use the "list-installed" and "get-dependencies" (maybe filter by provider) to find your a-pack or b-pack.

Use "nipkg.exe remove" to remove current front end callback before installing new one.

Easier solution might also be with calling "nipkg.exe cfg-set frontendcallbackpackname a-pack" and then "nipkg.exe cfg-get nipkg.frontendcallbackpackname" to see if you need to remove something before installing.

Or you might figure something in between ("infra depenedencies" and "nipkg.exe cfg") 🙂