LabVIEW
How granular do you get with your functional requirements?
I'm working with a Systems Engineer for the first time in my life. My team has never done requirements beyond "The system shall perform the acceptance test per document XYZ." Now we have a very comprehensive set of requirements in Jama. I hate Jama.
This topic is fascinating to me, and a thing that I now realize I haven't had to do in a long time. At a previous job I would be the one helping others with requirements writing, maybe because I was a stickler, or because I could interpret things more literal than others. I would often run through test procedures and fail things as they were written, when others would pass based on their interpretation.
As for your direct question I see no problem with either way it is written. But they will affect site or factory acceptance testing.
If you have a set of requirements where you include all sensors in one line, then to validate that one requirement you need evidence of all 5 sensors. If any one of them fail then you fail that requirement. The set of steps in your acceptance test will also need to instruct the user to do the steps for all 5 before validating the requirement.
I've also seen it written like this as an example:
1. The test software shall measure the output from the pressure input PT 112.
2. If the previously measure input frequency is not 500Hz +/- 10Hz the tester shall fail the DUT
3. If the previously measured input duty cycle is not 50% +/ -5% the tester shall fail the DUT.
4. The test software shall repeat steps 1 through 3 for PT 113, PT 114, PT 115, and PT 116, in place of PT 112.
Writing it this way is more convenient for when there are many steps taking place and then we just need to do the same steps over and over for other inputs. But from a testing perspective this can be more work to validate. Flattening these steps out over all pressure inputs is easier to test for, since each requirement is a single thing to check for. Requirement 4 here is actually a bunch of things, and may take a long time to validate, providing failing inputs for the frequency and duty cycle for so many sensors. Then again you can maybe convince the customer that looking at the report, and seeing the limits used is enough to validate lines 2 and 3, and all this really needs to prove is it is capable of measuring individual readings. In which case the requirement could be checked in a variety of other ways, like measuring the output with a scope, and then seeing that the tester has the same value in the report.
I have also seen a weird situation where the Requirement 4 actually said "The test software should repeat..." which is a bit weird. A shall statement needs to be done, and proven that it is done through verification. A should statement I believe means the software needs to do it, but there is no requirement to verify it does it. That would mean we need to prove it worked for PT 112, but for all others we just trusted the tester did what it should have.
Basically every Shall statement means lots of extra work. You need to writing the test procedure, then for every time the test procedure is ran the operator has lots of extra work to document it. Then this is repeated for factory acceptance test (FAT) and a site acceptance test (SAT). So you should limit your Shall statements to things you really do mean need to be proved it is doing. In this example there might be another requirement that the pressure sensor should output a 5V signal. This is a thing we can put in there but doesn't need to be verified. Then again maybe the customer doesn't care what the sensor is at all, just that the system can measure pressure within some tolerance. If that is the case maybe we can don't care about the signal type at all and can plug in a dummy pressure sensor and just see that the value goes bad when we expect, and that a golden DUT passes as we expect.
There's so much room for interpretation in the intent, and how to prove a thing does what it needs to. You can spend a ton of extra time proving the most simple thing, or go in circles forever because of a badly written requirement. My main advice is to think about how you will test for your Shalls when you write them. I remember one common requirement I struggled with was:
The test software shall be written in LabVIEW 2020.
Seems fine on its surface but when I have a built application how can I prove that? I can look at what runtime engine Windows has loaded when I run but now applications can run in different runtime engines. I can open up the EXE in a hex editor and try to find it there, but that can be unreliable. And in the end this is more of a design requirement and I'd suggest making it a should statement.
EDIT: Oh and NI still has their Requirements Gateway software. You have it scan for requirements in documents, then it can scan the source of LabVIEW or TestStand for where those requirements are being fulfilled, and then have it scan the SAT and FAT documents to see where they are being covered. It is a bit clunky at times, but it is great for checking for testing coverage.
@Shiv0921 wrote:
Aha I'm glad you mentioned Jama. I came across something suggesting Jama today and was wondering if I should consider using it. Now I see perhaps not. What's the pain with it? One of these systems that adds more work than it saves?
Jama is probably great. I'm just not accustomed to working with formal requirements.
