New Unit Test 'Framework' from JKI - Caraya

[tyk007]

Anyone had any experience with this yet?

https://github.com/JKISoftware/Caraya

[James_McN]

Trying to decide how to evaluate this properly.

First reactions are I both like and dislike the simplicity which really comes down to two questions:

• Can it still do what I need now and in the future?
• Does it mean writing more or less test code?

I use tests fixtures (setup, teardown) quite heavily in VI Tester at the minute and that is the obvious thing missing. Obviously these are just subVIs but then you have to think about whether the tests can run in parallel or do we need to serialise them, for example if your test fixture needs to be clean each for each test.

That said VI tester has a certain about of cruft produced for each test and perhaps simpler is better. It also has some nuances working with the project which this then removes.

It also means you have to define your on structure and working practices around how to organise these VIs but that shouldn't be too bad.

One thing on my wishlist is easy automation for CI but this would appear to go backwards/same as before. No CLI, No plugin architectures/configuration for reporting.

One plus though is that VI Tester is not good for RT as it loads everything in the my computer context which is OK but this will allow you to run the tests in whatever context you want.

Sorry bit of a braindump but I have been trying to digest this overnight!

[kubes_ch]

This is a realy nice unit Testing FW which is as simple as possible like cUnit or other UnitTest FW for other programming languages.

We used both, VI Tester and the other JKI Tool in past. For me, both have some disadvantages and personally i don't like this "configuration"-style that VI Tester has. Unit Tests have to be programmed in the same language as the code to test is.

One thing on my wishlist is a report in junit style. So we can use it directly in our CI System. But this seems not a big work because the structure now is more or less like junit.

[James_McN]

Are you sure your not confusing VI Tester with NI's Unit Test Framework? VI Tester is LabVIEW based where you create classes to perform tests a la xUnit.

But yes I agree, the simplicity of this is the main appealing factor.

[kubes_ch]

Sorry, of course you are right I confused with NIs Unit Test FW.

[Tomi_Maila]

If comparing JKI's VI Tester and the new Caraya framework, VI Tester is definitely better fit for rigorous test driven development as it is very process oriented and provides good framework and structure for developing the tests. VI Tester is also a better fit if you want a proven test framework for a mission critical project.

JKI's Caraya is still experimental and may not be the right tool for mission critical projects quite yet. It is a tool for developers that would not otherwise add unit test coverage or for projects that can't afford the time required to write more rigorous tests for VI Tester. With Caraya the developer is pretty much in full control of the process. Caraya tests are pure LabVIEW code allowing the developer to easily add set-up and tear-down steps. Actually probably more easily so than with VI Tester. The developer is also in full control on how the tests are executed (e.g. in-parallel vs sequentially). As such Caraya provides a very powerful paradighm for even the most advanced projects.

With regards to the reports, we currently support a text file based reports as well as the Caraya UI. I'm thinking of next adding a plugin support for custom reports so that you could implement your own report format. We could probably easily add some default report formats as well in addition to the text based reports. Would this be something anyone would find useful?

There is already CI integration, Carya can be called using JKI's internal CI integration framework and anyone could easily integrate Caraya with their own CI integraiton layer.

[Tomi_Maila]

Below is an example of a unit test case to test Obtain Queue operation for finite size queues to give you some sence on how to create test cases with Caraya.

If you needed to create a set-up and tear-down step you would have two options. 1) You could call them directly from this VI, set-up in the beginning and tear-down in the end or 2) you could call set-up and tear-down in the VI calling the test cases and pass the created references to the test cases via connector pane inputs.

I'll need to work on more documentation, tutorials and maybe a video introduction.

Tomi

[Tomi_Maila]

I just posted an introductory video to Caraya.

https://decibel.ni.com/content/message/124280#124280

[AndreasStark]

Hi Tomi,

is there an updated recommendation when to use Caraya vs VI Tester?

I'm new to both and a good comparison would be helpful.

Thanks in advance

Andreas

[bokorpet]

Hi Andreas,

do you know this page? https://instacoverage.io/details/#comparison

It may be a helpful source for you to compare the available tool options.

Regards,

Peter