Passing cluster input as argument to dll

[Abhi_kool]

Hey, I am new to LabView and I am tasked with investigating the use of python to write test cases for our labview codes. I found out that there is an option to build "dll" file. So I built a simple cluster made of a vector and an integer. The code simply adds the integer to all the elements of the vector and returns an output vector. I have built the dll file and have this as my function prototype:

void __stdcall func(Cluster *input_cluster, double output_array[], int32_t len);

typedef struct {

    int32_t dimSize;

    double Numeric[1];

} DoubleArrayBase;

typedef DoubleArrayBase **DoubleArray;

typedef struct {

    DoubleArray input_array;

    double Numeric;

} Cluster;

I am trying to use ctypes in python to run the dll and I can successfully pass in the arguments and it runs without any error. Unfortunately the output is not what I expect, it is just zeros. The problem I think is with the vector definition in the cluster. "DoubleArray" is a data type which is a double pointer to the structure "DoubleArrayBase". However it is hard to know from this header file definition whether the the double pointer is a pointer to an array of pointers or if it is just pointer to a pointer. If I were to do a C program of this I would have a pointer pointing to an array of pointers, which in turn point to the array of structs. I have tried just about all combination in my python code and nothing seem to work! I would gladly accept any help I can in figuring this out.

Attaching the labview code:

[rolfk]

You certainly could import that with ctypes in Python but unless you want to dig up to your ears into how compilers like to arrange structs in memory, and other such “interesting” low level compiler details, my recommendation is simply: DON’T!

Such an interface costs you a lot of time to develop, go through zillion code-debug-crash and back to code cycles, and even if it won’t crash anymore, you are never really sure if you got it really right without understanding pretty much every nitty gritty detail about memory allocations and layout and it’s a real maintenance nightmare in the long run. The main advantage would be mainly in trying to build some job security by hoping you are the only one who can maintain it. 😀

You want to use individual functional parameters for those elements and you definitely do not want to use LabVIEW arrays but simply C array pointers (which you can relatively easily map to numpy arrays in Python through ctypes.

[Abhi_kool]

Hi, thanks for your reply. The thing is, in our production software we do use clusters. So if we were to go down this path (using python to do testing) then we need to understand how labview handles the pointers and so on. What I don't get it is people compile dll and use that in their C programs (I have seen many questions regarding this as well), so if someone were to do the same, I wonder if it will run (or not run). Yea, so I will try it for some more time and then give up if I cannot solve it.

[rolfk]

If you program in C, some of the nitty gritty details are taken care of by the C compiler more or less automatically. But by FFIs (foreign function interface) like Python crypes or the LabVIEW Call Library Node, you the programmer are the “compiler”. It’s called foreign since it interfaces to a different programming model (usually C) than your current system and the rules of the translation and transformations have to be perfect or you get crashes or memory corruptions.

[Abhi_kool]

 I understand. I will post in this thread, if I ever manage to find a solution for this. I really appreciate your timely response 😄

[Andrey_Dmitriev]

In general nothing is wrong if you will do everything accurately with understanding how LabVIEW stores data in the Memory. Not sure why do you need third parameter (array's size is included in the cluster), but anyway, this is LabVIEW code, performs multiplication in LabVIEW and in DLL:

I used here small trick to unload DLL after call, otherwise you need to close VI every time before compilation of the DLL.

This is "reference" DLL Code:

#include <stdint.h>

typedef struct {
	int32_t dimSize;
	double Numeric[1];
} DoubleArrayBase;

typedef DoubleArrayBase **DoubleArray;

typedef struct {
	DoubleArray input_array;
	double Numeric;
} Cluster;

void __cdecl __declspec(dllexport) func(Cluster *input_cluster, double output_array[], int32_t len)
{
	for(int i = 0; i < len; i++){
		if (!input_cluster) break;
		if (!input_cluster->input_array) break;
		output_array[i] = (*(input_cluster->input_array))->Numeric[i] * input_cluster->Numeric;
	}
}

And the Python code does the same:

import ctypes as ct

# Load the DLL
mydll = ct.CDLL(r"PATH_TO_YOUR_DLL_FILE.dll")

# size of the input Array
dimSize = 3

# Define the structures in Python
class DoubleArrayBase(ct.Structure):
    _fields_ = [
        ('dimSize', ct.c_int32),
        ('Numeric', ct.c_double * dimSize)
    ]

DoubleArray = ct.POINTER(ct.POINTER(DoubleArrayBase))

class Cluster(ct.Structure):
    _fields_ = [
        ('input_array', DoubleArray),
        ('Numeric', ct.c_double)
    ]

# Define the function signature
func = mydll.func
func.argtypes = [ct.POINTER(Cluster), ct.POINTER(ct.c_double), ct.c_int32]
func.restype = None

# Create input data
input_cluster = Cluster()
input_cluster.input_array = DoubleArray()

# Prepare output array
output_array = (ct.c_double * dimSize)()

DoubleArr = ct.c_double * dimSize

# Test Data
input_cluster.Numeric = 40.0
vec = DoubleArr(10.0, 20.0, 30.0)

input_cluster.input_array = ct.pointer(ct.pointer(DoubleArrayBase(dimSize=dimSize, Numeric=vec)))

# Call the function
func(ct.byref(input_cluster), output_array, dimSize)

# Print the result
print(list(output_array))

and the result

>python dummyPy.py
[400.0, 800.0, 1200.0]

Hope it helps (LabVIEW 2024 x64/CVI 2020/Python 3.11.6 was used).

Andrey.

[Abhi_kool]

Hey, I have a question here. The DoubleArrayBase has a parameter named "Numeric" of size 1

typedef struct {
	int32_t dimSize;
	double Numeric[1];
} DoubleArrayBase;

However if your python code you assign it with a vector of size 3 (variable named "vec").

input_cluster.Numeric = 40.0
vec = DoubleArr(10.0, 20.0, 30.0)

input_cluster.input_array = ct.pointer(ct.pointer(DoubleArrayBase(dimSize=dimSize, Numeric=vec)))

How is this possible? Does the parameter size get allocated dynamically? If so it is not very apparent from the header file. Can you explain how that part works? 

[rolfk]

Well done, Andrey. I was to busy with other things (and lazy) to dig into this and have no immediate need for it myself, so I was simply trying to say that it is not easy if you don't understand C to a fairly deep level. And the average user here on the forums doesn't use LabVIEW if they know C programming that well. 😁

It looks good and resembles my own experiences in the past, although there I could control the shared library interface (I wrote the DLL/so in C) and I avoided trying to do variable sized data arrays in structs. So I could avoid things like

ct.POINTER(ct.POINTER(DoubleArrayBase))

However if you do anything with the DoubleArray in your LabVIEW DLL other than reading it, you are in deep trouble. Then the array will need to be allocated and deleted in Python by calling the according LabVIEW memory manager functions, otherwise the DLL will crash. And it gets even more interesting, you need to link to the exactly same LabVIEW runtime DLL that the DLL itself instantiates on initialization. And finding out which that is, can be a very interesting exercise.

I'm concerned about the 

[rolfk]

---

@Abhi_kool wrote:

Hey, I have a question here. The DoubleArrayBase has a parameter named "Numeric" of size 1

typedef struct {
	int32_t dimSize;
	double Numeric[1];
} DoubleArrayBase;

However if your python code you assign it with a vector of size 3 (variable named "vec").

input_cluster.Numeric = 40.0
vec = DoubleArr(10.0, 20.0, 30.0)

input_cluster.input_array = ct.pointer(ct.pointer(DoubleArrayBase(dimSize=dimSize, Numeric=vec)))

How is this possible? Does the parameter size get allocated dynamically? If so it is not very apparent from the header file. Can you explain how that part works? 

---

The C declaration is just that, a declaration. It does not say anything about how large the array portion really is. Older versions of C before C11 or so did not require support for declaring 0 sized arrays, nor unspecified sizes. And you can't just make it a pointer either, since that portion of the array is supposed to be inlined with the dim_size parameter. So the smallest size that makes this a valid datatype declaration for the broadest possible number of C compilers was 1.

How large the array really will be, will be defined always at runtime when "mallocing" it.

In C everything needs to be declared before it can be used, but it is very easy (and also dangerous) to reassign any pointer with any other pointer type at runtime.

The Python code doesn't care about that C declaration. It is your business to make the ctypes constructs compatible with that, which is the compiler task I was referring to that you need to do here.

However as I pointed out in my previous post, if your LabVIEW DLL ever tries to deallocate or resize that array, you are in deep trouble. The Python ctypes declaration as shown by Andrey uses Python memory allocations to make a compatible looking memory layout. If the LabVIEW code only reads this memory, nothing bad "should" happen. If it tries to modify it however, all hell will break loose if that array is not a properly allocated LabVIEW handle that was allocated with the according memory manager functions from the exact same LabVIEW runtime engine, that your DLL happens to use. Luckily, a LabVIEW generated DLL usually exports per individual data type functions to allocate, resize and deallocate those arrays. Anything else like trying to directly call the according LabVIEW runtime engine would end in unmaintainable insanity.

[Andrey_Dmitriev]

---

@rolfk wrote:

Well done, Andrey. I was to busy with other things (and lazy)

...

However if you do anything with the DoubleArray in your LabVIEW DLL other than reading it, you are in deep trouble. 

---

Thank you, Rolf, and yes, to be honest, I was also too busy and lazy as well, therefore more than half of the Python code was generated by AI, it was not functional, but after some minor modifications I was able to get at least same result and and least crash free. It should be used very carefully, of course with deep understanding who will allocate the memory and ho deallocate and so on, fully agree.

What you mentioned about using the LabVIEW's memory manager functions, is probably something like this with native and initially empty LabVIEW Array on output:

Then we will need code like this using NumericArrayResize():

#include "C:\Program Files\National Instruments\LabVIEW 2024\cintools\extcode.h"

typedef struct {
	int32_t dimSize;
	double elt[1];
} DoubleArrayBase;

typedef DoubleArrayBase **DoubleArray;

typedef struct {
	DoubleArray input_array;
	double Numeric;
} Cluster;

void __cdecl __declspec(dllexport) func2(Cluster *input_cluster, DoubleArray output_array)
{
	int len =  (*(input_cluster->input_array))->dimSize;
	//allocate destination
	NumericArrayResize(fD, 1, (UHandle *)(&output_array), len * sizeof(double));
   (*output_array)->dimSize = len;	
	
	for(int i = 0; i < len; i++){
		if (!input_cluster) break;
		if (!input_cluster->input_array) break;
		 (*output_array)->elt[i] = (*(input_cluster->input_array))->elt[i] * input_cluster->Numeric;
	}
}

And then it will be not so easy in python, you're perfectly right, because allocation will require to call according external LabVIEW functions (but this special case is out of scope, not applicable to initial question, where the trivial structures was used with preallocated "plain" array). I think impossible is nothing, we can probably use DSNewHandle() from python and so on, but I think, it is not normal use case from architectural point of view.