LabWindows/CVI

cancel
Showing results for 
Search instead for 
Did you mean: 

Running out of memory in temperature logger application

Hello,

 

I see a strange behavior of a temperature logger application I'm working on, when data buffer size is over (about) 1.5 Milion samples, I get memory (re)allocation error. Samples are integer values. Temperature measurement unit is an Agilent 34970 with a 34901A Mux card and a J type thermocouple.

Application is built around an UI with a Graph Chart and two timers, one for acquiring data and another for display,

samples are generated (in simulation mode, without Agilent 34970, I generate random values) in the sample Timer callback function, while during display Timer callback, they are represented on the Graph Chart using PlotY.

 

Memory allocation error comes up in simulation mode also. It seems that during operation, application uses a lot of memory - on Processes tab, in Task Manager, I can see up to 700 MBytes, maybe more. This a lot more that I was estimating initialy, that when using integers, data array should not exceed 6 Meg for 1.5 Milion samples. I do not want to be very accurate on estimating the memory requirements for this application, but when a simple application is using 700Meg of system memory, makes me think that something is not quite right ...Smiley Surprised

 

Anyway here is the data generation and display code, any suggestions to find the root cause of this issue are welcomed.

 

/*...*/


static BOOL simulated = TRUE;
static int panelHandle;
static int *Y_Array;
static int Y_Array_Size;

 

/*...*/

 

int main (int argc, char *argv[])
{

 /* Instrument and UI Initialization*/

 Y_Array_Size = 0;
 Y_Array = (int *) malloc (sizeof(int));

 if (Y_Array == NULL)
   return (-1);
 DisplayPanel (panelHandle);
 RunUserInterface ();
 free(Y_Array);
 /*... */
}

 

int CVICALLBACK cbf_SampleTimer (int panel, int control, int event,
  void *callbackData, int eventData1, int eventData2)
{
 
 int  Meas;
 int  i,idx,idx_start,lenght,Y_Array_size_start;
 ViInt32   DataPoints;

 switch (event)
  {
  case EVENT_TIMER_TICK:
   {
    if(simulated == FALSE){

     /* ... */
   
     } else {
     DataPoints = (ViInt32) Random(1480,1520);
     for(i=0;i<DataPoints;i++){

     /*...   Create comma separated measurement string*/

      }
     }
    Y_Array_size_start = Y_Array_Size;
    Y_Array_Size += DataPoints;
    Y_Array = (int *) realloc (Y_Array, Y_Array_Size * sizeof(int));
    if (Y_Array == NULL){
      SuspendTimerCallbacks ();
      MessagePopup ("Error !!!", "Can't Allocate More Memory for Y_Array!");
      QuitUserInterface (0);
      break;
     }
    for (i=0; i< DataPoints; i++){

     /* ...     read value from the measurement string*/

     Y_Array[Y_Array_size_start+i] = Meas; /*Place value in data Array */

     }
   } break;
  }
 return 0;
}
int CVICALLBACK cbf_DisplayTimer (int panel, int control, int event,
  void *callbackData, int eventData1, int eventData2)
{
 int BinarySwVal;
 switch (event)
  {
  case EVENT_TIMER_TICK:
   {
     GetCtrlVal (panel, PANEL_BINARYSWITCH, &BinarySwVal);
    if(BinarySwVal){
     DisplayInProgress = 1;
     DeleteGraphPlot (panel, PANEL_GRAPH, -1, VAL_IMMEDIATE_DRAW);
     PlotY (panel, PANEL_GRAPH, Y_Array, Y_Array_Size, VAL_INTEGER,
         VAL_THIN_LINE, VAL_EMPTY_SQUARE, VAL_SOLID, 1, VAL_GREEN);     /*Display whole array */
     DisplayInProgress = 0;
     }
   }
  break;
  }
 return 0;
}

0 Kudos
Message 1 of 9
(3,860 Views)

maybe not the problem at all, but... in your code, you wrote:

/* ...     read value from the measurement string*/

are you sure the  measurement string does not need to be freed once you are done with it ?

(note: i never worked with an Agilent 34970, so this is pure speculation on my side. i don't know how you retrieve the measurement string.)

 

also, i personally would avoid to call realloc at each timer tick. along the array, use 2 values: size and count, size keeps track of the total size of the array and count keeps track of the number of valid samples stored in the array. allocate a big array at the beginning, and make it grow by big steps only when needed. the idea is to always have enough room available so that you avoid calling realloc too often.

0 Kudos
Message 2 of 9
(3,855 Views)

dummy_decoy wrote:

maybe not the problem at all, but... in your code, you wrote:

/* ...     read value from the measurement string*/

are you sure the  measurement string does not need to be freed once you are done with it ?

(note: i never worked with an Agilent 34970, so this is pure speculation on my side. i don't know how you retrieve the measurement string.)


 

I use a global variable, a fixed length string of 65K char, in which an instrument library function will place measurements looking like the following string: 27.150000000000,27.320000000000,27.190000000000, 27.010000000000 ...

 

Each measurement is represented in the string as a float, 15 char long, than comma, then next measurement. So at each sample-timer tick, I will read from the instrument about 50 measurements (in simulation mode I generate about 1500 random values to accumulate samples faster).

 

Last week, I had a similar problem, I was running out of memory after 100K samples, but I found that I forgot to delete previous graph plot, and I guess that all plots were accumulating, the system become very sluggish. In the meantime I fixed this and hoped to have got rid of the problem, now it's better, but it still running out of memory after 1.5 Million samples.

0 Kudos
Message 3 of 9
(3,851 Views)

dummy_decoy,

 

I made a trial, based on your suggestion to reduce the number of calls to realloc, I went further, and declared Y_Array as a fixed, 8 Million integer element array. The problem is still there, the application will run out of memory after Million samples. I also changed the simulation mode to place samples directly in Y_Array, without the instrumentation string processing, that I described in a previous post.

I managed to narrow further the issue, and it seems that graphic library functions are creating this problem, I placed a switch on the GUI to disable plotting of the data array, and the application worked without a problem taking 7 Million samples, without having excessive memory allocation - it used about 50 MBytes.

 

Beside the fact that plotting such a big array to a graph, might be unusual, is there something else wrong in the code ? Having this two timer configuration, should require using thread safe variables ?

 

Regards

0 Kudos
Message 4 of 9
(3,825 Views)

Did you ever find the solution?  I have a similar temperature recording appication that bombs out after several  hours of recording at 1sps.  In the task manager it says the program is using something like 600mb of memory!!!

 

I am using the scattergraph with the append method.

0 Kudos
Message 5 of 9
(3,385 Views)

Hello,

 

I have used a local Array in display callback, In which I first copy all values from the main array. In this way I tried to separate Display from Sampling task, as I was suspecting that main array was altered by Sampling Task while Display is in progress.

 

Regards

0 Kudos
Message 6 of 9
(3,362 Views)

Hi All,

 

One thing you may consider is not plotting all 1.5 millions points of data. If you think about what is practically being displayed, this is overkill. There are only so many pixels on your screen (certainly not 1.5million wide). The resolution of your plot is going to be limited by the pixel width of your control. If you are not using any zoom features, then I would suggest only plotting a subset of those points of data. You could try using an averaging algorithm to reduce your data set to something more managable.

 

Mini Me, did the local array reduce the memory your application is using?

National Instruments
0 Kudos
Message 7 of 9
(3,342 Views)

D Biel

 

I agree with you about the fact that 1.5 million samples cannot be presented on a X axis of a graph display.

But the user interface must allow user to zoom into this volume of data, and using the zoom feature of the Graph control was a convenient way of doing it

Are there limitations specified by NI on the volume of data passed to PlotY ?

 

Regards

 

 

0 Kudos
Message 8 of 9
(3,333 Views)

miniME,

 

No there are not limitations. The limitations come from the memory allocated to the process. There are a few attributes of the graph that can be used to help reduce the memory usage. See the CVI Help topic Library References>>User Interface Library>> Controls>>Control Types>>Graph Control>>Programming with Graph Control. In particular the section titled Optimizing Memory Usage. The ATTR_COPY_ORIGINAL_DATA will specify if a copy of your data is kept or if it uses a pointer to the original data.

National Instruments
0 Kudos
Message 9 of 9
(3,307 Views)