Rif.: Read datas of .txt file according to the list using CVI

[RobertoBozzolo]

xiepei,

as I already told you there is no easy way to calculate the array size while reading an ASCII file. You could for example make a first pass on the file reading and counting lines, next dimension your array, move the pointer and start over with the file read. This process is easy to program but make intensive access to disk and therefore can last a lot of time on large files.

Another way is to make an initial dimensioning of the array, next enlarge it if you are reading more lines from the file. Something like this:

#include <formatio.h>
#include <toolbox.h>

int	r, cnt = 0, aSize = 1000, fH = 0;
double	*mis = NULL;
char	buf[512];

// Initial array dimensioning
mis = malloc (aSize * sizeof (double));

fH = OpenFile ("myFile.txt", VAL_READ_ONLY, VAL_OPEN_AS_IS, VAL_ASCII);
while (TRUE) {
	r = ReadLine (fH, buf, 511);
	if (r == -2) break;
	if (++cnt > aSize) {
		aSize += 500;   // Increase array size
		mis = realloc (mis, aSize * sizeof (double));
	}
	// Parse line read and fill the array
	Scan (buf, .......);
}
CloseFile (fH);

After that you have read 'cnt' lines and filled up a corresponding array to plot.

BRW Have you tried scanning the lines as I suggested you in my first reply?

But I seem to understand from the discussion that you have some control on the program that originates data. In this case passing to a binary file can ease up all the matter. Thake a look at this solution:

// PROGRAM THAT ORIGINATES DATA
#include <formatio.h>
#include <toolbox.h>

typedef struct {
	char	datetime[20];
	double	wj;
	double	mv;
	double	ma;
} myData;
myData *d = NULL;

int		i, cnt, fH = 0;

// Dimension the data array
d = calloc (cnt, sizeof (myData));

// Write array elements
sprintf (d[i].datetime, "%s  %s", DateStr (), TimeStr ());
d[i].wj = ...;
d[i].mv = ...;
d[i].ma = ...;

// Writing the file ('cnt' elements)
fH = OpenFile ("myFile.txt", VAL_WRITE_ONLY, VAL_TRUNCATE, VAL_BINARY);
WriteFile (fH, (char *)&d, cnt * sizeof (myData));
CloseFile (fH);





// PROGRAM THAT READS DATA
#include <formatio.h>
#include <toolbox.h>

typedef struct {
	char	datetime[20];
	double	wj;
	double	mv;
	double	ma;
} myData;
myData *d = NULL;

int		i, cnt, size;

// Calculate number of elements
GetFileSize ("myFile.txt", &size);
cnt = size / sizeof (myData);

// Dimension array
d = calloc (cnt, sizeof (myData));

// Read the file ('cnt' elements)
fH = OpenFile ("myFile.txt", VAL_READ_ONLY, VAL_OPEN_AS_IS, VAL_BINARY);
ReadFile (fH, (char *)&d, cnt * sizeof (myData));
CloseFile (fH);

[佩林]

Hi

Thank you very much for  your help.I'm sorry.I was out of office. I have modified my code as the following  according to your advice.

static double wj[1000],mv[1000], ma[1000],et; 
static char buf[512];
static struct tm ti;

int CVICALLBACK HfCallback (int panel, int control, int event,
  void *callbackData, int eventData1, int eventData2)
{
 int r, cnt = 0, aSize = 1000, fH = 0;
    double *mis = NULL;
 //   char buf[512];

 switch (event)
 {
  case EVENT_COMMIT:
    // Initial array dimensioning
    mis = malloc (aSize * sizeof (double));
    fH = OpenFile ("psm.txt", VAL_READ_ONLY, VAL_OPEN_AS_IS, VAL_ASCII);
    while (!0)
    {
             r = ReadLine (fH, buf, 511);
             if (r == -2) break;
             if (++cnt > aSize)
    {
             aSize += 500;   // Increase array size
             mis = realloc (mis, aSize * sizeof (double));
              }
             // Parse line read and fill the array
    Scan (buf, "%d[x]%d[x]%d[x]%d[x]%d[x]%d[x]%s[dt#]%f[x]%s[dt#]%f[x]%s[dt#]%f",
                          &ti.tm_mday, &ti.tm_mon, &ti.tm_year, &ti.tm_hour, &ti.tm_min, &ti.tm_sec,wj, mv, ma);
//    // Adjust fields for tm struct correct format
     ti.tm_mon--; ti.tm_year -= 1900;
                // Calculate timestamp
     et = mktime (&ti);
     
    PlotY (psmppanelHandle, PANEL_PSMP_GRAPH_WJ, wj, cnt, VAL_DOUBLE, VAL_THIN_LINE, VAL_EMPTY_SQUARE, VAL_SOLID, 1, VAL_RED);
    PlotY (psmppanelHandle, PANEL_PSMP_GRAPH_MV, mv, cnt, VAL_DOUBLE, VAL_THIN_LINE, VAL_EMPTY_SQUARE, VAL_SOLID, 1, VAL_RED);
    PlotY (psmppanelHandle, PANEL_PSMP_GRAPH_MA, ma, cnt, VAL_DOUBLE, VAL_THIN_LINE, VAL_EMPTY_SQUARE, VAL_SOLID, 1, VAL_RED);
              }
              CloseFile (fH);
   break;
 }
 return 0;
}

However,when I debug the project and click the data playback botton,only the first group data displays correctly and the others all display zero.Besides,x axis kept bigger,like strip chart  and the whole program are all froze up.

I've been thinking for a long time,but still find the reason.So,still have to ask you for help.

The attchment is the image when I debug project.

Best regards.

xiepei

[RobertoBozzolo]

Hi xiepei,

there is a basic misunderstanding as to how to read data from the file: I suggest you to consider the following items.

1. You are dynamically handling a double array (mis) that you are not using in your code! On the contrary, you are using statically allocated arrays to accumulate data (wj, mv, ma). Modications needed:

double *wj = NULL, *mv = NULL, *ma = NULL, *et = NULL; 

// Initial array dimensioning
wj = malloc (aSize * sizeof (double));
... and so on

// Further memory allocation
wj = realloc (wj, aSize * sizeof (double));
... and so on

2. You are always reading data in element 0 of the arrays and you are not creating the timestamp. Modifications needed:

  Scan (buf, "%d[x]%d[x]%d[x]%d[x]%d[x]%d[x]%s[dt#]%f[x]%s[dt#]%f[x]%s[dt#]%f",
                          &ti.tm_mday, &ti.tm_mon, &ti.tm_year, &ti.tm_hour, &ti.tm_min, &ti.tm_sec, &wj[cnt], &mv[cnt], &ma[cnt]);
// Adjust fields for tm struct correct format
ti.tm_mon--; ti.tm_year -= 1900;
// Calculate timestamp
et[cnt] = mktime (&ti);

3. You are plotting data on every read pass from the file. Additionally, you are using PlotY so that timestamp information is lost. Modifications needed:

// Move these lines out of the loop!
PlotXY (panel, control, et, wj, cnt, VAL_DOUBLE, VAL_DOUBLE, VAL_THIN_LINE, VAL_EMPTY_SQUARE, VAL_SOLID, 1, VAL_RED);
...and so on

4. Finally, you must deallocate dynamic resources on exit:

// At callback end:
if (wj) free (wj);
... and so on

From what I can understand from this thread, you should spend some times studying some arguments on the help and on a good programming book:

- Array treatment (pointers): the way arrays are treated in C explains why you are getting values in array[0] elements only; this affects Scan the same as its ANSI C equivalent sscanf or any other function that should return values into array elements

- Dynamic memory allocation vs. static allocation

- Output on video: both PlotXY and PlotY treat arrays and plot the whole set of data in a singel pass; this explains why you must call them after reading all data and may explain the error you are reporting: it is possible that you have read more lines than the allocation of arrays and you are trying to plot a larger set of data that array size ('cnt' > 1000 in your code)

I hope these suggestions are well accepted: CVI is a powerful instrument the may help you a lot in your work if correctly used. The arguments I suggested you to study are not limited to your actual problem: they are items of general use, instead, that you may use in several applications of yours.

[佩林]

Hi !

Thank you for your help. Your suggestions are well accepted.I just graduated from school and start working.I wilI spend more time studying some arguments on the help and on a good programming book.

Here's the problem that program is always into  dead circulation.

Another problem is when moving these lines out the loop,it not only enters into dead circulation ,but also shows nothing on a graph control.It will draw the curve when moved these lines into the loop.

  PlotXY (psmppanelHandle, PANEL_PSMP_GRAPH_WJ, et, wj, cnt, VAL_DOUBLE, VAL_DOUBLE, VAL_THIN_LINE, VAL_EMPTY_SQUARE, VAL_SOLID, 1, VAL_RED);
         PlotXY (psmppanelHandle, PANEL_PSMP_GRAPH_MV, et, mv, cnt, VAL_DOUBLE, VAL_DOUBLE, VAL_THIN_LINE, VAL_EMPTY_SQUARE, VAL_SOLID, 1, VAL_RED);
         PlotXY (psmppanelHandle, PANEL_PSMP_GRAPH_MA, et, ma, cnt, VAL_DOUBLE, VAL_DOUBLE, VAL_THIN_LINE, VAL_EMPTY_SQUARE, VAL_SOLID, 1, VAL_RED);

Besides,when I use PlotY,it will display datas better on a graph. When I use PlotXY,it only draws the curve between the minmum and maximum .

I hope you could help me see the procedure and find the reason. The attachment is my program.

Thank you very much indeed.

Best regards.

[RobertoBozzolo]

Hi xiepei,

your code is almost good, I only had to exclude ProcessSystemEvents () call inside the loop and avoid scanning empty lines. As per the user interface, simply modify graph x axis format to absolute time and use PlotXY, as I already told you. The result is in the attachment: I saved the user interface for CVI8 (I'm using 8.5) so you shouldn't have problems in using it.

[佩林]

Hi !

Thank you for your patient help.I've modified my procedure according to your code.It could read the specified datas from the file and displays them on a graph.

However, it has a little problem. The time displayed on the X axis is inconsistent with the data time in the file. See the attachment;Graph

Is it because this is txt file rather than a database file ?

In that case,is it better to modify graph x axis format to floating point or use PlotY ?

Frankly speaking,I quite agree with your opinion , modifying graph x axis format to absolute time and using PlotXY.

Best Regards.

xiepei

[RobertoBozzolo]

The inconsistency you are seeing in time axis is because I interpreted the date part in italian style (DayMonthYear) while in the file it is written in MonthDayYear format; simply inverting the corresponding tm fields in Scan () function gives correct results. Additionally, using mktime introduces a time shift based on your time zone. Replacing it with MakeDateTime function ensures that recorded time is displayed as-is on graph axis.

[佩林]

Hi !

Thank you very much indeed.

Simply inverting the corresponding tm fields in Scan() function  has given correct results. Month,day and year are displayed correctly on graph axis.However,hour,minute,second are not displayed correctly. When I replace mktime with MakeDateTime function,error occurs. The following is the part of modified code.

double datetime; 
et[cnt] =MakeDateTime (ti.tm_hour,ti.tm_min,(double)ti.tm_sec,ti.tm_mon,ti.tm_mday,ti.tm_year, &datetime); 

The attachment is the error when I debug project.

I apologize  to you for bothering you once again.

I wish you a happy weekend !

Best regards.

xiepei

[RobertoBozzolo]

I had no problems reading the data file you posted earlier, so I suppose there may be some problem in getting time value from the file, possibly because of a corrupted line or so: when the program halts on error you should verify the content of  ti.tm_sec filed and the string read from file and stored in msg variable.

Also, you should exclude the adjustment on tm_mon and tm_year fields, which are only needed in case you use mktime () function. See some documentation on tm_struct, e.g. here, for a comparison with values accepted from MakeDateTime.

[佩林]

Hi !

Thank you very much indeed.

 You have given me a lot of help.It may be the result of a corrupted line . I'll look into it carefully.

Wish you a happy weekend .

Best regards.

xiepei