csv file size difference between python and labview

[IPach]

Hello everyone,

I have a csv file stored from a DAQ system. This file has its columns separated by ";" and its size is 14387kb. There is an issue on the file storage logic that duplicates the datapoints stored, so the real data should be 7193.5 kb.

Here comes the mystery, I load the csv in python, drop the duplicates and save the file in csv, separated by ",". The magic is that the new file is 128kb!!! 56 times smaller!! I've checked the number of decimals and is the same in both files.

Anyone knows what is happening here?

[MaSta]

One thing is unicode. Doubles the size. So the second save could be non-unicode. 

Next, if the default is american CSV format, with comma as decimal point and semicolon as separator and values perhaps being stored as

xx,xxxx (3 or more decimals) and converting the CSV to have comma as separator, your file might have lost all the decimals. But I guess

that'd be too obvious and would have seen for yourself.  

[Basjong53]

Can you attach the csv file?

[crossrulz]

---

@Basjong53 wrote:

Can you attach the csv file?

---

I'd also like to see the generating code to see if we can fix the duplication issue.