How can I read CSV files of rows with different number of columns?

[iyjang]

HI all, 

I'm trying to load CSV files with DIAdem 2014, and I found there is a great tool, 'DataPlugin'.

However, with CSV DataPlugin tool, there is critical problem with my data files. (See attached picture)

As you can see the attached sample CSV file, it includes many lines with different columns.

And here, DataPlugin reads the number of columns of 1st line (e.g.: 4 columns), and set it as the number of columns to read.

So I cannot load the whole data(e.g.: 53 columns) if the data of first line has columns less than that.

Can anyone recommend any approach, please?

With best regards,

Young

Admin Note 3/24/16: Files removed per user's request

[AndreasK]

Your file seems to be a buslog file.

Is there a binary soure file that might be readable with an existing dataplugin or the buslog converter.

I would assume that beeing able to read the file is only half the truth.

At the end you need one group for each distincted Start Text.

Is this true?

[iyjang]

Andreas,

I don't think there's source file that is readable with existing plugins.

And for the rest part of your answers seem quite difficult for me. I'll be thankful if you can explain with a little more specific words..

Anyways thanks for your comment.

[AndreasK · Accepted solution]

I just add some lines of script and attach the related dataplugin. (I just added the code to have a direct look.)

The uri file is intalled by double click and will show up as new load type in the diadem file dialog.

It would be nice if you could provide some information how the data was created. Maybe we can add it as official plugin to our DataPlugin web page.

Option Explicit

Sub ReadStore(File)

  File.Formatter.Delimiters = ","
  File.Formatter.LineFeeds = VBLf
  File.Formatter.DecimalPoint = "."
  
  dim startLine : startLine = file.GetNextLine()
  if 1 <> instr(startLine, "[Tenergy Bus Log Data]") then
    call RaiseError()
  end if
  
  While not File.Position = File.Size
    dim groupName : groupName = file.GetNextStringValue(eString)  
    if not root.ChannelGroups.Exists(groupName) then
      root.ChannelGroups.Add(groupName)
    end if
    dim grp : set grp = root.ChannelGroups(groupName)
    dim i : i = 1
    do while true
      dim val : val = file.GetNextStringValue(eR64)
      if isempty(val) then
        exit do
      end if
      if grp.Channels.count < i then
        call grp.Channels.Add("Ch_" & i, eR64)
      end if
      dim chObj : set chObj = grp.Channels(i)
      chObj.Values(chObj.Size + 1) = val      
      i = i + 1
    loop
    Call File.SkipLine()
  wend

End Sub

[iyjang]

Thanks very much for your assistance, Andreas K.

It works well with my software, and seems to be nice script which can be modified for future applications.

Anyways, the related tasks will take some time to proceed further, so I'll let you know if we think we should add this to Dataplugin web page.

Thanks again.