Strategy for logging many DAQ channels and modbus addresses to database

[HobieTcat]

I am trying to log data from about 1000 tags from a PLC and about 100 analog channels from a cDAQ-9179. There seems to be some limitations to the number of Modbus addresses(256) that can be read at one time as well as the size of clusters(256) that I am trying to use to handle the data. I am logging data to a SQL Server database at 1 sample per second as well as displaying a few hundred of the tags and channels on 4 Front Panels that are on different monitors. I am running into problems dealing with all the data as well as dealing with all the data points when I just need points scattered around in the arrays and clusters. The fastest the loop can run is around 4 seconds is one problem, but just trying to deal with arrays and bundles that are hundreds of elements long is proving to be a challenge too.

Is there a good method to deal with large amount of data points?

[ZYOng]

Please share your code so that we can provide recommendations. 

[HobieTcat]

Here is a version of most of it. I am hoping this code is not the basis for anything, it doesn't work very well and is not scalable. I am wondering if anyone else has done a project where the limits of Modbus addressing or the size of clusters was an issue and they found a good way to handle it instead of unwieldly bundles and arrays? And hopefully a faster way to get Modbus data.

[ZYOng]

I have little experience with Modbus so I gonna left that to other people.

I have two recommendations:

1. Move the DAQmx Create Virtual Channels VI outside of the For Loop. You only need to initialize the DAQmx hardware once, just like Modbus Master and Database.

2. Writing to the database takes time as well. You should consider using Producer/Consumer Architecture and moving the DB Insert to another loop. 

[HobieTcat]

Thanks! I'll check it out.