02-15-2018 10:58 AM
First of all thanks again for all of your thoughtful replies.
My head is still wrapped around the DAQmx transition diagram.
Here are some more observations, the thought process behind them probably misconstrued.
I will start at the beginning. I have a 10+ year old laptop, Compaq brand if any of you kids remember it. It is a Core Duo at 2 GHz 4GB ram, only 3.5GB addressable, and runs 32 bit Win7. This is my worst case scenario test case for my programs. If they can run there, they can run anywhere.
When the stars are correctly aligned, I can stream 8 channels at 2MSa/s for each channel in LOG ONLY Mode with this laptop. (NI-6366) Not too shabby. The key was manually creating the DAQmx buffer for this task. Try to make it too big, then not enough resources error, too small then cannot keep up with the acquisition error. It is my understanding that this buffer "lives" in kernel memory space. You can find discussions where users increased their user memory using bcdedit /set increaseuserva and then had problems with DAQmx allocating memory.
When I was using DAQmx 17 or earlier I only used the DAQmx primitives, Start, Stop, and Clear, nothing else. Never had a problem. Once I updated to 17.6, then a constant start/stop/clear/create task would give me this not enough resources error. This occurred even if I was only acquiring a few thousand points. Unplugging, resetting the DAQ had no effect, I had to restart Windows, that is why I believed some sort of garbage collection is not occurring.
Why did I update to 17.6? Well, before the holidays, my hard drive crashed and I needed to reinstall everything. So I thought, why not the latest and greatest. Can I revert? Yes, I can, but I have no idea whether new versions will follow the old model or the new model introduced in 17.6. At some point I will need to update, might as well do it now is my thought.
I have not yet tested my final version with my Compaq testcase. However, a preliminary test that used the abort and other "Control tasks", and reused tasks where possible, looked promising. No more out of resources errors. I do not understand it, but maybe I do not need to, if it works.
I could add delays between start and stop, but prefer not to. I do not want the user to feel like they do not have control. "Why is it waiting, is something wrong, should I restart it, etc." One thing that stuck with me was reading about the old MacOS. The driver for the mouse was the hardest thing to crash. This way when the system froze, which occurred often, the user will still be able to move the mouse giving them some sense of control. So I always try to make my programs from that point of view, make the user feel like they have control, and to me, that means a responsive UI.
As far as educating users, I rather not. This program is taken into the "field" to run for weeks at time. Depending on the conditions, their "task" may change. Below is a screen shot of the front panel, there is not too much choice. It is an embarrassingly simple program, that is why it so frustrating when it does not work. (If you like the Win10 style controls you can download them at DMC, BSD or CC license.)
Thanks again for all of your replies, you guys have saved my job multiple times!
Cheers,
mcduff
02-15-2018 01:15 PM
The DAQmx task transition diagram is helpful but not complete. For example, it doesn't show that when a task is stopped, it reverts to whatever state it was in when it was asked to start. How do I know about this? Probably from one or several discussions with NI folks here on the site some years back. The diagram is a helpful starting point but it's not the final word on everything about it.
I'd continue to be *quite* leery of any solution that depended on setting the task state to "abort". Some info suggests it should be benign, other info hints that it may not be. I would start from a position of distrust.
I would definitely try backing down to a DAQmx version that previously didn't require messing around explicitly with the task state model. Don't be shocked if you end up having trouble now with the older version -- after all DAQmx likely wasn't the only change when you rebuilt your system from the ground up.
The things you relay about kernel memory are likely quite relevant, I just have no specific insight. I've never "needed to know" during lots of years of lots of DAQ on PCI, PCIe, and PXI boards, so I'm inclined to wonder if this is one more of those limitations of USB-based DAQ.
My counterargument about the "responsive UI" goes like this: retrying a Create/Start multiple times induces only a *necessary* delay -- anything shorter would result in a failure to function at all.
I *do* agree with the idea of holding onto the old task ref *if possible* for faster restarts; most of my side of the conversation has taken as a given that we're in a scenario where it isn't.
Suggestion for next steps:
1. Revert DAQmx to version that worked smoothly for you without explicit use of task state model
2. Remove any calls to DAQmx Control Task.vi Rely on stuff like Create Virtual Channel, Timing config, Start, and Stop.
3. Do not clear tasks until exiting the program. When reconfig is needed, Stop the task, reconfigure using the same task ref, then Start it again.
I *think* this is similar to where you were at before the hard drive crash, right? If it still works, then boom, you're done. If not, we'll have something more to talk about.
-Kevin P
02-15-2018 03:30 PM
Kevin,
For the most part I wholeheartedly agree with you, and in an ideal world I would follow some of those recommendations, but I'm stuck.
DAQ: Stop // Stops Task
DAQ: EndTask // Clears Task
DAQ: StartNewTask // Creates a New Task, Easier to debug with a task name
DAQ: CalculatePoints // Calculations nothing to do with task
DAQ: CalculateIterations // Calculations nothing to do with task
Message: DataSamplingRate // nothing to do with task
Message: PlotUpdateTime // nothing to do with task
Data: DeviceName // nothing to do with task
Data: NameArray // nothing to do with task
Message: SendScalingValues // Calculations nothing to do with task
DAQ: SetChannels // Set Channels in Task
DAQ: SetClock // Set Sampling Rate
DAQ: Logging // Set Logging Properties
DAQ: SetBuffer // Manually set size of the buffer
DAQ: SetFileProperties // Set the File write properties important for Log Only
DAQ: SetTransferProperties // USB Transfer Properties
DAQ: SetCouplingAndBias // AC/DC Coupling Bias for IEPE sensor
Data: SetDataArray // nothing to do with task
DAQ: RegisterEvents // Using DAQ events to update the display
then DAQ: Start
Again this had previously always worked, sorry to be a broken record. Now I am basically using the same macro except when I reuse a task, the DAQ: EndTask is removed.
Kevin, you have been a great help and resource, I am adding your name to the "About" page. Thanks for taking the time and effort to reply to these posts. Hopefully my answers have not been too frustrating.
Cheers,
mcduff
02-15-2018 03:42 PM
I needed to whiteout my real identity, 🙂
mcduff
02-15-2018 04:29 PM
@mcduffI needed to whiteout my real identity, 🙂
mcduff
Batman
02-15-2018 05:39 PM - edited 02-15-2018 05:44 PM
No frustration with the messages, it's the problem itself that's a brow-furrower.
As things stand *right now*:
- what are all the calls to "DAQmx Control Task" and when within that state sequence do they each occur?
- does StartNewTask call "DAQmx Create Task", "DAQmx Create Virtual Channel", or both? (Personally, I've almost *never* used "DAQmx Create Task" directly but it appears that's the only way to assign a name to a task. The task property isn't writeable.)
- do you get resource errors only on newly created tasks, only on re-used task refs, or can it happen on either?
- when errors occur, are they only thrown in the Start state? (I would expect this to be true when there are no explicit calls to "DAQmx Control Task").
- have you yet tried to put a loop around the calls to DAQmx Start? Isn't it worth a try to see whether the lack of resources is merely temporary? Even if you want a better long-term fix, it'd be worth knowing whether this could be a fail-safe before your trip starts. (I'm not particularly confident it'll help, but it's at least plausible it *might* and is a pretty easy thing to try).
In looking over all the different DAQmx related states you outlined, there are some other things I haven't typically done. These are other areas where I can't speak from experience and would thus tend to focus some of my suspicion
- I don't typically use direct TDMS logging
- I haven't often used DAQmx events
- I doubt I've used either one while also making explicit calls to DAQmx Control Task
- I also doubt I've used either one while also stopping a task, then hanging onto the task ref to reconfigure and restart it
I do notice a state for registering a DAQmx event, but no state for unregistering. Maybe this could be a monkey wrench in the process of freeing and reallocating task resources?
-Kevin P
02-15-2018 06:10 PM
Need to pick up the kiddos soon, so I'll try to answer your questions.
what are all the calls to "DAQmx Control Task" and when within that state sequence do they each occur?
I use the control task before a start and stop task as of now. They are integrated within my DAQ Start and Stop State.
does StartNewTask call "DAQmx Create Task", "DAQmx Create Virtual Channel", or both? (Personally, I've almost *never* used "DAQmx Create Task" directly but it appears that's the only way to assign a name to a task. The task property isn't writeable.)
It calls DAQmx Create Task. I did this so I could explicitly watch Tasks Clear and restart, blank names were not working. Hard to debug with probes.
- do you get resource errors only on newly created tasks, only on re-used task refs, or can it happen on either?
Before I changed my program only on newly created tasks. It often occurred after some arbitrary limit, for example, after the 20th new task was started. Note that the previous 19 tasks were cleared supposedly.
- when errors occur, are they only thrown in the Start state? (I would expect this to be true when there are no explicit calls to "DAQmx Control Task").
When I got the error the device would basically be reset. I would then start from the beginning and select the device and start a new task, but, I would get the same error immediately.
- have you yet tried to put a loop around the calls to DAQmx Start? Isn't it worth a try to see whether the lack of resources is merely temporary? Even if you want a better long-term fix, it'd be worth knowing whether this could be a fail-safe before your trip starts. (I'm not particularly confident it'll help, but it's at least plausible it *might* and is a pretty easy thing to try).
No, worth trying though.
In looking over all the different DAQmx related states you outlined, there are some other things I haven't typically done. These are other areas where I can't speak from experience and would thus tend to focus some of my suspicion
- I don't typically use direct TDMS logging
Highly recommended. Extremely efficient. Most people where I work do not like TDMS, so I make a converter to convert them to MatLab. Added bonus, if the user is logged in as an administrator, you can use a DAQmx property to preallocate the file, not sure why you need to be an admin, but highly efficient. Why do I care about efficiency, sometimes where we work there are no power spigots, and people want to take there data as long as possible with a laptop battery. With the log only mode the CPU usually stays below 5%.
DAQmx Events
Changed my life. 🙂 I use the JKI State Machine and communicate between loops with user events. I can have DAQmx events come to the same queue. In my case, I do not let the user arbitrarily pick any allowed sampling rate, I only allow sampling rates that are evenly divisible by 10. (Actually a little more stringent than that, but not important). Why 10, I have the DAQmx Event tell me when 100ms of data has been collected, no need to poll, it is an event! This gets added to the data display and a plot for the user to see. (Minimum time collection is 100ms) For my system, I put the condition that the data display is updated every 1 million points or every 10 s, whichever one occurs first. This keeps the memory requirements light, and gives a responsive feedback to the user.
I do notice a state for registering a DAQmx event, but no state for unregistering.
Good catch!
So when I reuse a task I need to unregister it before I can re-register it. I have never unregistered the task when I clear it, and start a new one. The Trace Execution never gave a memory leak for this situation. I do reuse the event wire as such, see below. But maybe I should unregister it.
The event wire is reused inplace. Note for Log Only data there is no display and no need to register for the event.
Thanks again.
Cheers,
mcduff
02-15-2018 06:25 PM
On a phone now. But, look through my tag cloud for required reading and or DAQmx states. Sean wrote a masterful explaination of implicit and explicit transitions and when they are supposed to happen and what they should do.
Caveat should isn't does. Hence my preference for explicit transitions.
02-15-2018 08:14 PM
I think this is the tag you are talking about. https://forums.ni.com/t5/LabVIEW/Deleting-channels-from-task-reconfiguring-task/m-p/1544490#M571637
Will take awhile to sink in.
mcduff
02-16-2018 07:49 AM - edited 02-16-2018 07:56 AM
Yes, that's the one.
I should have tagged it duct tape recommended. Worth a read more than once.
You might also be interested in the benchmark code I had posted earlier in the thread. Kevin's suppositions about timing were confirmed ....on that platform of software! And my benchmark code could be improved.
Great discussion