03-29-2010 12:56 PM
Hello, my name is Ken and i write on behalf of rooky team 3336 from Swansboro high School in NC. Our regional is in Raleigh is this weekend and we have recently discover a problem that will likely prevent us from competing and I hope I can find some help. I have reviewed reams of documentation and forums and thus far can not find a solution. Sorry that this is quite lengthy but I felt it critical to list all the facts as we know them. Help!
Here are the details:
Problem:
Randomly all I/O dies (changes to an off state) for a fraction of a second and the come back. This included PMW signals to the Jaguars, solenoid output from the cRIO, and relay output. This happens at random intervals and can range from every 10 seconds to up to 20 minutes.
Additional information:
We are running Java.
During the brief I/O loss periods the “Link” LED on the cRIO appears to change to solid (stop flashing).
Forcing the operation mode to autonomous to force the indicator lamps (robot signal lamp and the LED on the side car) to light solid reveals that during these interruptions the indicators begin to flash.
The user watch dog has been disabled for trouble shooting with no impact.
The latest mandatory Driver software (DSupdate1.1for2010.zip) and mandatory NI updates (FRClabviewUpdate2.1for 2010.zip) have been downloaded an installed (we think this is when the problem began). See question on version validation.
We have eliminated wireless communication as root cause by running with a crossover cable from the classmate to the cRIO.
The driver station “communications” indicator stays green during these I/O drop outs.
The cRIO imaging tool has a date of 2009, although the NI updates have been applied.
Driver-station reports NI version V20 and DS version 02.10.??.?? or 10.02.??.?? (can’t remember exactly but it appears to be Feb, 2010).
We are using the classmate for development and the driver station. The problem exists using both log-ons.
Power for all devices stays up during these I/O drop outs.
Robot code functions fine except during these I/O drop outs.
Conclusions:
Due to the indicator (on the side car) beginning to flash I believe that the system watchdog is triggering the shutdown of I/O.
Due to the “Link” LED on the cRIO changing to a solid state I believe we are momentarily loosing communications between the driver station and the cRIO long enough for the system watchdog to issue the shutdown of I/O.
These brief losses in communications are not long enough for the drive station to detect, react to, and refresh the screen thus the communication status stays green.
Questions:
Are my conclusions on track? If not what other possible issues might there be?
If my conclusions are correct:
What possible causes might be causing the loss of communications causing the system watchdog to trip?
I understand the system watchdog timeout value is controlled by the imaging of the cRIO and can not be changed. How do we validate the revision of the cRIO image, and what should it be?
The “Java Updates” document from the “First Robotics Resource center” states that the latest Java version requires the V20 NI update (we have). How do I verify the current Java revision and what should it be? If Java is a down rev, and we have V20 could this be the issue?
What is the current “Imagining tool” revision?
What is the current revision of the Driver-station as reported on the driver-station screen?
What might be root cause of communications issues (especially when running wired)?
What else am I missing?
Any additional help beyond answering the direct questions would also be appreciated. We are stuck, have run out of time and are desperate! Should this problem occur during competition we will have one spastic robot! Pleeeaasseeee! Thank you!
03-29-2010 02:00 PM
Hi Ken,
I don't have the answers to all your questions just yet but here are some:
What possible causes might be causing the loss of communications causing the system watchdog to trip?
In the messages window, do you see messages that say "Watchdog expiration System:x, User: 0"? where x is a number that indicates how many times the watchdog was not fed. If not, and you see the value after the user increasing, it's the user watchdog that is complaining about not being fed. If it is the System watchdog that is causing this error, I'm not sure what may be causing it.
I understand the system watchdog timeout value is controlled by the imaging of the cRIO and can not be changed. How do we validate the revision of the cRIO image, and what should it be?
You can check which version of the image you have by looking in the Diagnostic tab of the Driver Station. It should say that the DS version is 10.02.08.0 and that the cRIO is image is v20.
The “Java Updates” document from the “First Robotics Resource center” states that the latest Java version requires the V20 NI update (we have). How do I verify the current Java revision and what should it be? If Java is a down rev, and we have V20 could this be the issue?
I'm not too familiar with programming in Java. However, if the Java revision is not using the v20 update, it should throw an error when you try to deploy/run the code and the program should not run at all.
What is the current “Imagining tool” revision?
The imaging tool revision should not be a factor in this. It essentially deploys the specified cRIO image and configures the IP address. The important versions would be of the image itself, the DS and the Java updates.
What is the current revision of the Driver-station as reported on the driver-station screen?
10.02.08.0
What might be root cause of communications issues (especially when running wired)?
Are you using a cross over cable?
Is there anything in your code that is taking a long time to execute? (for example, do you have a wait in there or are you trying to view images from the camera in the largest size?)
Also have you tried to build and deploy the default robot code? If not, please try this and see if you still experience the I/O drop out. If you do not see the I/O drop out, then I suspect there is something that you've changed in your program that might be causing the issue.
~Olivia
03-30-2010 11:37 AM
Olivia,
Thak you very much for the help!
Last night we had a total network failure. Somehow the router and bridge went south. I spent most of the night attempting to figure out what was wrong. All the setting looked ok but we couldn’t ping the bridge or cRIO. Finally gave up and reset both back to defaults and then reconfigured (with same setting). This fixed the problem but at midnight I quite. Unfortunately I didn’t get a chance to even work on our real problem.
Tonight I will continue, however I wanted to answer the questions you asked. We hav not seen any error messages. tonight we will capture the console output (if any) and post it. Also, we have used a cross over cable with the same problem
Thanks agin, and stay tuned, pleeeaaaassssseeeee!
Ken