Dead RoboRIO 1.0 - FRC Team 4028

[gabeseq]

Monday night at our team meeting, we were met with an unexpected RoboRIO failure during our fall training session. I had one student deploy code to a test board to test her work, and everything worked as expected. I then had a second student then deploy her code to the RoboRIO, and she was met with this error which I’ve never seen before:

RoboRIO Deploy Error1005×817 33.4 KB

On the off-chance that there was some problem with her laptop, I attempted to deploy my example code to the RoboRIO, as I had done several times over the past few weeks, and I was met with the same error. Same thing when trying to re-deploy the first student’s code to the RIO.

After some preliminary poking around, I ended up completely removing all WPILib installs from my laptop and re-installing the latest 2023 WPILib and was still getting the same error when trying to deploy any code to the RIO.

I then thought I would try to re-image the RoboRIO. I fired up the RoboRIO imaging tool, and it could not detect the RIO when connected with the USB cable, even when running the tool as administrator. I disabled my WiFi and turned off all firewall settings, and still couldn’t see it. I deleted and re-installed the latest 2023 FRC Game Tools from NI, and then I was able to see the RIO, but only while the RIO was restarted in safe mode. When trying to reformat with the latest image, I was met with this error.

RoboRIO Error957×567 137 KB

When trying to update the firmware of the RIO, I was met with yet another error.

RoboRIO Error 2895×523 198 KB

I opened a support ticket on Monday but have not received a response, so I'm curious if anyone here has seen this type of behavior before and what, if anything, you managed to do to fix it. Currently, my best guess is that the main storage (I’m assuming eMMC, the documentation doesn’t get very specific here) has died, and that the entire unit will need replaced.

Any help is greatly appreciated

[gabeseq]

After not receiving an email for my support ticket on Monday, I called NI support this evening. I was told I opened the wrong type of request, and was on hold for 15-20 minutes while they cloned my issue from one tech support channel to another. After having my call transferred to tech support to help with this issue, they informed me that they can't help me either, and my only support channel is via this forum.

With that in mind, I'd like to make clear the issue:

I can only connect to the RIO when running the imaging tool as administrator while the RIO is in safe mode. When I do so, I get the errors in the screenshots above.

Thanks!

Gabe, FRC Team 4028

[SSPTest]

Hi gabeseq,

Apologies for the delay in our response. I understand your team has been having issues with the RoboRIO and require some assistance.

I see you've tried several troubleshooting steps (which is perfect), and would like to confirm that you've covered the following:

• Reformat device
• Reimaging the device
• Disable firewalls and antivirus
• Made sure there is only one roboRIO in the network
• Tried a different computer
• Started the RoboRIO in safe mode as administrator

Moving forward, I have a few questions/suggestions for you to try:

• How do the LEDs look? Which color do they show, are any of them flashing?
• What steps did you follow to reformat the device?
• Could you try using a different USB cable that is known to work properly?
• Sometimes debris can fall into the roboRIO through the spaces on the board, try opening the device and using a flashlight to closely inspecte the board, and used compressed air to blow off the board.
• One of the errors shown can sometimes show up if there is no disk space, which can also cause imaging failures, if possible, let's make sure the device only has the necessary software installed.
• Have you tried re-imaging with the previous tool, 2022_v3.0? NI changed the numbering system for their releases, and they may have "built in" incompatibilities with installation

[gabeseq]

I'm not at our shop right now, but I can provide some additional details.

Yesterday I did carefully disassemble the RoboRIO to inspect for any debris inside, and it was very clean, no noticeable damage, debris, or anything else of note.

The power LED turns on green, the RSL light is solid, and the status light does illuminate, but I can't recall if it has any particular pattern to it. When we connect to it with FRC Driver Station running, the COMM light turns red to indicate there is no code. When in safe mode, the status light goes solid for a few seconds and then blinks in groups of 3, as expected. The RSL light is no longer on when in safe mode.

To try to reformat the device, I connected to the RIO using a USB cable (I actually tried 3 different cables that we have used as tethers for other robots) and was unable to see it with the RIO imaging tool. The only combination of factors that allowed me to recognize the RIO were to run the tool as administrator, while the RIO was in safe mode. Having my firewalls enabled or disabled, or having various network interfaces enabled or disabled seemed to make no difference. When running the "Reformat" command, I get the error above about corrupt firmware. When trying to reinstall the firmware using the Firmware Upgrade button, I get the other undefined error message.

How would I ensure that only the necessary software is installed? I even tried to connect to it with SSH and while I don't remember specifically which SSH error I was met with, it was not successful in getting me logged in.

I have not tried a previous version of the imaging tool, only the latest 2023 Patch 1. I will try downloading and using the 2022 version next time I'm at our shop.

[JessNI]

Hi gabeseq,

My name is Jessica, I work at NI and mistakenly used a different account to respond (SSPTest), but rest assured that I will continue to work with you on this case. Apologies for any confusion this may have caused.

Thank you for your reply! Yes, our next step would be to try using the 2022_v3.0 version of the software.

How would I ensure that only the necessary software is installed?

• This is fine as long as we know that the disk space is not an issue and the RoboRIO isn't running out of free space.

• Also, some teams have experienced an issue where imaging fails if the computer name has a dash (-) in it. If so, let's try renaming the computer.

If this is not helpful, please let me know so that I can send you a direct message with further steps.

Jessica Espinoza

Technical Support Representative

NI Support

[gabeseq]

Thank you for the information. I will try these steps the next time I can get into our shop. Our next meeting is Thursday, but I may be able to get by there before then.

[gabeseq]

I uninstalled all NI FRC game tools from my computer and reinstalled the 2022 version of the software. I ensured that my computer name has no dash, or any other symbol in it. When following the same steps as before, I am met with the exact same error messages even with the 2022 software. Per some advice on Chief Delphi, I did open Driver Station and observe that there is 0 MB of free disk space on the RIO. However, if my theory about the flash storage chip being dead is true, that would probably show 0MB disk space as well. I do not know how to free up any disk space, as I am completely unable to connect to it unless it is in safe mode and SSH/FTP are not possible at all.

Are there any additional steps I can try?

[JessNI]

Hi gabeseq,

I've sent you a direct message with next steps.

Regards,

Jessica Espinoza

Technical Support Representative

NI Support

[gabeseq]

Our team has yet another roboRIO 1.0 that is exhibiting the exact same behavior. I've followed these steps again to try and troubleshoot and have been met with the same failures each time. This is NOT the replacement unit that we received last year, but a different unit in our shop that we have been using for testing. Would it be possible to have this unit replaced as well? Thanks.

[JessNI]

Hi gabeseq,

Please post a different thread on the forums to properly keep track of the issues.