Control System Issues

[Mastonevich]

Just wanted to share 2 problems 1987 had over the weekend.

1.
While the students were working in the pits on some minor coding, our driver station started randomly losing connection with the robot while it was tethered. It was as random as random could be.

We had our entire control system unplugged at one time or another. We would reboot the the robot and see if we had connection. I think the max number of reboots was 4 attempts, before we would simply not have robot communication. This entire process took us 4 hours and two missed matches.

We ended Friday by taking our driver station back to our shop (thankfully we were at our local regional this time) where we had a test electronics board setup. We were able to recreate the problem. Thanks to 1730, our sister team, for troubleshooting with us as they were in the pits with us and after heading back to our shop they brought over a backup driver station. 1730 was always able to connect to the test bed, but we were not. After an hour of doing this we noticed that our DS software was a little different version. I think we had the 1.28 release, they had the latest. We updated the DS which requires a reboot and after the reboot, all was well.

I think the reboot fixed our issue but we have nothing 100% concrete.

I am guessing others already reboot their driver station as part of pre-match setup, and it will certainly be part of 1987s pre-match procedures now.

It leaves me wondering could there be a memory leak with the DS software, either induced by something we did or just inherent? Other thoughts, ideas?

Setup: 1.28 DS software, C++ Code with WPI Library on current release

2.
We had to replace our digital sidecar twice during GKC. No idea why they were going bad, but they were. As I understand other teams went through digital sidecars during the regional also, but I did not personally see it first hand.

[jspatz1]

Originally Posted by Mastonevich

2. We had to replace our digital sidecar twice during GKC. No idea why they were going bad, but they were. As I understand other teams went through digital sidecars during the regional also, but I did not personally see it first hand.

1986 went through two digital sidecars as well. The CSAs I spoke with felt they were seeing unusually high failure rates in this year's KOP sidecars.

[Woolly]

Originally Posted by jspatz1

1986 went through two digital sidecars as well. The CSAs I spoke with felt they were seeing unusually high failure rates in this year's KOP sidecars.

That's worrying. We're using our KOP sidecar, but we haven't had an issue with any electronics besides a blown spike fuse.


I am curious about what situations they failed in not to mention where they are placed on the robot.

[Greg McKaskle]

The last reported and confirmed DS memory leak was from 2010 beta. The changes between the '13 versions only have to do with info being logged to the DS files.

What dashboard was being used? Could that have contributed?

Without more information, my suspicion is that the OS itself wanted a reboot. I occasionally awaken a laptop from sleep and see that all sorts of stuff is bogging it down. Do you have any idea what was running? What OS version?

My recommendation would be to reboot it each morning of a competition. I don't know that I see a need to do it before each match. I'm sure there are teams that reboot it once or twice before each match. Whatever works for your team.

Greg McKaskle

[Mark McLeod]

We boot our Driver Station netbook at the beginning of a competition and shut it down after the end.

[Mastonevich]

DS:
We were using the regular dashboard.

It is a Win7 machine. We use it as programming laptop and our driver station. It is a pretty beefy machine with a SSD harddrive. Windriver and the DS were probably the only thing running at the time, but I can't say for sure.

We did have interesting issue that day. We use C++. The students tried reverting the code in wind river. The deleted the project, but they didn't delete the files from the file system. They then imported old code with the same project name. Who knows what happens as far as the class files on the system and how it produces the .out file from that point. It may have been possible to trigger from that, but I am skeptical. I am going to try to recreate this scenario sometime and see if we can recreate the issue.

Sidecar:
Our sidecar is in the middle of our robot surrounded by lots of other robot stuff. If I remember at Alamo, I will try to take a picture and post.

@Greg, yes it is like shooting in the dark with issues like this. Hopefully this will bring some awareness and others can take down more details if it happens to them.

[Jon Stratis]

We lost our KoP sidecar at Lake Superior and had to replace it with one from a previous year. The 5V LED went out, and we lost all PWM signal. I figured some metal shavings got in there and shorted something.

But if you want to talk about confusing control system issues... Last night we were at our build space to practice driving on last year's robot. While the driver was practicing, the programming team was working on getting another robot up and running again so we can have some more interaction in the practice. Well, they accidentally downloaded (over wireless!) to the wrong robot! We spent at least half an hour trying to figure out why it wasn't driving anymore (rebooting both the robot and driver station multiple times, opening up the robot to look at the electronics, etc) before we discovered why.

[Woolly]

Originally Posted by Mastonevich

Sidecar:
Our sidecar is in the middle of our robot surrounded by lots of other robot stuff. If I remember at Alamo, I will try to take a picture and post.

Is it mounted horizontally or vertically, because we tend to notice an increase in reliability of our electronics when they're mounted on a vertical plane. Seems to be that less dust, dirt, metal shavings, and other assorted fouling settles on/inside them.

However, 2 sidecars failing in one regional seems to be excessive if that's the cause unless it's mounted right next to or directly under a moving part, or you did a lot of drilling and filing without a vacuum nearby.

[Mastonevich]

Of course there was some drilling being done and maybe that was the cause. We were very careful to cover up and use a vacuum though. This is the first sidecar we have ever lost.

[Kevin Sevcik]

Originally Posted by Mastonevich

Sidecar:
Our sidecar is in the middle of our robot surrounded by lots of other robot stuff. If I remember at Alamo, I will try to take a picture and post.

@Greg, yes it is like shooting in the dark with issues like this. Hopefully this will bring some awareness and others can take down more details if it happens to them.

If you continue having issues at Alamo, you can hunt me down. We have a spare driver station and several spare Sidecars.

[Mastonevich]

Originally Posted by Kevin Sevcik

If you continue having issues at Alamo, you can hunt me down. We have a spare driver station and several spare Sidecars.

Thanks for the offer, I just hope we don't have to take you up on it.