Go to Post Something has gone awry this year in FIRST. I think more teams are concerned with winning and their robot that they don't even stop to think about the kids on their team, or the learning experiences of others. - Amanda Morrison [more]
Home
Go Back   Chief Delphi > Technical > Electrical
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 20-03-2014, 21:50
rtrahms's Avatar
rtrahms rtrahms is offline
Registered User
FRC #1778 (Chill Out!)
Team Role: Mentor
 
Join Date: Dec 2013
Rookie Year: 2013
Location: Seattle, WA
Posts: 17
rtrahms is a jewel in the roughrtrahms is a jewel in the roughrtrahms is a jewel in the rough
cRIO cutting out 5 s after collision

Hi all -
Chill Out Team 1778 here. We are chasing an intermittent problem with our robot, and it is usually connected to a large collision. We've been troubleshooting the connections, but wondered if anyone has suggestions on how to isolate this problem effectively.

Robot config:
Basic kitbot chassis with 4 drive motor CIMs, and two additional motors for gate & rolller attachment. All 6 motors controlled by black Jaguars on a CAN bus wired to a 2CAN bus controller and terminated with a 100 ohm terminator. The 2CAN is also connected to a cRIO and on the network side to a wireless router.

Normally everything works fine, but if the robot gets hit hard on the gate, the robot will stop all response for 5 seconds before continuing. During this time the 2CAN shows a flashing red light, and then after 5 seconds returns to green flashing. We have checked the cRIO to 2CAN cable, and nothing seems to be wrong there. 5 seconds isn't enough time for the cRIO to reboot, so something else appears to be going on.

Thoughts?

Thanks in advance,
Rob
  #2   Spotlight this post!  
Unread 20-03-2014, 21:56
Kevin Phan's Avatar
Kevin Phan Kevin Phan is offline
College Student
FRC #0357 (Royal Assault)
Team Role: Alumni
 
Join Date: Apr 2013
Rookie Year: 2010
Location: PA, United States
Posts: 95
Kevin Phan will become famous soon enoughKevin Phan will become famous soon enough
Re: cRIO cutting out 5 s after collision

I know that my electrical engineer mentor would hot glue the can connections on the jags to prevent them from coming out.
  #3   Spotlight this post!  
Unread 20-03-2014, 21:56
yash101 yash101 is offline
Curiosity | I have too much of it!
AKA: null
no team
 
Join Date: Oct 2012
Rookie Year: 2012
Location: devnull
Posts: 1,191
yash101 is an unknown quantity at this point
Re: cRIO cutting out 5 s after collision

Check your wiring. This is typically caused by some loose wires. They are typically in the least suspecting places and have a very high resistance. Something may have gotten disconnected after the collision!
  #4   Spotlight this post!  
Unread 20-03-2014, 22:01
DonRotolo's Avatar
DonRotolo DonRotolo is offline
Back to humble
FRC #0832
Team Role: Mentor
 
Join Date: Jan 2005
Rookie Year: 2005
Location: Atlanta GA
Posts: 7,019
DonRotolo has a reputation beyond reputeDonRotolo has a reputation beyond reputeDonRotolo has a reputation beyond reputeDonRotolo has a reputation beyond reputeDonRotolo has a reputation beyond reputeDonRotolo has a reputation beyond reputeDonRotolo has a reputation beyond reputeDonRotolo has a reputation beyond reputeDonRotolo has a reputation beyond reputeDonRotolo has a reputation beyond reputeDonRotolo has a reputation beyond repute
Re: cRIO cutting out 5 s after collision

Well, some careful thought can tell us what it isn't: cRio power, for starters. Reboot would take longer than 5 seconds.

But you'll have to think about what else it can't be. When you have all those listed, see what's left: One of those has a loose wire somewhere.

Aside from hot glue, this is why we use Zip ties every 4-6" and bundle wires into a harness: Reliability and strain relief.

Good Luck.
__________________

I am N2IRZ - What's your callsign?
  #5   Spotlight this post!  
Unread 20-03-2014, 23:05
Levansic's Avatar
Levansic Levansic is offline
Registered User
AKA: Len Evansic
FRC #0585 (Cyber Penguins)
Team Role: Mentor
 
Join Date: Jan 2012
Rookie Year: 2008
Location: Tehachapi, CA
Posts: 185
Levansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud of
Re: cRIO cutting out 5 s after collision

Check your Ethernet cables and the condition of your router ports. Also check the power connection to the 2CAN.

Red flashing means the 2CAN lost the cRio. As Don mentioned, it takes longer than 5 seconds for the cRio to reboot. It takes ~35 seconds for the router to recover from losing power (that hit us last weekend). Now that I think about it, the 2CAN takes about five seconds to reboot...

We've had small bits of plastic break off of Ethernet plugs and receptacles. The result is a physically loose and electrically intermittent connection. Everything looks fine, until a jolt in the wrong direction breaks a connection.

-Edit- I just re-read the OP. Sounds like a direct connection from cRio (8-slot) to 2CAN? My suggestion was based on the four slot cRio, assuming a router was in play.

Last edited by Levansic : 20-03-2014 at 23:11.
  #6   Spotlight this post!  
Unread 20-03-2014, 23:24
jeremylee's Avatar
jeremylee jeremylee is offline
Registered User
FRC #2526 (Crimson Robotics)
Team Role: Mentor
 
Join Date: Feb 2013
Rookie Year: 2011
Location: Otsego, MN
Posts: 104
jeremylee is a jewel in the roughjeremylee is a jewel in the roughjeremylee is a jewel in the rough
Re: cRIO cutting out 5 s after collision

We have seen issues with the ethernet connections on the 2CAN being touchy in the past. Try wiggling some of these connections and see what happens. We had multiple 2CANs that had the same issue, worth a check.
  #7   Spotlight this post!  
Unread 21-03-2014, 07:10
rtrahms's Avatar
rtrahms rtrahms is offline
Registered User
FRC #1778 (Chill Out!)
Team Role: Mentor
 
Join Date: Dec 2013
Rookie Year: 2013
Location: Seattle, WA
Posts: 17
rtrahms is a jewel in the roughrtrahms is a jewel in the roughrtrahms is a jewel in the rough
Re: cRIO cutting out 5 s after collision

Thanks all for the great suggestions! We will check all those areas, and consider more ways to stabilize and provide strain relief!
Rob
  #8   Spotlight this post!  
Unread 22-03-2014, 00:57
Ian Curtis Ian Curtis is offline
Best Available Data
FRC #1778 (Chill Out!)
Team Role: Engineer
 
Join Date: Feb 2005
Rookie Year: 2004
Location: Puget Sound
Posts: 2,521
Ian Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond repute
Re: cRIO cutting out 5 s after collision

We originally had some suspect cables, these have all been replaced and no amount of jiggling or tugging can cause this fault in the pit. Additionally, artificial vibration in the pit (read: whacking robot with a hammer) cannot replicate this failure.

We replaced the 2CAN today, and still experienced the fault. [just read the comment from 1736... doh!] We have tried powering the 2CAN from both the regulated 12V and 24V regulated sources, and still experienced the fault. I have a hard time believing it is current related, as we have done plenty of high current maneuvering on our practice field without observing it. Additionally, the drivers seem to be reversing direction no problem on the competition field.

At this point we are very good friends with the CSAs. I know there have been some weird fringe electrical issues in the past with the 120 amp breakers doing weird things, and with bad PDBs, but I assume I'll be saying "DOH! Can't believe we missed that!!" when this is all over.

We played lots of back to back matches today, so tomorrow we are changing our cabling such that if we lose the 2CAN the cRio will still be able to talk to the field. As Don said, I don't think we'll get much additional information out of this -- if the cRio was losing power we'd be dead in the water for a long time.
__________________
CHILL OUT! | Aero Stability & Control Engineer
Adam Savage's Obsessions (TED Talk) (Part 2)
It is much easier to call someone else a genius than admit to yourself that you are lazy. - Dave Gingery

Last edited by Ian Curtis : 22-03-2014 at 01:40.
  #9   Spotlight this post!  
Unread 22-03-2014, 07:41
Michael Hill's Avatar
Michael Hill Michael Hill is offline
Registered User
FRC #3138 (Innovators Robotics)
Team Role: Mentor
 
Join Date: Jul 2004
Rookie Year: 2003
Location: Dayton, OH
Posts: 1,580
Michael Hill has a reputation beyond reputeMichael Hill has a reputation beyond reputeMichael Hill has a reputation beyond reputeMichael Hill has a reputation beyond reputeMichael Hill has a reputation beyond reputeMichael Hill has a reputation beyond reputeMichael Hill has a reputation beyond reputeMichael Hill has a reputation beyond reputeMichael Hill has a reputation beyond reputeMichael Hill has a reputation beyond reputeMichael Hill has a reputation beyond repute
Re: cRIO cutting out 5 s after collision

Quote:
Originally Posted by Kevin Phan View Post
I know that my electrical engineer mentor would hot glue the can connections on the jags to prevent them from coming out.
Al has been adamantly against hot gluing electrical connections because it's apparently "not an industry standard" (though I've seen it a thousand times).
  #10   Spotlight this post!  
Unread 22-03-2014, 23:48
Ian Curtis Ian Curtis is offline
Best Available Data
FRC #1778 (Chill Out!)
Team Role: Engineer
 
Join Date: Feb 2005
Rookie Year: 2004
Location: Puget Sound
Posts: 2,521
Ian Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond repute
Re: cRIO cutting out 5 s after collision

Should anyone stumbled upon this in the future, the issue stopped happening today. We had a tightly timed loop with no wait in our teleop code that was driving 100% CPU usage. We added a 10ms delay and our CPU usage dropped down to ~75% (on an 8 slot cRio). We also changed our cabling, so that instead of cRio -> 2CAN -> Router, it went cRio -> Router -> 2CAN. Not sure if it was one of these or the combination, but it stopped happening!

We may try to isolate it further in the off-season.
__________________
CHILL OUT! | Aero Stability & Control Engineer
Adam Savage's Obsessions (TED Talk) (Part 2)
It is much easier to call someone else a genius than admit to yourself that you are lazy. - Dave Gingery
  #11   Spotlight this post!  
Unread 23-03-2014, 00:04
geomapguy's Avatar
geomapguy geomapguy is offline
Centenary '18
AKA: Graham Maxwell
FRC #1818
Team Role: Mentor
 
Join Date: Jan 2012
Rookie Year: 2011
Location: Alexandria, LA
Posts: 535
geomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud of
Re: cRIO cutting out 5 s after collision

Quote:
Originally Posted by Michael Hill View Post
Al has been adamantly against hot gluing electrical connections because it's apparently "not an industry standard" (though I've seen it a thousand times).
We use this http://eclecticproducts.com/ag_adhesives.htm made by one of our sponsors.

And one of our other sponsors uses to secure electrical connections on the pharmaceutical robots they make.

Quote:
Originally Posted by Ian Curtis View Post
We had a tightly timed loop with no wait in our teleop code that was driving 100% CPU usage. We added a 10ms delay and our CPU usage dropped down to ~75% (on an 8 slot cRio).
This can certainly starve the code
__________________
The Cowboys - FIRST Robotics Team #1818 2015-
The Bolton Bear Bots - FIRST Robotics Team #3666 2011-204 (30-26-0)
2014 Oklahoma Regional - 20/62 (6-4-0), 5/62 in OPR
2014 Arkansas Regional - 14/39 (8-7-0, Quarterfinalists with 1706 and 1939), 24/39 in OPR
2013 Lone Star Regional Dean's List Finalist!
2013 Lone Star Regional - 22/57 (6-5-0), 17/57 in OPR
2012 Bayou Regional - 16/49 (6-5-0)
2011 Dallas Regional - 10/52 (4-2-5, #6 Alliance Captain, Quarterfinalists with 2948 and 3350)
  #12   Spotlight this post!  
Unread 23-03-2014, 00:08
RufflesRidge RufflesRidge is offline
Registered User
no team
 
Join Date: Jan 2012
Location: USA
Posts: 992
RufflesRidge has a brilliant futureRufflesRidge has a brilliant futureRufflesRidge has a brilliant futureRufflesRidge has a brilliant futureRufflesRidge has a brilliant futureRufflesRidge has a brilliant futureRufflesRidge has a brilliant futureRufflesRidge has a brilliant futureRufflesRidge has a brilliant futureRufflesRidge has a brilliant futureRufflesRidge has a brilliant future
Re: cRIO cutting out 5 s after collision

Quote:
Originally Posted by Ian Curtis View Post
We also changed our cabling, so that instead of cRio -> 2CAN -> Router, it went cRio -> Router -> 2CAN. Not sure if it was one of these or the combination, but it stopped happening!
Glad you got things working (if not entirely solved). As an aside for anyone else reading, the original configuration as described should not have passed inspection as it violates R56.
  #13   Spotlight this post!  
Unread 23-03-2014, 00:10
Ian Curtis Ian Curtis is offline
Best Available Data
FRC #1778 (Chill Out!)
Team Role: Engineer
 
Join Date: Feb 2005
Rookie Year: 2004
Location: Puget Sound
Posts: 2,521
Ian Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond repute
Re: cRIO cutting out 5 s after collision

Quote:
Originally Posted by geomapguy View Post
We use this http://eclecticproducts.com/ag_adhesives.htm made by one of our sponsors.

And one of our other sponsors uses to secure electrical connections on the pharmaceutical robots they make.

This can certainly starve the code
I know just enough about electricity and software to get myself in a whole bunch of trouble. Is there a plausible explanation why 100% CPU usage would make the 2CAN go out temporarily in a high energy collision?

I can see how our new routing could maybe be the fix. Even though we couldn't replicated it by tugging on cables, maybe there was something specific to the on-field collisions that made certain connectors in certain ports temporarily disconnect.
__________________
CHILL OUT! | Aero Stability & Control Engineer
Adam Savage's Obsessions (TED Talk) (Part 2)
It is much easier to call someone else a genius than admit to yourself that you are lazy. - Dave Gingery
  #14   Spotlight this post!  
Unread 23-03-2014, 00:28
geomapguy's Avatar
geomapguy geomapguy is offline
Centenary '18
AKA: Graham Maxwell
FRC #1818
Team Role: Mentor
 
Join Date: Jan 2012
Rookie Year: 2011
Location: Alexandria, LA
Posts: 535
geomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud ofgeomapguy has much to be proud of
Re: cRIO cutting out 5 s after collision

Quote:
Originally Posted by Ian Curtis View Post
I know just enough about electricity and software to get myself in a whole bunch of trouble. Is there a plausible explanation why 100% CPU usage would make the 2CAN go out temporarily in a high energy collision?

I can see how our new routing could maybe be the fix. Even though we couldn't replicated it by tugging on cables, maybe there was something specific to the on-field collisions that made certain connectors in certain ports temporarily disconnect.
I'm not sure if the two are connected but 100% CPU usage is never good. Maybe some "experts" can chime in about if they is any connection between the two.

Last year we put while loops and sequence tasks in TeleOp and it killed our code. Wasn't until this year that we discovered Periodic Tasks.
__________________
The Cowboys - FIRST Robotics Team #1818 2015-
The Bolton Bear Bots - FIRST Robotics Team #3666 2011-204 (30-26-0)
2014 Oklahoma Regional - 20/62 (6-4-0), 5/62 in OPR
2014 Arkansas Regional - 14/39 (8-7-0, Quarterfinalists with 1706 and 1939), 24/39 in OPR
2013 Lone Star Regional Dean's List Finalist!
2013 Lone Star Regional - 22/57 (6-5-0), 17/57 in OPR
2012 Bayou Regional - 16/49 (6-5-0)
2011 Dallas Regional - 10/52 (4-2-5, #6 Alliance Captain, Quarterfinalists with 2948 and 3350)
  #15   Spotlight this post!  
Unread 23-03-2014, 00:29
orangemoore orangemoore is offline
Registered User
AKA: Roger Moore
FRC #3135 (Robotic Colonels)
Team Role: College Student
 
Join Date: Dec 2012
Rookie Year: 2013
Location: Chicago
Posts: 2,310
orangemoore has a reputation beyond reputeorangemoore has a reputation beyond reputeorangemoore has a reputation beyond reputeorangemoore has a reputation beyond reputeorangemoore has a reputation beyond reputeorangemoore has a reputation beyond reputeorangemoore has a reputation beyond reputeorangemoore has a reputation beyond reputeorangemoore has a reputation beyond reputeorangemoore has a reputation beyond reputeorangemoore has a reputation beyond repute
Re: cRIO cutting out 5 s after collision

Quote:
Originally Posted by RufflesRidge View Post
Glad you got things working (if not entirely solved). As an aside for anyone else reading, the original configuration as described should not have passed inspection as it violates R56.
I am glad you pointed that out. I think that we may have our 2CAN wired wrong too. We haven't had any problems but it seems we may be able to prevent some.
__________________
Stronghold


Student 2013-2016
3135 -- Robotic Colonels
3507 -- Robotheosis (FTC)
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 19:27.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi