Today at the Suffield Shakedown we were completely unable to run because we were unable to establish communications. After having the FTA personnel (thank you to them) look at our 2 DLinks it was established that they were in working condition and configured properly. This was also confirmed by us resetting the system for another team number who agreed to try using our DLink to connect on their robot to test the bridge. However, we were unable to connect to the FMS when we used it.
To start the story at the beginning, about 1.5-2 weeks ago we were working fine using the DLink in bridge mode on our robot and continuing to use the Linksys router from 2 years ago to provide comms to our driverstation and our programming laptops. We had been waiting for a few days to get access to the robot again after further mechanical and electrical work. After what I can only refer to as an “event” that happened we were no longer able to get power to our cRIO and therefore (or possibly additionally) we were unable to communicate with it wirelessly over the DLink. The “event” in question has not been quantified per se, ie. we don’t know what happened… there was no snap or crackle, there was no smoke or smell… but the system stopped working. We are lucky enough to have a 2nd cRIO and replaced the “bad” system but still no communications… it appeared that perhaps the DLink had also been effected, so we swapped it with our old bridge from 2 years ago… and miraculously it worked. So we proceeded to use that configuration for programming and testing while ordering a replacement for our DLink. The DLink arrived on Thursday and we set it up on Friday… but the new DLink gave us the same puzzling results as the first one. We were able to see the DLink wirelessly but not the cRIO on the other side of it and we were also able to see the cRIO if we plugged into the DLink with our driverstation or laptop. However this did not give us a working competition configuration. Believing that it was likely a bridge configuration issue, we headed off to the scrimmage with our fingers crossed. However, as I already said… we did not find an answer to our problems.
Just for a little more background:
we are programming in C++ and updated to the v28 this morning to conform to that new requirement… however, after hearing of potential issues with v28 we also tried backing up to the v27 version cRIO image.
we verified the voltage into the DLink (recognizing the old bridge uses a different voltage… therefore they are using different power cords).
we verified the DLink is in working condition as mentioned above.
*]we have the latest cRIO and driverstation updates installed.
Has anyone else seen this type of problem? Any ideas?
I’m truly sorry for what didn’t happen for your robot yesterday. N.H. had similar problems & will be addressed prior to official competitions. At least you had this happen prior to a real Regional & can look for a solution. Your team is the definition of gracious professionalism. I never once heard a complaint & your team just kept trying. I hope you beat the odds & climb to the top.
Can you go through what does work on the dlink and when things don’t work, what are the errors or diagnostics.
For example, it sounds like the cRIO wired to the dlink and laptops wired to the dlink will be able to program and enable, correct?
Next, describe the configuration of the dlink wifi and the config of the laptop trying to communicate with it. Finally, can you ping the cRIO, the dlink?
When we tried using the DLink in bridge mode after replacing the cRIO (which had experienced “the event”) it was unable to communicate to the laptop or to the driverstation. Driverstation indicated no communication.
YES! well… to clarify, we have not done any programming, etc. directly plugged in because it was working using the old bridge so we have been using that configuration to continue our programming, testing, driving, etc. However, when we are directly plugged into the DLink we are able to both ping and use the netconsole to communicate the cRIO.
Using the laptop we tried using the netconsole display to see if we could talk to the cRIO with no response. We also tried pinging both the DLink (10.x.x.1) and the cRIO (10.x.x.2) from the command prompt on the laptop (going through our wireless router) and we were able to ping the DLink but not the cRIO. We also tried pinging the cRIO from our wireless router diagnostics with out success.
As far as configuration:
DLink: we followed the instructions on the “How to configure your Radio”; IP address= 10.2.30.1, mask= 255.0.0.0, gateway= 10.2.30.4. It was also tried as programmed from the competition kiosk with the WPA-key removed.
Laptop: is configured without any security on the wireless; IP address= 10.2.30.6, mask=255.0.0.0, gateway= 10.2.30.4
Router: IP address = 10.2.30.4, mask=255.0.0.0
driverstation: IP address = 10.2.30.5
Just to reiterate… we were able to run the system from end to end (ie. laptop and driverstation to router to bridge to cRIO) successfully using last year’s bridge. Also we have been trying to test using the 2 DLinks, one as an access point and one as a bridge…with the same results. Using this configuration we were able to get 2 laptops on each end of the setup (ie. laptop-accesspoint-bridge-laptop) to see eachother, but when one laptop was replaced with the cRIO the laptop could not see it. [Note that the removed laptop had been configured to have the same IP address as the cRIO, 10.2.30.2]
Any thoughts on whether upgrading the firmware on the DLink (from 1.2.1 to 1.3.1) might be helpful?
Greg, A little more information that might mean something to you…
We have tried re-imaging the cRIO with v28, then back to v27, then back to v28, then back to v25… based on information that other teams using C++ had been having issues with the newer cRIO images and communications. The FPGA versions (as displayed on the netconsole when the system starts up) are - Hardware and Software Version: 2011, Hardware and Software Revision: 1.5.3. [Note that these revisions don’t appear to change with the cRIO image… I presume that this means it is not part of the image, or that it has not changed between these images.]
We have also removed our code from the system (ie. undeployed or not loaded code after re-imaging)… to ensure that our code is not contributing to the problem.
The FPGA doesn’t always change, so the version number for it may well be the same for different updates. Similarly, the DS hasn’t been updated all season.
I’d suggest forgetting about the history of the updates and focus on a few fundamental things.
Unplug other dlinks and routers and bridges that may have the team IP or team SSID. The cool thing about wifi is no cables. The downside to wifi is, no cables, and it is very easy to have something next door or elsewhere in the room that is what you are really connecting to.
Is the dlink an AP? What is the SSID of the dlink? What are the frequency settings? Can the laptop see the network from the dlink? Can it connect using the Windows control panel or system bar?
Open the network control panel and check the IP of the wifi and wired NIC. This is the IPV4 settings. Click on the advanced tab and see if other IP addresses are also assigned to the NIC. If so, clear them out so there is only the .5 for LAN and .9 for wifi. Make sure wireless is turned on.
I know this is a lot of stuff to check, but those are some of the things I’d check if I were there.
Here is the situation. Let me lay out the network:
10.2.30.5 Drivers Station
10.2.30.4 DLink DAP-1522 in AP Mode
10.2.30.1 DLink DAP-1522 in Bridge Mode
10.2.30.2 cRio
.5 cabled to .4 can ping .4 & .1 not .2
.5 cabled to .1 can ping .1, .2, & .4
If .4 is removed and .1 is set to AP Mode .5 can ping .1 & .2
Bridge mode is very problematic. Since we can get a network path from the drivers station to the bridge we are suspicious that the problem is with the cRio. However changing some of the wireless settings such as 2.4 to 5 ghz will cause communication problems as well.
We are trying to simulate a FRC competition which is why the AP along with the bridge on the robot is in the mix. We had absolutely no comms at Suffield no matter what we did.
Everything was working on our previous cRio which stopped working, no power to it, not sure why andwe need to talk to NI about it including using the Linksys AP from last year. We eliminated the Linksys since I was have trouble getting the to the DLink bridge using two different Linksys APs. The DLink to Dlink seems to work better.
We have reloaded the cRio with several different versions of firm ware from 25 to 28. We have not recycled power on the cRio by itself while leaving the rest of the robot active. We are trying a new cRio now borrowed from another team.
This is getting interesting. We re-imaged the new cRio to our team ip and plugged the network in DS to AP to BR to cRio. Initially we were only able get from theDS to the BR. We hit the reset button on the cRio and had sucess reaching it. So we wired up the original cRio. No change. DS to AP to BR to cRio NG. DS wired to the bridge we are able to ping the cRio, the BR and the AP. Boggles the mind. Bear in mindthis cRio was used in competition last year although it may be 3 years old, and if we use the old Linksys bridge we have no issues reaching the cRio. Encryption is disabled.
One more tidbit of information. Communication to the new cRio will not occur unless the reset button is pressed. A simple power on/off doesnt do the trick.
Okay, we are pretty sure we know your problem. However, don’t thank me, thank Joe Hershberger (I swear, it’s uncanny how fast he processes stuff like this).
Anyway, MAC Address Cloning is probably turned on in your DAP-1522. You’ve got to turn that off. MAC Address Cloning requires ARP traffic in order to get a MAC address to clone, and since the cRIO boots up first it does its gratuitous ARP way before the DAP is done booting - so, the DAP never sees anything to “clone” until you press the reset button on the cRIO - at that point the cRIO boots up, gives a gratuitous ARP, and the DAP clones that MAC Address and starts communicating.
Thanks for that idea… but… we verified that the MAC address cloning is turned off… several times.
Although it does sound remarkably like what we are seeing… so perhaps there is an issue with what it displays for the settings as opposed to what it is actually set to… if so then all bets are off.
I recreated one instance where the cRIO and DLink failed to communicate properly and, in addition to the cRIO reset, I was able to simply unplug and replug in the cRIO Ethernet connection to get them talking again.
The symptoms in this particular case were no connectivity between the DLink and cRIO, but the cRIO Ethernet port lights were solid green and yellow, i.e., no packet traffic.
Unplugging and re-connecting to the cRIO port 1 started the yellow traffic light blinking and everything was fine again.
The few times I experimented with cycling power just on the DLink didn’t cause the problem, however, I only tried that twice.
I was sprinting through a test series while the mechanical and electrical guys were looking the other way…
Our DLink shipped three hours before you asked your question, so I’m not able to revisit it.
There is good news to report. We RMAed a cRIO that had a power issue with National Instruments. The new cRIO that arrived has zero communications issues. The robot connects flawlessly through the Dlink bridge to the previous years Linksys AP to our driver station and programming laptop.
NI has also agreed to RMA the cRIO that still had the comms problem. This unit is an older one by the serial number so I am think that the problem lies someware in the low level firmware that prevents the ip address to ARP properly. This may also explain some of the other issues that have been posted where a cRIO reset is required after the robot powers up in order to establish communication.
We should be good to go at our first regional with a running robot, not just a pretty lawn ornament like we were at the Suffield Shakedown.
