![]() |
ping and tracert networking question
Need some help from network gurus. My internet access at home is via WISP. When it is working properly, I can ping and tracert the tower with very little latency. Occasionally it will stop working, and I've discovered that when that happens I can ping the tower but tracert does not work (or takes an inordinately long time). What would be some possible explanations that would fit that set of facts? |
Re: ping and tracert networking question
This happens on my home network as well. The situation isn't exactly the same, since I don't have a WISP. Instead, I have sort of a LAN within a LAN that I use for testing purposes(I work as a researcher for a distributed systems and security lab). For me, the problem is old wifi hardware that is probably nearing the end of its lifespan, which I am also too lazy to fix until it dies completely :D
Can you give more information on what hardware you are using? That could be the problem. Also, do you know anyone else who uses the same WISP who may be having similar issues? If that is the case, it's probably an issue on your provider's end, in which case you should contact them and see if they can isolate the issue. |
Re: ping and tracert networking question
Ubquiti AirOS PowerStation2 |
Re: ping and tracert networking question
Quote:
|
Re: ping and tracert networking question
Quote:
|
Re: ping and tracert networking question
Quote:
How frequently does this occur? Multiple times per day? Per week? Also, I assume you've taken a look at your PowerStation during these times to make sure nothing was obviously out of place? |
Re: ping and tracert networking question
How many hops in your traceroute when it is working?
Are you going to a DNS entry or an IP? |
Re: ping and tracert networking question
Quote:
Actually, it has a Murphy sensor. Its happens at the worst possible times. Like when I'm about to submit my tax return, or in the middle of a purchase or a bank transaction. Quote:
|
Re: ping and tracert networking question
Quote:
|
Re: ping and tracert networking question
Quote:
|
Re: ping and tracert networking question
It could be DNS resolution. Most implementations of tracert will attempt to resolve each IP along the way, and this can take a long time with a flaky DNS server.
A couple suggestions: a) Try running tracert with the -d flag (Windows) or -n (Linux). b) Try using an open DNS service, like OpenDNS (208.67.222.222/208.67.222.220) or Google Public DNS (8.8.8.8/8.8.4.4). |
Re: ping and tracert networking question
Quote:
Quote:
|
Re: ping and tracert networking question
Quote:
Quote:
|
Re: ping and tracert networking question
Quote:
|
Re: ping and tracert networking question
Quote:
|
Re: ping and tracert networking question
Quote:
So if the DNS doesn't turn out to be the issue you probably have lost your connection to them over that link. If you have the information to access the equipment on your side you can probably query the status of the connectivity to the tower from the web interface. I think that unit supports SNMP as well so you could query the status of the link from that. They appear to have an MIB for that purpose available. Then you could work it out such that you can monitor the signal level and data transmission rates. |
Re: ping and tracert networking question
Just out of curiosity, what addresses are you pinging and what addresses are you tracerting too?
|
Re: ping and tracert networking question
This question leads to an iceberg answer, in that there's a lot to say. Here's the tip, so to speak:
> I can ping the tower but tracert does not work (or takes an inordinately long time). The short answer is that you are most likely not pinging the tower, but something more local. The DNS servers are not likely to be local either, so DNS name resolution isn't likely to work either when you are having connectivity issues. The analogy here is that to reach an endpoint (specified by IP address, possibly after the additional step of DNS name resolution from a textual name to an IP address that is an independent network operation that itself requires a healthy network connection), you have to traverse a network of roads. You can think of each intersection on this network of roads as having its own IP address. To get almost anywhere, the first few intersections are going to be the same, they take you from your home, out of your neighborhood, and onto a major road. Once you're on this road, chances are you are going to get where you are going, unless there is a problem toward the other end of the journey, when you are getting back onto local roads in the destination neighborhood. Or, if there's no one home at the address on the other end (the site or server is out but you can get to everything else). There is a span of road that is sometimes out in your case, most likely the over-the-air link. If this happens, you can reach intersections or even end addresses that are local and do not require traversing the span where there's an outage, but can't get further. Take a tracert when things are working, and let the IP addresses be resolved back into textual names (don't turn of DNS resolution when tracert for a working connection). These names may give you clues. Keep the list of names and IP addresses to some well-known site handy for comparison when things are out. You will likely find that when you run a tracert that stops at some point, you can ping the addresses that it could reach, you just can't ping ones that are further away on the network. The first place you can reach when things are working but cannot when they are out is the far end of the link that is giving you problems. If you want, you can post details here and we can comment further. Hope this helps! |
Re: ping and tracert networking question
Quote:
3 hops: PC -> router -> radio -> tower. BTW, I have ruled out the router. If I bypass the router and connect the PC directly to the radio when the problem occurs, it does not fix the problem. I do not have access to the radio, other than to cold-boot it by cycling the power (PoE), which I have tried, to no avail. When the problem occurs, I can successfully ping the tower, but tracert to it times out (or takes an inordinately long time to complete). This setup had been working almost flawlessly for 3 years. Problems started occurring occasionally about a year ago, and have become more frequent and longer in duration. |
Re: ping and tracert networking question
Quote:
Quote:
Another thing you might try is opening a browser and typing the IP address you have for the tower and see if you can get to an admin interface. You should certainly be able to do this with the IP address of the router, to see how this works. Apologies if you've already tried this -- it is hard to find the right level of detail without being insulting and/or writing a whole lot here. Do you have a list of names that you obtained when things were working (tracerout output but not using the option to suppress DNS translation)? It might be interesting to match this against the list of hops you supplied above and knowing just where things stop when they are not working would be very helpful. It could be further into the network on the other side of the radio link, but I'd suspect this link until I knew for sure it was good. Are things worse when it is raining, at certain times of day, or any other pattern you've noticed? Sorry for so many questions and so little actual help! BTW, if you can get to the admin interface for the radio, you should check to be sure your firmware is current. Same goes for the router, but this isn't likely to be as important in this case. |
Re: ping and tracert networking question
This may be useful:
Step 1 - Login to the web management Open the web browser and type the default IP address of the AirOS powered device http://192.168.1.20 into the browser address field. You will be prompted to enter the default administrator login credentials: User Name: ubnt Password: ubnt After successful administrator log on you will see the Main page of the web management interface. There should be a way to see the signal strength -- if you had access to the antenna, this could help to make sure it is aligned optimally. One final thing, it can be confusing to tell when tracert is first trying to ping an address vs. when it has done so and gotten a response, or has timed out. This is a good reason to use ping in isolation as a double check. |
Re: ping and tracert networking question
I have neither physical nor network access to the AirOS PS2 radio on my roof, other than to cold-boot it by cycling the power (PoE), which I have tried, to no avail. On a moonless night I can sort of see the signal strength LEDs with a pair of binoculars. I have not yet been able to observe them during an outage event. Outage events are random and not correlated with weather conditions. PC---(ethernet)--->router---(ethernet)--->radio---(wireless~1mi)--->local_tower---(microwave_beam~2mi)--->main_tower--->fiber_optic_backbone ping/tracert script log when working correctly: Code:
The current date is: Wed 11/20/2013ping/tracert script log when not working correctly: Code:
The current date is: Sun 11/17/2013I am planning to discuss the problem with my provider once I become a bit more knowledgeable so I know what questions to ask and what data to collect. PS - When internet is working, if I type 192.168.1.20 into my browser's address bar, it does not connect with anything - it just times out. If I ping it, it times out. If I type 10.0.0.1 into my browser's address bar, I get an AirOS login splash screen, but I don't know if that's for my radio or the local tower or ... |
Re: ping and tracert networking question
192.168.1.1 is your router -- you can get an admin screen, check firmware version, etc. by pointing a browser to this address, but this is not the problem and so there's no need to do anything with this. If you log in to the router, you should be able to see the "default gateway" IP address for the next hop, which would be handy to have -- though you probably already have it from the tracert output.
I'd guess 10.0.0.1 is your radio -- you could try logging in and looking around to confirm, unless your provider supplied the radio and has changed the default info. I found this utility you could run that would help to confirm this in any event: <http://www.ubnt.com/support/downloads/utilities>. Is a.b.c.d in your output 10.0.0.1, or 10.b.c.d? Anything that starts with 192.168. or 10. is not exactly a real IP address, in that these are used all over the place. These addresses are private only to each of a great many different segments of the internet, around the edges. For example, 192.168.1.1 is used probably millions of places, so there's no issue with pasting numbers in these ranges here. The tracert outputs are interesting. Do you see the 60 second delay even when you use the option to suppress DNS name translation? If not, this all makes sense. You really want to do a tracert to something like chiefdelphi.com when things are working and note the difference when things are not working. The idea is to look one hop past where things get when you experience an outage. I'd have to investigate further, but your radio could have a sort of bridge mode. However, I think it is probably using a 10. address on the over-the-air side. It suspect that it has an address that starts with 192.168. as well. The utility above should help you find this, and hopefully you can log in via this address to get some more info. |
Re: ping and tracert networking question
192.168.1.1 is the LAN side of my router. See the annotation in the tracert logs I included in my previous post. I have full physical and network access to the router. I do my own router setup. That's how I was able to change the DNS server as mentioned in an earlier post. The default gateway at the PC is 192.168.1.1 (my router). The router is set up to get an internet gateway IP dynamically. When it does so, it always gets 10.0.0.1 (the a.b.c.d in the tracert logs). As mentioned earlier, when the problem occurs it does not go away when I bypass the router and connect directly to the radio. So the router is not the problem. As mentioned in previous posts, I do not have any access to the radio. 10.0.0.1 gives me an AirOS login splash screen, but I do not have password access, so I cannot log in, and I can't even tell for sure if it is the AirOS radio on my roof, or an AirOS product somewhere downstream (like perhaps the local tower). As mentioned in a previous post, I will try the "-d" tracert option next time the problem occurs. That has not happened yet. When things are not working, there is a 60-second tracert delay even when doing tracert 10.0.0.1 (2 hops). As mentioned in the previous post, I do not know for sure if that means the problem is at the radio, or if the radio is acting as a "bridge" so that its IP does not appear in the tracert (is this what would happen with radio in bridge mode?). The AirOS PS2 supports bridge mode. I do not know (and have no way to determine) if it is setup for that mode though. I will try the Ubiquiti "Discovery Tool" and see what happens and report back here. |
Re: ping and tracert networking question
I suspect the tracert delay for 10.0.0.1 is simply trying to reverse lookup 10.0.0.1 (and other addresses along the way) to a DNS text name -- the servers to do this are on the other side of the outage and so the requests/retries to do this are timing out. The actual ping times being reported are much lower, sub-second.
Something like "tracert -d 198.101.239.6" (chiefdelphi.com) would be more interesting, since this would show the hop that it going out; when you run it when things are not out and when they are, you'll have some interesting data I think. The whole IP address thing is at what is called "layer 3" in a stack of protocols. Bridging is "layer 2" but an outage in a layer 2 link will take out a (non-redundant) layer 3 link, so the first step is to work out where you lose layer 3 connectivity, then possibly there's a next step of diving into anything more complicated than a point-to-point link at layer 2. |
Re: ping and tracert networking question
Does anyone know: if the radio on my roof is in "bridge" mode, will I or will I not see an IP address for it when I do a tracert ? |
Re: ping and tracert networking question
A bridge still has a unique addressable IP and will show up as a node in a trace route.
|
Re: ping and tracert networking question
Quote:
Based on that, would it be correct to conclude that the IP address on the ethernet side of my radio (the side my router is connected to) must be 10.0.0.1, since that is the very next tracert hop after 192.168.1.1, which I know is my router ? Would it also be correct to conclude that the radio is likely the problem, and not DNS, since I get the abnormal delay when doing only a 2-hop tracert (pc-->router-->radio), and 10.0.0.1 would not be sent to a DNS server for reverse DNS lookup? |
Re: ping and tracert networking question
From your descriptions, I'd certainly suspect the radio as the weak link.
Your WISP should be able to check the radio connectivity and performance statistics, but when in doubt they usually just replace the unit. |
Re: ping and tracert networking question
Could be the radio, could just be marginal alignment -- the antenna is highly directional. Regardless, if it isn't yours, you want to have the provider take a look. I highly doubt any provider would let you get access to the admin UI for a tower or any other part of their network, so the fact that you can bring up a login screen is another strong indication that you are seeing the rooftop radio at 10.0.0.1.
The whole DNS thing basically boils down to the fact that, unless you run your own private DNS server on your personal LAN, any DNS operation has to contact a server on the Internet (read on the other side of the air hops). So, when things are not well, this: 1) isn't going to work; 2) is going to incur a large delay as the system waits for the response that is never coming. The radio could be doing a number of things: routing, bridging, NAT, etc. The main point here is things are going wrong inside your provider's network. If they let you access the admin UI for the radio, you could probably tell a whole lot more. If there's an option to use your own radio, it might not be the worst way to go. If someone comes out to service your radio, maybe they will leave you with the login info (doubtful, but one never knows). They should be able to check the signal strength from their network and may know your signal isn't great or be able to check when you are on the phone. If any of this doesn't make sense, let me know... I assume you tried ubnt/ubnt to log into the radio? Did the utility tell you anything? Here's something on layer 2 (bridge) tracert: <http://www.fragmentationneeded.net/2...raceroute.html>. A bridge will typically have a layer 3 IP address, but it doesn't follow that it is going to show up in tracert. If you are using Windows, try the pathping command -- it is more informative than tracert and possibly somewhat less confusing. |
Re: ping and tracert networking question
1 Attachment(s)
Quote:
Quote:
Quote:
Are you quite sure of that? It seems pointless to do that. Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
Code:
C:\>pathping 10.0.0.1 |
Re: ping and tracert networking question
Quote:
Quote:
The idea with pathping is to get statistics taken over a larger sample size, you might find there are drops even when things seem to be working fine -- but you need to use an address that is further out than 10.0.0.1, which is why I have been using 'chiefdelphi.com'. If it is easy for you to do, running the util without the router in the middle could make a difference. Some packets are not forwarded across routers and the util likely uses such poackets to discover radios. Thanks for all of the info thus far!!! |
Re: ping and tracert networking question
Quote:
|
Re: ping and tracert networking question
Quote:
|
Re: ping and tracert networking question
Quote:
Quote:
|
Re: ping and tracert networking question
I'm with others who are hypothesizing that you're having link level problems. DNS issues in ISPs get fixed very quickly, as working DNS is required for nearly everything else that ordinary people do on an internet connection.
|
Re: ping and tracert networking question
Quote:
Quote:
|
Re: ping and tracert networking question
Quote:
So it could be both. A good way to notice would be to run a ping or performance test against a relatively stable local site and see if there are periods where it takes much longer to respond. It's probable something in the link is retrying to make the delivery in that case. I have used *a lot* of Hughes Networking satellite gear and DNS transactions were a major pain because the small transactions would easily waste a ton of time clearing the satellite uplink and back (2-4 seconds of lag to complete a DNS request over bi-directional satellite is not unusual). This sort of thing can make a big headache if the pages you most frequently visit contain absolute encoded hyperlinks with the domain listed. Sometimes the DNS will keep being looked up and avoid the requester cache...especially if they've set the time to live low. Also the DNS packets are usually small and usually satellite TCP/IP systems send a big block to give the illusion of broadband...till you only need a tiny DNS request...then the rest of the block is wasted because your need to resolve the DNS first. It was often annoying enough I'd setup a local caching DNS server and use a 56k modem to feed it (which sounds painfully slow till you realize that the modem's latency is magnitudes less). It's true that Ether's radio connection should have much lower latency. However the path between the radios in the link is probably open to interference of many kinds. Any of them might be occasionally impacting the link performance causing headaches. I've set up hundreds of long distance wireless links and it is not an uncommon issue. Ether is there any chance the service provider will let you use SNMP to their radio to get the statistics? The advantage to them is that they don't need to give you the administration password for the web / command interface. |
Re: ping and tracert networking question
Guys: My internet connection has been working flawlessly for the past 11 days (ever since I changed the DNS server setting in my router as suggested by Jared). So that seems to be a pretty strong indicator that the problem I was experiencing is not the link between my radio and the local tower. Would a WireShark capture of several tracert or WGet transactions shed any light? I did notice something weird though. In the router's "Connection Status" window it always shows the "Lease Expires" data/time equal to the "Lease Obtained" date/time. It was not like this in the past. Is this normal? Lease Obtained Mon Dec 02 19:24:21 2013 Lease Expires Mon Dec 02 19:24:21 2013 |
Re: ping and tracert networking question
Quote:
The point I was making is that wireless has a habit of concealing subtle issues that might be temporary or recurring. TCP/IP is a pretty reliable protocol it'll generally cover for quite a bit before a total link failure. On a long distance link you'll quite commonly loose link quality and never really notice it because the protocol will retry the delivery. A prime example is a highly directional link I inherited from someone. It would only fail on windy days in the summer because there were trees that grew high enough that when the wind caught them and they were full of leaves it made strange interference. The link would rarely go completely down. People would start experiencing performance issues on large downloads and partially loaded pages (most of the page would load but a few large images would fail). Quote:
Have you tried to power the computer off and on again (props to the "IT Crowd")? In the big picture, on the other hand, if it's working for now it might be better to leave it be. Plug and pray :) |
Re: ping and tracert networking question
Quote:
My internet was flawless for the first three years (see post #19), and I still have the exact same line-of-sight access to the local tower (no new structures erected in the vicinity). Quote:
|
Re: ping and tracert networking question
Quote:
For example: when motherboard capacitor start to fail the computer may start exhibiting random instability. That very same computer probably worked for years before the issues presented. Unfortunately the state of 'working' in wireless IP links comes in many shades. It is even possible that over time you've inherited a little mechanical slippage and the antenna pointing is slightly off. Anyway it is really neither here nor there, if you are happy with the performance what's the sense of worrying about it? Just as a matter of sanity when I start to have issues like this with links I try to put myself into a position where I can see clearly if there are link level issues so usually I deal with that when the urgency is low. Quote:
If you go to a command prompt and do an: ipconfig /all Does that also report the same obtain and release timestamp for that network interface? It should because those values are stored in Unix timestamps in the registry. I looked at 3 different Windows systems: XP, 7, 8 All pointing at DHCP servers with infinite leases. The obtained time reports from the last update. The renew time reports when the lease expires. However the IP handed back to that MAC address is the same. The servers are Windows 2003 and Windows 2008. There is a single exception I could find: Microsoft Remote Access Service (RAS) will clone the timestamps because it hides from the client that the assignment is DHCP and it's handing it off from a pool at startup. In that case it should be dated: Jan 1st, 1980 This happens because RAS hands back an incomplete DHCP handshake and if you look at the DHCP server side the actual time stamps are there. In all likelihood your equipment is NATing your public IP and some piece of equipment is handing out DHCP internally to your side of the connection (which facilitates the NAT process). Perhaps there's something a little off with how the DHCP has been implemented. However again if it's working you may as well leave it alone till it doesn't but so far as I can see generally what you see there is not normal. If you are game: Remove your router from the connection and use the computer directly connected. Reboot your computer and see what your DHCP reports. Might just be the result of your router and nothing else. |
Re: ping and tracert networking question
Quote:
Quote:
Quote:
Code:
Ethernet adapter Local Area Connection:Quote:
|
Re: ping and tracert networking question
Quote:
PC connected directly to radio: Code:
With PC connected directly to radio:PC connected to radio through router: Code:
With PC connected radio through router:Router Connection Status: Code:
Connection Status |
Re: ping and tracert networking question
Quote:
It looks like your router is either botching the timestamp decoding, simply replicating the information or just plain has a bug. It shouldn't be reporting different information from your computer even if the MAC address of the network interface is different. I looked real quick to see if you list the make and model of your router somewhere and I did not see it. If you provide that and I have access to one I'll test it for you. |
Re: ping and tracert networking question
Your router has a DHCP server -- it acts as a DHCP client wrt the network and as a DHCP server to hand out local addresses. When you remove the router, your computer has a different DHCP server (the one on the network, not the one on the router). Welcome to the convoluted ways of networking, tip of the iceberg thing again!
I'd like to see the packet statistics from a pathping run, or at least know that you've looked at these and not seen anything amiss. Again, you want to pick an endpoint for this testing that is very far away (like chiefdelphi.com / google.com / etc.). Do you remember what DNS servers you were using before the trouble? You might want to try tracert and/or pingpath to these IP addresses... Very glad to hear that things have improved for you!!! |
Re: ping and tracert networking question
Quote:
|
Re: ping and tracert networking question
Ether,
It looks like your router is doing NAT. Here are two suggestions: 1 - Turn off NAT in the router and let your radio assign the IPs to local machines, or replace the router with a switch. 2 - Update the software in the router. -Hugh |
Re: ping and tracert networking question
Quote:
When the computer is connected the lease does expire. It could be they assigned an IP to his router's MAC but that seems unlikely. Even if they did that it should renew as usual to the same IP. Ether's information on page 3 shows that the ISP renews the DHCP every hour. Ether's router is providing NAT and DHCP and is renewing his internal DHCP every day. When his router times out it should renew every hour on the public interface. When his computer times out it should renew every day on the router's private interface. The question is: why does his router indicate that this obtained and renew timestamp are the same? His computer when connected in place of the router does not agree with that and shows 1 hour intervals. Updating the router firmware might correct any bugs the manufacturer found including those impacting DHCP. |
Re: ping and tracert networking question
Your router is almost certainly doing NAT. This makes multiple computers on your local LAN look like a single computer from the perspective of the network. You probably do not want to turn this off -- but when you directly connect the PC to the radio, you effectively do this. What is happening with DHCP is somewhat related. You need a single IP address on the network side, and your router gets this by having an internal DHCP client. The DHCP server on the network hands out the lease there, and whatever value the server choses is what the router gets. It automatically renews the lease, often at the halfway point in its lifetime. The router is getting the same sort of leases when it is connected as when you directly connect the PC to the network, but the displayed info may be rounded or some similar issue. It doesn't seem that you are seeing DHCP lease issues of any sort, although this is an interesting theory it doesn't fit the facts, particularly that things got better when you changed your DNS server config info. BTW, did you do this at the router?
Each computer on your LAN (including wireless) needs an IP address, and each has a built-in DHCP client. The router has a DHCP server, and you can tweak the configuration to hand out addresses in a different range, or to limit the number of addresses it will hand out. The router controls the DHCP lease lifetime for these leases. |
Re: ping and tracert networking question
1 Attachment(s)
Quote:
Quote:
Quote:
Quote:
2) I just now checked it again. I apparently have the latest version. Quote:
|
Re: ping and tracert networking question
I assume this <http://wiki.ubnt.com/AirOS_3.4> manual applies to your device?
Since you can't log in, I'm not sure how you can tell for sure. Your ISP may have updated the firmware to something newer, if this device supports it. However, the answer most likely does not change -- yes, this device can act as a layer 2 bridge. For reference, here is the info for your router: <http://documentation.netgear.com/wpn...FullManual.pdf>. Quote:
The data from pingpath should make it possible to work out where along the way the over-the-air hops are happening. I'm hoping to find some hops where there are some drops as one clue. Another clue is just the IP address info and where in the list of hops each address appears. At any rate, this is some hard data that would be very helpful to have. |
Re: ping and tracert networking question
Using SNMP to get some info from your radio isn't a bad idea, but it depends on this being enabled at the radio and having a default (or known) security setup. One tool for looking at SNMP info is here: <http://en.wikipedia.org/wiki/Net-SNMP>. Basically, not having info from the radio opens a major gap in what we can observe. FWIW, it is pretty common to have read-only SNMP access on by default, for purposes of network management. You would use the IP address where you saw the AirOS splash screen with the SNMP tool to obtain some useful info, hopefully.
|
Re: ping and tracert networking question
Quote:
|
| All times are GMT -5. The time now is 23:29. |
Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi