The Planet Blog

A question that I often face – what IS a traceroute? Most individuals know that it represents the “hops” or routers along the path from a system’s IP to the destination IP entered. What’s most often misunderstood is the response time numbers printed for each hop. Some assume that it is the time any packet takes to make that leg of the journey. But, that just isn’t the case.

Those times actually measure the time elapsed from when the packet was sent to when a response was received. In today’s Internet, most routers have very strict limits on how many of these responses they can generate in any particular time period. So a lack of response isn’t indicative of the latency of that leg, also because the router may use a very different part of its “brain” to generate these control responses than it would to simply forward a normal packet the times may be misleading. This means that a router under modest load may respond with wildly different times, as its busy doing other “housework.”

So how do we know where the lag is coming from with a traceroute? Only end-to-end pings can really show latency or packet loss. But, certain patterns in a traceroute can help pinpoint a possible source.

One method of detection is a cliff-like increase in latency that builds from one hop forward. In other words, the traceroute suddenly has a steady jump in the return times of each hop from one spot to the next. Notice I didn’t say stars in the route. In today’s Internet landscape, stars don’t reflect the certain issue they once did. Certain providers have restrained the routers so much that they constantly throw stars.

If you see every hop after a line is throwing stars then that link may be losing packets. An end-to-end ping showing this loss is just about the only way to verify that for sure. As if all these rate limits weren’t enough to render our poor traceroutes meaningless, there’s another issue making it even more difficult.

The Internet often involves paths that get somewhere a different way than they get back. In other words the Internet is asymmetric. This asymmetry means that the packet you sent to the hop in the traceroute got to the router one way – through one set of providers, links, etc. – and the router’s response got back to you another way. This means a lack of response, the star, or latency could indicate an issue with either path. It also means that the cliff I spoke of earlier could mean that past that hop the return path has an issue not the actual route you see. Yes, traceroute can only show you the outbound path. This is a weakness in the technique it uses. Only the outbound path is visible to the tracing packets. This makes diagnosing or finding the offending hop difficult if it’s located in the return path.

So how do we find the issue? Anyone that’s opened a network-related ticket with The Planet knows we like to have traceroutes (to indicate the path), then pings of 100 or so, from the IP at The Planet seeing the problem to the IP on the Internet. And if possible, and we know this isn’t very easy, the same from the Internet side of the issue back to The Planet IP.

If we have this sort of information we can usually determine where the problem exists. Of course like taking your car to the shop many times these traces and pings don’t show the problem because it is intermittent. They are still useful and at least give us a baseline to work from when we’re looking into the issue. Intermittent issues can also be helped by reporting times of day the issue happens and/or if it is limited to certain IPs or servers.

So the next time someone tells you that a 380ms spike in hop 5 means that the router is overloaded, or that a star in line 10 shows that we’re losing packets, you might let them know that it’s never that simple in today’s Internet.

- Turbo

This entry was posted on Thursday, June 7th, 2007 at 8:00 am and is filed under Tech Stuff. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.