If the server responds slowly, the VPN connects only now and then, and a game or a video call stutters, the first thing to find out is on which part of the path packets get lost. The problem may be at your provider, on a backbone between countries, in the data centre or on the server itself, and that decides who can fix it. In this article we look at how traceroute works, take the three measurements needed for diagnostics and learn to read the result. We tested all commands on a test server with Debian 12, and the output examples come from real measurements.
How traceroute works
Data does not travel from your computer to the server directly, but through a chain of routers, each of which is called a hop. A traceroute shows this chain. It sends packets with a gradually growing limit on the number of hops, called TTL. A packet with a TTL of one "dies" at the first router, which reports this back to the sender, a packet with a TTL of two reaches the second, and so on. These replies make up the list of nodes with the latency to each of them.
The MTR tool and its Windows version WinMTR do the same, but continuously rather than once. They send packets to every hop again and again and count the loss percentage and latency spread for each. That is why diagnostics need MTR rather than a one-off tracert. A single traceroute shows the route at one moment, while MTR shows how each section behaves over a minute or two.
This method has a peculiarity that often leads to misreading the results. For a router, answering traceroute's service requests is a secondary task, and many routers limit such replies or do not answer at all, even though they pass normal traffic without loss. That is why loss on an intermediate hop does not by itself mean a problem, what matters is whether it continues to the very end of the route. Besides, the path from you to the server and the path from the server to you often go through different nodes, so only measurements in both directions give the full picture.
1. Check the speed of the server itself
First, let us make sure the server's own channel is fine. Connect to the server over SSH, for example via Termius, and run the test.
# Test of the server, its network and speed to several points around the world
wget -qO- bench.sh | bashThe test takes a few minutes. The script first shows the server specs and disk speed, and at the end a table with upload speed, download speed and latency to servers in different countries. Take a screenshot of the final table, there is no need to send the text output.
Run the tests one at a time
A speed test and a traceroute running at the same time interfere with each other. The speed test fills the channel, and the traceroute shows delays that do not really exist.
2. Run a traceroute from your computer to the server
Before measuring, turn off any VPN and proxy on your computer, otherwise you will check the route to the VPN server instead of your own. If the problem only shows up at certain times, for example in the evening, measure exactly then.
- Download WinMTR from the official website or from the GitHub repository and unpack the archive.
- Start WinMTR and enter the server IP address in the Host field.
- Click Start and wait until the Sent column reaches 200–300 packets. That takes 3–5 minutes.
- Click Stop and save the result with the Export TEXT button or take a screenshot of the window.
On macOS it is easiest to install mtr with Homebrew.
# Install mtr
brew install mtr
# Traceroute to the server, 200 packets, with network numbers (AS)
sudo mtr -rwzc 200 SERVER_IPIf you would rather not install Homebrew, the built-in traceroute SERVER_IP command will do, but it checks the route only once.
# Install mtr (Debian and Ubuntu)
sudo apt update && sudo apt install -y mtr-tiny
# Traceroute to the server, 200 packets, with network numbers (AS)
sudo mtr -rwzc 200 SERVER_IPOn AlmaLinux, Rocky Linux and CentOS Stream, mtr is installed with sudo dnf install -y mtr.
We use several mtr options. -r prints a finished report instead of a live screen, -w does not cut node names, -z shows the network number (AS) a node belongs to, and -c 200 sets the number of packets.
3. Run a reverse traceroute from the server
Now let us check the path in the other direction, from the server to your computer. For this you need your public IP address. With the VPN turned off, open a site like ifconfig.me and copy the address it shows.
# Install mtr on the server (Debian and Ubuntu)
apt update && apt install -y mtr-tiny
# Traceroute from the server to your computer
mtr -rwzc 200 YOUR_IPTake a screenshot or copy the whole report.
If the traceroute does not reach your IP
Many home routers and providers do not answer traceroute requests, and the last lines of the report will show ???. That is normal. In that case, run the traceroute to an address that definitely answers and is close to you, for example a public DNS server of your provider.
If you need a VPN port rather than ping
Regular mtr checks the path with ICMP service packets. Sometimes these get through while connections to a specific port, for example a VPN on port 443, are cut. In this case turn on TCP mode, and mtr will knock directly on the port you need.
# Traceroute in TCP mode to port 443
sudo mtr -T -P 443 -rwzc 100 SERVER_IPIf regular mtr reaches the server without loss, but in TCP mode loss starts at some node, someone along the way is restricting that particular port. For an IPv6 address, add the -6 option.
How to read the result
The mtr report has several columns for each hop.
- Loss% shows the share of packets the node did not answer.
- Snt shows how many packets were sent.
- Last, Avg, Best, Wrst show the latency in milliseconds, the last, average, best and worst.
- StDev shows the latency spread. The larger it is, the less stable the connection.
Here is an excerpt of a real report from our test server in Germany to Yandex's DNS server. We have hidden the provider's node names.
HOST: vm4096.hosted-by.f3cloud.com Loss% Snt Last Avg Best Wrst StDev
1. AS??? 100.64.0.1 0.0% 30 0.1 0.1 0.1 0.3 0.0
2. AS??? ??? 100.0 30 0.0 0.0 0.0 0.0 0.0
3. AS??? ??? 100.0 30 0.0 0.0 0.0 0.0 0.0
4. AS??? provider node 0.0% 30 6.1 6.1 6.0 6.3 0.1
5. AS??? provider node 0.0% 30 6.0 6.1 6.0 6.3 0.1
6. AS??? ??? 100.0 30 0.0 0.0 0.0 0.0 0.0
7. AS174 be9515.rcr31.fra06.atlas.cogentco.com 0.0% 30 6.5 6.6 6.4 7.0 0.1
8. AS174 be4517.rcr22.fra06.atlas.cogentco.com 0.0% 30 6.9 6.8 6.6 7.4 0.2
9. AS174 149.11.21.114 0.0% 30 6.4 6.4 6.2 6.7 0.1
10. AS??? ??? 100.0 30 0.0 0.0 0.0 0.0 0.0
11. AS??? ??? 100.0 30 0.0 0.0 0.0 0.0 0.0
12. AS13238 dns.yandex.ru (77.88.8.8) 0.0% 30 44.8 44.7 44.6 44.9 0.1At first glance there are five nodes with 100% loss, but this is exactly the case when everything is fine. These routers simply do not answer service requests, while all 30 of 30 packets reached the final point, and the latency there is stable. The latency growing from 6 to 44 ms between hop 9 and hop 12 is explained by distance, since the traffic leaves Frankfurt towards Russia here.
| What you see in the report | What it means |
|---|---|
| Loss on just one intermediate hop, 0% after it | the node does not answer requests, there is no problem |
| Loss starts at hop N and continues to the end | a real problem on the section starting from hop N |
| Latency jumps at one hop and stays there | usually a border between countries or a long backbone, if the growth is constant |
| A big spread between Best and Wrst on the last hops | an unstable connection, often congestion at peak hours |
| All packets are lost on the last hops and nothing reaches the server | the server is unreachable from your side, the IP address may be blocked along the way |
| Loss only from your side while the reverse traceroute is clean | look for the problem at your provider or in your home network |
Compare Wi-Fi and cable
If loss appears already at the first hop, it is your router or Wi-Fi. Repeat the measurement over a cable or from another network, for example mobile internet.
What to send to support
If you are not sure how to read the result, just send the measurements to us in a ticket, and we will sort it out. Include the following in your request.
- The server IP address, your city and your provider.
- The date and time when the problem is noticeable, with the time zone.
- A screenshot of the server speed test from step 1.
- The traceroute from your computer to the server from step 2.
- The reverse traceroute from the server from step 3.
- What exactly is wrong, for example "ping in a game jumps in the evening" or "the VPN drops on mobile internet".
If you have several networks, for example home internet and mobile, send measurements for each one separately and label which is which. This immediately shows whether the problem is tied to a particular provider.
Frequently asked questions
WinMTR shows 100% loss on one of the hops, is that bad?
Usually not. If there is no loss on the following hops and on the last one, that node simply does not answer service requests, while traffic passes through it normally. The warning sign is loss that starts at some hop and continues to the end of the route.
The mtr command is not found on the server
Install it with the command for your system. On Debian and Ubuntu it is apt install -y mtr-tiny, on AlmaLinux, Rocky Linux and CentOS Stream it is dnf install -y mtr.
How many packets should I send?
For a regular check, 100–200 packets are enough, that is 2–4 minutes. If the problem appears from time to time, increase the number to 500 or 1000 and run the measurement when it is noticeable.
The speed test shows good speed, but sites and the VPN are slow
The speed test on the server only checks the server's own channel. If it is fine, the problem is somewhere on the path between you and the server, and the traceroutes from steps 2 and 3 will show it. For a VPN, also run a traceroute in TCP mode to your VPN port.
Our resources
Telegram channel: https://t.me/f3cloud_news
F3 Cloud: https://f3cloud.com