This site is now 100% read-only, and retired.

Dealing with lossy links?

Posted by Anonymous on Tue 12 Dec 2006 at 09:35

Sometimes you have to deal with lossy IP connections. Your ISP has packet loss somewhere, or a cable in your network is rotten, or a switch is soaked with traffic.

While most of the time it's best practice to tackle the source of the problem, sometimes you just have to get tough.

Possibly because you're the admin trying to fix things, or because you have life-saving important data that you must transmit at the expense of your neighbour's bittorrent practices.

I'm looking for a system to minimise the effects of this.

Currently I'm thinking about a VPN over UDP (nicely transparant for all IP applications) that does redundant transmits, eg each packet twice or three times depending on the loss.

A Linux client is of course mandatory, and a Debian vpn terminator (set up beforehand) wouldn't hurt.

Any ideas?

 

 


Re: Dealing with lossy links?
Posted by Anonymous (84.20.xx.xx) on Tue 12 Dec 2006 at 20:26
Yes. I am experiencing such situations a lot and till now I only partially decreased the effect of a lossy line with lowering the TCP retransmision interval. And configuring all the programs to allow very long timeouts (for TCP connections).
Anyway, a efficient solution (but not so clean) would be a kernel patch that changes the way things are sent into the network or a virtual network device bound to a real one. I was thinking of an algorithm that would regulary check the connection quality (pinging and measuring lost packets and heavy delayed ones) and depending on this force resending of various packets that go out of your network device (for TCP this shouldn't break everything, just unneeded traffic; but this could kill other protocols). So in this way you wouldn't need to set up any VPN or anything - just resend everything enough times.
Hm, yeah, a modification of the TCP behavour would be simpler (removing the factor 2 increase of delays between resends).

[ Parent ]

Re: Dealing with lossy links?
Posted by Anonymous (59.178.xx.xx) on Wed 13 Dec 2006 at 07:26
I assume you're tweaking things in /proc/sys/net/ipv4/ and have gained some wisdom from the experience.

What do you try, for what kind of situation?

curious,
PJ

[ Parent ]

Re: Dealing with lossy links?
Posted by simonw (84.45.xx.xx) on Tue 12 Dec 2006 at 20:37
[ View Weblogs ]
Just get it fixed.

If you ISP has noticeable packet loss, change ISP, if you are sure it isn't your own hardware.

If you have to start tweaking TCPs error handling your connection is so close to toast you might as well give up on the link.

One might occasionally get episodes of packet loss when a specific router, or connection is overloaded, but this is typically only something that happens rarely, if you see it routinely your ISP is cutting costs beyond what is acceptable, or incompetent.

[ Parent ]

Re: Dealing with lossy links?
Posted by Anonymous (193.251.xx.xx) on Tue 12 Dec 2006 at 20:55
... or you live in a developing country where the only link is a satellite link that is neither fast, nor stable, and most of the time overloaded. But then, sending everything twice or thrice is no good solution either.

For that special sitaution I was toying with the idea of using a rented server in a "safe, internet-heaven" place, like Europe or so. Then making a link between here (Tchad) and that server, putting some good error-correction/ packet recovery in. But well, Tcp did it well enough till now.

Greets

ineiti

[ Parent ]

Re: Dealing with lossy links?
Posted by Anonymous (59.178.xx.xx) on Wed 13 Dec 2006 at 07:45
Simon says:
If your ISP has noticeable packet loss, change ISP, if you are sure it isn't your own hardware.
Once upon a time I had a pretty rotten "ISP" that had a grossly overloaded gateway. It didn't lose packets, but slowed down ridiculously, presumably due to retransmit attrition. Ping times of, say, 4s (That's right, 4000ms 8-/ )

I wrote up an article about the analysis I did at that time. Readers of this thread may find it interesting.

I ditched that ISP as soon as I could.

PJ

[ Parent ]

Re: Dealing with lossy links?
Posted by Anonymous (149.157.xx.xx) on Wed 13 Dec 2006 at 14:28
Hi,

Assuming you're using TCP and you have control over the TCP sender, you might try out a few of the other congestion control algorithms.

Most of the current schemes are loss-based, ie they assume that packet loss indicates congestion and they therefore back off when a packet is lost. Reno (the current standard algorithm) increases the congestion window (the amount of data you may have unacknowledged) by 1 packet per round trip time. If any packet gets lost, it presumes the network is congested and halves the congestion window to react.

Obviously therefore, Reno backs off too often on lossy links -- as do all loss-based congestion control algorithms. However, many of the others back off by less and increase back up again more quickly. I would consider trying out h-tcp, bic and cubic. There are some that are not loss-based (eg Vegas) but they have other problems. Your kernel may not have the "advanced congestion control" enabled, so you might have to recompile and compile the modules. Once you compile that option and compile the module for each congestion control algorithm you can just do:

modprobe tcp_htcp
modprobe tcp_bic
modprobe tcp_cubic

Note that this is not a client setting, it affects the data *sender* so it's generally useful on servers. If you need to improve web access, you could change the congestion control on a proxy server upstream of the lossy link and have the people access through it.

http://www.csc.ncsu.edu/faculty/rhee/export/bitcp/
http://www.hamilton.ie/net/htcp/

The latest linux kernels (>2.6.19) uses Cubic by default.

Gavin

[ Parent ]

Re: Dealing with lossy links?
Posted by Anonymous (201.42.xx.xx) on Sat 16 Dec 2006 at 00:15
If you modprobe more than one module, which algorithm will be used?

[ Parent ]

Re: Dealing with lossy links?
Posted by gavinmc (149.157.xx.xx) on Mon 18 Dec 2006 at 09:46
The current algorithm is dictated by the string in:

# cat /proc/sys/net/ipv4/tcp_congestion_control
reno

When you load the module, it will automatically change the entry in the above file, setting itself as default. One they are loaded, you can switch between them yourself with commands like:

# echo htcp > /proc/sys/net/ipv4/tcp_congestion_control
# echo cubic > /proc/sys/net/ipv4/tcp_congestion_control

I believe it is also possible to choose the algorithm per application using a setsockopt(), though I've never done it.

http://www.uwsg.iu.edu/hypermail/linux/net/0509.2/0040.html
http://www.uwsg.iu.edu/hypermail/linux/net/0509.2/0037.html

Gavin

[ Parent ]