Network emulator

Discussion of testing theory and practice, including methodologies (such as TDD, BDD, DDD, Agile, XP) and software - anything to do with testing goes here. (Formerly "The Testing Side of Development")

Moderator: General Moderators

User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Network emulator

Post by VladSun »

Linux only! (of course)
It will emulate network delay, network packet loss and network bandwidth limits.
Edit the values in the EDIT section.

Code: Select all

#!/bin/bash

#-EDIT BEGIN-----------------------------------------------------------------------------------------#


DELAY=100                       #[ms]
DELAY_VARIATON=10               #[ms]
DELAY_CORRELATION=10            #[%]

PACKET_LOSS=0.5                 #[%]
PACKET_LOSS_CORRELATION=        #[%]

TRAFFIC_RATE=300                #[kbit]

DEV="lo"                        #[ lo | ethX | wlanX | etc. ]
IP="127.0.0.1"

#-EDIT END-------------------------------------------------------------------------------------------#

TC=`which tc`

$TC qdisc del dev $DEV root 2>/dev/null

if [ "$1" = "stop" ];
then
        exit 0
fi

$TC qdisc add dev $DEV handle 1: root htb r2q 1700
$TC class add dev $DEV parent 1:1 classid 1:20 htb rate ${TRAFFIC_RATE}kbit
$TC qdisc add dev $DEV parent 1:20 handle 12: netem delay ${DELAY}ms ${DELAY_VARIATON}ms ${DELAY_CORRELATION}% loss ${PACKET_LOSS} ${PACKET_LOSS_CORRELATION}
$TC filter add dev $DEV protocol ip parent 1: u32 match ip src $IP flowid 1:20
To stop emulation run ./your_file_name stop
Last edited by VladSun on Sun Sep 05, 2010 3:19 am, edited 2 times in total.
There are 10 types of people in this world, those who understand binary and those who don't
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Re: Network emulator

Post by josh »

What would you test with this in PHP?
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Network emulator

Post by VladSun »

fsockopen :P

It's not PHP specific, it's a web development testing tool.
There are 10 types of people in this world, those who understand binary and those who don't
jarofgreen
Forum Commoner
Posts: 71
Joined: Sun Jul 11, 2010 12:40 pm

Re: Network emulator

Post by jarofgreen »

Progressive loading.

You want to make sure your web-page will look good as it loads slowly over a slow connection. One thing in particular is that people do a form, then do a JS onload event that attaches functionality to the inputs. If you do it badly, what can happen is that a user on a slow connection can get the form, start to use it, then the page finishes and the onload fires, and somehow, that form is messed up for that user.

Very handy. I was going to be looking for something like this soon myself. How does this work - does it effect traffic from a certain IP, so could I run it with the IP of my development server for instance?
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Re: Network emulator

Post by josh »

I think this is too pro for me to understand, vlad can you list an example how I would test this out? :D
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Re: Network emulator

Post by Weirdan »

josh wrote:I think this is too pro for me to understand, vlad can you list an example how I would test this out? :D
Well, that seems to be easy:

Code: Select all

sudo aptitude install iproute
./slowdown.sh
# enjoy your slow connection =)
./slowdown.sh stop
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Network emulator

Post by VladSun »

jarofgreen wrote:Very handy. I was going to be looking for something like this soon myself. How does this work - does it effect traffic from a certain IP, so could I run it with the IP of my development server for instance?
The "IP" config item should be the IP address you are trying to reach. It may need to be edited together with the "DEV" config item (I've just added it) - the network interface you are connecting through.

With the configuration shown, it can be used in situations where both the developer's IDE and the HTTP (or any other network service) server run on a single machine.

This script must be run with superuser privileges (i.e. sudo ./netem.sh)
There are 10 types of people in this world, those who understand binary and those who don't
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Re: Network emulator

Post by Weirdan »

Interestingly, it works when I use it against local machine (DEV="lo" IP="127.0.0.1") but doesn't seem to work with remote connection (DEV="eth0" IP="xxx.xxx.xxx.xxx"). By "doesn't work" I mean bandwidth limits not being applied. I tried to limit bandwidth to 30kbit, but still get ~1.4MBps (megabytes per second) downloads from that remote host.

Tested using wget <url>
User avatar
AbraCadaver
DevNet Master
Posts: 2572
Joined: Mon Feb 24, 2003 10:12 am
Location: The Republic of Texas
Contact:

Re: Network emulator

Post by AbraCadaver »

Using this method you can only affect egress (outgoing packets). To affect ingress you have to use something like Intermediate Functional Block pseudo-device.
mysql_function(): WARNING: This extension is deprecated as of PHP 5.5.0, and will be removed in the future. Instead, the MySQLi or PDO_MySQLextension should be used. See also MySQL: choosing an API guide and related FAQ for more information.
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Re: Network emulator

Post by Weirdan »

AbraCadaver wrote:Using this method you can only affect egress (outgoing packets).
What if I rate-limited outgoing ACKs? I realize it's not quite the same as bandwidth limit, but my understanding is that it would cause remote host to shrink send window quickly, effectively limiting bandwidth for the connection.
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Network emulator

Post by VladSun »

AbraCadaver wrote:Using this method you can only affect egress (outgoing packets). To affect ingress you have to use something like Intermediate Functional Block pseudo-device.
Or simple ingress tc
There are 10 types of people in this world, those who understand binary and those who don't
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Network emulator

Post by VladSun »

Weirdan wrote:
AbraCadaver wrote:Using this method you can only affect egress (outgoing packets).
What if I rate-limited outgoing ACKs? I realize it's not quite the same as bandwidth limit, but my understanding is that it would cause remote host to shrink send window quickly, effectively limiting bandwidth for the connection.
I remember there was a TCP window-based TC implementation... can't find it now.... It's very hard to estimate the number of dropped ACK packets, so to achieve the expected incoming bandwidth. The main reason is there are a number of window management algorithms, which behave completely different.
There are 10 types of people in this world, those who understand binary and those who don't
User avatar
AbraCadaver
DevNet Master
Posts: 2572
Joined: Mon Feb 24, 2003 10:12 am
Location: The Republic of Texas
Contact:

Re: Network emulator

Post by AbraCadaver »

That would be hackish and kind of defeat the purpose here to be able to add real values such as bandwidth, latency, jitter, etc. and see the effects. Use IFB, something like:

[text]modprobe ifb
ip link set dev ifb0 up
tc qdisc add dev eth0 ingress
tc filter add dev eth0 parent ffff: protocol ip u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb0

# then start using the ifb0 dev
tc qdisc add dev ifb0 root netem delay 100ms[/text]

It's been a while, but as far as delay goes you may have to use the one-way latency since it will be applied ingress and egress. So the above would give 200ms round-trip latency.
mysql_function(): WARNING: This extension is deprecated as of PHP 5.5.0, and will be removed in the future. Instead, the MySQLi or PDO_MySQLextension should be used. See also MySQL: choosing an API guide and related FAQ for more information.
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Re: Network emulator

Post by Weirdan »

You see, I don't need anything fancy and there's no requirements for any kind of precise control. Just make all connections to specific host(s) "slow", make them fast again. It's to be used in testing to see how web application would behave for a client on a slow connection.

For now what you guys discussing is a bit over my head, as I only have a basic understanding of network stack and absolutely no experience with linux networking (I had some experience on FreeBSD, but that was like 10 years ago).
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Network emulator

Post by VladSun »

Nice one, AbraCadaver :)
Maybe Weirdan needs something less complex like in http://blog.stevedoria.net/20050906/ing ... nux-and-tc
There are 10 types of people in this world, those who understand binary and those who don't
Post Reply