Hi, I want to achieve the following on a 100mbit dedicated server. traffic coming from Nginx on port 81: max 50 mbit usage all other traffic can use all they like. I created 3 different classes with tc qdisk via the following ssh commands tc qdisc del dev eth0 root tc qdisc add dev eth0 root handle 1: htb default 1 r2q 160 tc class add dev eth0 parent 1: classid 1:1 htb rate 105000kbit burst 1k tc class add dev eth0 parent 1:1 classid 1:2 htb rate 105000kbit ceil 105000kbit burst 1k tc class add dev eth0 parent 1:1 classid 1:3 htb rate 50000kbit ceil 50000kbit burst 1k tc qdisc add dev eth0 parent 1:2 handle 2: sfq perturb 10 tc qdisc add dev eth0 parent 1:3 handle 3: sfq perturb 10 Code (markup): This goes without problems. I then want to apply this to Iptables where it goes wrong iptables -t mangle -A OUTPUT -o eth0 -j CLASSIFY --set-class 1:2 iptables -t mangle -A OUTPUT -o eth0 -m tcp -p tcp --dport 81 -j CLASSIFY --set-class 1:3 Code (markup): when entering these commands it does not do anything; port 81 still uses the full 100 mbit from the server. the output from iptables -L is: serveur:~# iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination all -- anywhere anywhere Code (markup): I have hardly any understanding of tc and iptables so I have no idea what is wrong... I hope someone can figure out what's wrong Additional info: fast:~# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP qlen 100 link/ether 00:1c:c0:* brd ff:ff:ff:* inet 76.73.*/29 brd 76.73.* scope global eth0 inet 76.73.*/29 brd 76.73.* scope global secondary eth0:0 inet 76.73.*/29 brd 76.73.* scope global secondary eth0:1 inet 76.73.*/29 brd 76.73.* scope global secondary eth0:2 inet 76.73.*/29 brd 76.73.* scope global secondary eth0:3 inet6 fe80::21c:*/64 scope link valid_lft forever preferred_lft forever Code (markup):
It seems that your iptables doesn't show up in the iptables -L listing. Do you get any error message?
fast:~# tc qdisc del dev eth0 root fast:~# tc qdisc add dev eth0 root handle 1: htb default 1 r2q 160 fast:~# tc class add dev eth0 parent 1: classid 1:1 htb rate 105000kbit burst 1k fast:~# tc class add dev eth0 parent 1:1 classid 1:2 htb rate 105000kbit ceil 10 5000kbit burst 1k fast:~# tc class add dev eth0 parent 1:1 classid 1:3 htb rate 50000kbit ceil 500 00kbit burst 1k fast:~# tc qdisc add dev eth0 parent 1:2 handle 2: sfq perturb 10 fast:~# tc qdisc add dev eth0 parent 1:3 handle 3: sfq perturb 10 fast:~# iptables -t mangle -A OUTPUT -o eth0 -j CLASSIFY --set-class 1:2 fast:~# iptables -t mangle -A OUTPUT -o eth0 -m tcp -p tcp --dport 81 -j CLASSIF Y --set-class 1:3 fast:~# iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination all -- anywhere anywhere fast:~# Code (markup): no error message iptables v1.4.2 When I try to make a new chain I get this: fast:~# iptables -N NGINX fast:~# iptables -L NGINX Chain NGINX (0 references) target prot opt source destination fast:~# iptables -t mangle -A NGINX -o eth0 -j CLASSIFY --set-class 1:2 iptables: No chain/target/match by that name fast:~# iptables -P NGINX ACCEPT iptables: Bad built-in chain name Code (markup):
Hi, the error was that it should have been --sport 81 (I think) I used a script i found here: http://www.faqs.org/docs/Linux-HOWTO/ADSL-Bandwidth-Management-HOWTO.html and modified it to my needs, it works on my server. #!/bin/bash # # myshaper - DSL/Cable modem outbound traffic shaper and prioritizer. # Based on the ADSL/Cable wondershaper (www.lartc.org) # # Written by Dan Singletary (8/7/02) # # NOTE!! - This script assumes your kernel has been patched with the # appropriate HTB queue and IMQ patches available here: # (subnote: future kernels may not require patching) # # http://luxik.cdi.cz/~devik/qos/htb/ # http://luxik.cdi.cz/~patrick/imq/ # # Configuration options for myshaper: # DEV - set to ethX that connects to DSL/Cable Modem # RATEUP - set this to slightly lower than your # outbound bandwidth on the DSL/Cable Modem. # I have a 1500/128 DSL line and setting # RATEUP=90 works well for my 128kbps upstream. # However, your mileage may vary. # RATEDN - set this to slightly lower than your # inbound bandwidth on the DSL/Cable Modem. # # # Theory on using imq to "shape" inbound traffic: # # It's impossible to directly limit the rate of data that will # be sent to you by other hosts on the internet. In order to shape # the inbound traffic rate, we have to rely on the congestion avoidance # algorithms in TCP. Because of this, WE CAN ONLY ATTEMPT TO SHAPE # INBOUND TRAFFIC ON TCP CONNECTIONS. This means that any traffic that # is not tcp should be placed in the high-prio class, since dropping # a non-tcp packet will most likely result in a retransmit which will # do nothing but unnecessarily consume bandwidth. # We attempt to shape inbound TCP traffic by dropping tcp packets # when they overflow the HTB queue which will only pass them on at # a certain rate (RATEDN) which is slightly lower than the actual # capability of the inbound device. By dropping TCP packets that # are over-rate, we are simulating the same packets getting dropped # due to a queue-overflow on our ISP's side. The advantage of this # is that our ISP's queue will never fill because TCP will slow it's # transmission rate in response to the dropped packets in the assumption # that it has filled the ISP's queue, when in reality it has not. # The advantage of using a priority-based queuing discipline is # that we can specifically choose NOT to drop certain types of packets # that we place in the higher priority buckets (ssh, telnet, etc). This # is because packets will always be dequeued from the lowest priority class # with the stipulation that packets will still be dequeued from every # class fairly at a minimum rate (in this script, each bucket will deliver # at least it's fair share of 1/7 of the bandwidth). # # Reiterating main points: # * Dropping a tcp packet on a connection will lead to a slower rate # of reception for that connection due to the congestion avoidance algorithm. # * We gain nothing from dropping non-TCP packets. In fact, if they # were important they would probably be retransmitted anyways so we want to # try to never drop these packets. This means that saturated TCP connections # will not negatively effect protocols that don't have a built-in retransmit like TCP. # * Slowing down incoming TCP connections such that the total inbound rate is less # than the true capability of the device (ADSL/Cable Modem) SHOULD result in little # to no packets being queued on the ISP's side (DSLAM, cable concentrator, etc). Since # these ISP queues have been observed to queue 4 seconds of data at 1500Kbps or 6 megabits # of data, having no packets queued there will mean lower latency. # # Caveats (questions posed before testing): # * Will limiting inbound traffic in this fashion result in poor bulk TCP performance? # - Preliminary answer is no! Seems that by prioritizing ACK packets (small <64b) # we maximize throughput by not wasting bandwidth on retransmitted packets # that we already have. # # NOTE: The following configuration works well for my # setup: 100/100M FDC box and 100/100M ovh box DEV=eth0 RATEUP=101000 LIMITUP=75000 # # End Configuration Options # if [ "$1" = "status" ] then echo "[qdisc]" tc -s qdisc show dev $DEV # tc -s qdisc show dev imq0 echo "[class]" tc -s class show dev $DEV # tc -s class show dev imq0 echo "[filter]" tc -s filter show dev $DEV # tc -s filter show dev imq0 echo "[iptables]" iptables -t mangle -L MYSHAPER-OUT -v -x 2> /dev/null # iptables -t mangle -L MYSHAPER-IN -v -x 2> /dev/null exit fi # Reset everything to a known state (cleared) tc qdisc del dev $DEV root 2> /dev/null > /dev/null #tc qdisc del dev imq0 root 2> /dev/null > /dev/null iptables -t mangle -D POSTROUTING -o $DEV -j MYSHAPER-OUT 2> /dev/null > /dev/null iptables -t mangle -F MYSHAPER-OUT 2> /dev/null > /dev/null iptables -t mangle -X MYSHAPER-OUT 2> /dev/null > /dev/null #iptables -t mangle -D PREROUTING -i $DEV -j MYSHAPER-IN 2> /dev/null > /dev/null #iptables -t mangle -F MYSHAPER-IN 2> /dev/null > /dev/null #iptables -t mangle -X MYSHAPER-IN 2> /dev/null > /dev/null #ip link set imq0 down 2> /dev/null > /dev/null #rmmod imq 2> /dev/null > /dev/null if [ "$1" = "stop" ] then echo "Shaping removed on $DEV." exit fi ########################################################### # # Outbound Shaping (limits total bandwidth to RATEUP) # set queue size to give latency of about 2 seconds on low-prio packets #ip link set dev $DEV qlen 30 # changes mtu on the outbound device. Lowering the mtu will result # in lower latency but will also cause slightly lower throughput due # to IP and TCP protocol overhead. #ip link set dev $DEV mtu 1000 # add HTB root qdisc tc qdisc add dev $DEV root handle 1: htb default 26 # add main rate limit classes tc class add dev $DEV parent 1: classid 1:1 htb rate ${RATEUP}kbit # add leaf classes - We grant each class at LEAST it's "fair share" of bandwidth. # this way no class will ever be starved by another class. Each # class is also permitted to consume all of the available bandwidth # if no other classes are in use. tc class add dev $DEV parent 1:1 classid 1:19 htb rate 50kbit ceil ${RATEUP}kbit prio 0 tc class add dev $DEV parent 1:1 classid 1:20 htb rate 15000kbit ceil ${RATEUP}kbit prio 1 tc class add dev $DEV parent 1:1 classid 1:21 htb rate 15000kbit ceil ${LIMITUP}kbit prio 2 tc class add dev $DEV parent 1:1 classid 1:26 htb rate 50kbit ceil ${RATEUP}kbit prio 3 # attach qdisc to leaf classes - here we at SFQ to each priority class. SFQ insures that # within each class connections will be treated (almost) fairly. tc qdisc add dev $DEV parent 1:19 handle 19: sfq perturb 10 tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev $DEV parent 1:21 handle 21: sfq perturb 10 tc qdisc add dev $DEV parent 1:26 handle 26: sfq perturb 10 # filter traffic into classes by fwmark - here we direct traffic into priority class according to # the fwmark set on the packet (we set fwmark with iptables # later). Note that above we've set the default priority # class to 1:26 so unmarked packets (or packets marked with # unfamiliar IDs) will be defaulted to the lowest priority # class. tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 19 fw flowid 1:19 tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 20 fw flowid 1:20 tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 21 fw flowid 1:21 tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 26 fw flowid 1:26 # add MYSHAPER-OUT chain to the mangle table in iptables - this sets up the table we'll use # to filter and mark packets. iptables -t mangle -N MYSHAPER-OUT iptables -t mangle -I POSTROUTING -o $DEV -j MYSHAPER-OUT # add fwmark entries to classify different types of traffic - Set fwmark from 20-26 according to # desired class. 20 is highest prio. iptables -t mangle -A MYSHAPER-OUT -p tcp --dport ssh -j MARK --set-mark 19 # secure shell iptables -t mangle -A MYSHAPER-OUT -p tcp --sport ssh -j MARK --set-mark 19 # secure shell iptables -t mangle -A MYSHAPER-OUT -p tcp --sport http -j MARK --set-mark 20 # Local web server iptables -t mangle -A MYSHAPER-OUT -p tcp --dport http -j MARK --set-mark 20 # Local web server iptables -t mangle -A MYSHAPER-OUT -p tcp --sport 81 -j MARK --set-mark 21 # Local web server iptables -t mangle -A MYSHAPER-OUT -p tcp --dport 81 -j MARK --set-mark 21 # Local web server iptables -t mangle -A MYSHAPER-OUT -m mark --mark 0 -j MARK --set-mark 26 # redundant- mark any unmarked packets as 26 (low prio) # Done with outbound shaping # #################################################### echo "Outbound shaping added to $DEV. Rate: ${RATEUP}Kbit/sec." Code (markup):
Bumping this thread with a new question. How to do this if a server runs with 'pfifo_fast' in stead of 'htb'? I wanted to apply this to another server from ovh but it errors at the qdisc add. I tried googling and found out this this specific is not possible with pfifo_fast so I want to ask if anyone knows if I can put port 80 at least on a higher priority or has another solution for this