top of page
Design
High level design

Fig1A. Topology 3A

Our network (Fig. 1)  consists of source nodes, cross-nodes, an openflow switch, a QoS edge router, core routers, a setback router, an iPerf server, and a FTP server. A switch (OFS) is always paired with a QoS edge router (QER) and an FTP server (FS) is always paired with a set-back router (SBR). These QoS edge routers connect to core routers (CR) which eventually connects with a set-back router before reaching the destination FTP server. There are no OpenFlow switches or QoS edge routers on the destination side so QoS is not preserved on the reverse path for acknowledgments or responses. OpenFlow switches and QoS edge routers exist at some and not all packet injection points in the network. In reality, the existence of the openflow switches and QoS edge routers will depend on the demand for this QoS from the source users, in order to generate the most revenue. For this project we will have these switches and QoS edge routers at some nodes and not at others. For example, the cross traffic will not be going through openflow switches or QoS routers. This is to introduce some variability in cross traffic where those injection sources will consist of purely traffic without any enhanced QoS.

 

When source nodes send packets out into the network, their packets will first go through the OpenFlow switch and then through QoS edge router.  The Qos edge router has a list of source IP addresses which are privileged users. When the QoS edge router receives these packets, it marks only the packets containing FTP data coming from privileged users are provided high priority by setting the TOS bits in the header. The other packets which come in do not get their TOS bits modified, their TOS bits should stay what they originally were. In our proposal, we mentioned using specifically the DSCP portion of the TOS bits for our service. However, our main idea for how the transmission queue was going to manage prioritized packets was to simply place the prioritized packets at the front of the queue but behind the other queued prioritized packets. We found that this could be done in a much more straightforward way with the pfifo_fast queuing discipline, but with the use of three queues; so we did not need any behavior specific to DSCP and chose to go with this approach.

 

The core routers and the QoS edge routers will have the queuing discipline (qdisc) pfifo_fast enabled and the packets will be forwarded into the network based on this qdisc and their TOS bits. This is how the packets of subscribed users will get prioritized over the packets of standard users. For every hop in a packet’s path to its destination, the routers will make the proper prioritizations. In the packet’s path, the final core router will eventually transmit to the set-back router right before the packets reach the destination FTP server. This set-back router is used to set the TOS bits of the prioritized packets back to the TOS bits that are expected by FTP. The packets are then finally transmitted from the set-back router to the FTP server.

 

This QoS shall be implemented in a network where there may be varying amounts of cross traffic. This cross-traffic will be created by having cross-nodes transmit iPerf traffic across the network through the core routers to the destination iPerf server. This cross-traffic will be used to force packets to compete for transmission at the core routers.

Low level design

  • Queuing discipline: pfifo_fast

  • The queuing discipline (qdisc) pfifo_fast will be taken advantage of in this QoS to provide prioritized  packets with a shorter delay. pfifo_fast is a modified version of FIFO where there are three transmission queues or “bands” of different priority. Each individual band implements FIFO. Band 0 has the highest priority and is typically used for packets marked with the “Interactive” TOS bits. All packets in this band are transmitted before the packets in band 1 can start transmission. Accordingly, all the packets in band 1 must be transmitted before the packets in band 2 can start transmission. The default qdisc for the majority of linux kernels is pfifo_fast and this holds for our kernel version 3.13 which is what we will be using for our gateway and all routers.If this is not the default qdisc, it is necessary to change it to pfifo_fast in order for this QoS to function. Essentially, the packets for subscribers will have their TOS bits in the IP header set so that the packets are queued in band 0 regardless of the original type of service of the packet. This qdisc needs to be enabled specifically on all the routers in the system so that any router in the packet’s path will know how to give the packet prioritization.
     

  • OpenFlow Switch

  • The switch is placed so that we can create the scenario of having a few users in the same network.
     

  • QoS edge router

  • When an FTP connection is initiated from a source node to the FTP server, some initial packets are sent to the server which have specific TOS values. During the connection initiation, if those TOS values are changed to anything else, the FTP connection will fail. For this reason, we can’t blindly change the TOS for all the packets coming from the privileged users on the QoS edge router. When the FTP connection is being requested from the client, the TOS bits of the packets are 0x0. When the packet is sent from the client with the username to login with, the TOS bits are set to 0x0010. When the packet is sent from the client with the password to login with, the TOS bits are also set to 0x0010. After this initial sequence of packets, the TOS bits for all the FTP transmissions are set to 0x0008. Since these are the packets containing the data which we are trying to prioritize the transmission for, we focus on the packets which have the TOS set to 0x0008.  
     

  • A custom module loaded into the QoS edge router’s kernel will have a netfilter NF_INET_PRE_ROUTING hook which will identify the source IP of the packet entering the router on and check the TOS bits. If the packet came in from a privileged user and if the packet has its TOS bits set to 0x0008, we assume that the packet is FTP data which should be prioritized. A netfilter NF_INET_POST_ROUTING hook sets the TOS bits in the IP header of the packet to 0x14 which is the “Interactive” type of service and will thus assign the packet to band 0 which is the transmission queue for the pfifo_fast qdisc with the highest priority. If the packet came through unpaid user, the hook leaves the TOS bits as they were so that the type of service is preserved but not further prioritized.
     

  • If the TOS bits were modified, the original IP checksum will be incorrect for the modified IP header. A new checksum is calculated immediately after changing the TOS bits and is set in the header. The packet is now allowed to continue to the appropriate transmission queue to be sent to the next hop according to the forwarding algorithm. If a packet arrives at a QoS edge router from a link connected to the core routers and not from the switch, the packet will be forwarded towards its destination node as is normal behavior and it will not be processed for QoS by our service. Specifically, the QoS edge router will not try to set the TOS bits for an acknowledgement or response packet on it’s way to the source node.
     

  • Core Router
     

  • When a core router receives packets, they are managed and placed in the appropriate transmission queue by the pfifo_fast qdisc. Prioritized packets, which have TOS bits of 0x14, are sent to band 0. Non-prioritized packets are placed in the correct band according to their original TOS. The packets are then forwarded to their next hop. This prioritization occurs for prioritized packets for each router that they traverse until they get to the set-back router.
     

  • Set-back Router
     

  • The set-back router receives all packets destined to the FTP server. The FTP server is expecting its FTP data packets to have a TOS bit value of 0x0008 and will not accept them if this is not the value. Since the QoS edge router previously modified those TOS bits, this router needs to set the bits back to 0x0008 before they are transmitted to the FTP server so that they may be transmitted successfully. A netfilter NF_INET_PRE_ROUTING hook identifies prioritized packets by checking for TOS bits equal to 0x14. For all those packets, a netfilter NF_INET_POST_ROUTING hook sets the TOS bits back to 0x0008 so that they will be accepted by the FTP server. Therefore, there is no QoS for the final hop from the set-back router to the server. Since the TOS bits were modified, the original IP checksum will be incorrect for the modified IP header. A new checksum is calculated immediately after changing the TOS bits and is set in the header. The packet is then allowed to continue to the server.
     

  • FTP Server
     

  • The FTP server is the desired destination for packets sent from the source nodes. This server may be sent files of arbitrary types including images, videos, text, etc. When receiving the packets for these files, the server expects the TOS values to be set to 0x0008 or they will not be accepted. To setup the FTP server, changes have to be made to the /etc/vsftpd.conf file. The following lines should be uncommented:

  • write_enable=YES

  • local_unmask=022

  • chroot_local_user=YES

  • The following lines need to be added to the same file:

  • allow_writable_chroot=YES

  • pasv_enable=Yes

  • pasv_min_port=40000

  • pasv_max_port=40100

  • The following line needs to be added to /etc/shells:

  • /usr/sbin/nologin

  • Since the FTP server does not sit behind an openflow switch and QoS edge router, there is no QoS on the reverse path.

Fig1B. Topology 3B

Purpose

Project under progress as a part of coursework for Internet Protocols (ECE/CSC 573), a graduate level course at the NC State University.

Param Vijay Dinkar
Ruthvik Mandava
Karthik Haria
Michael Ehmke

Project Team

bottom of page