RESULTS | privilege-based-qos

During testing, we discovered that the pfifo_fast qdisc was not behaving as expected based on all of our research online and through what was described in the Linux man pages. Pfifo_fast is the default qdisc for interfaces in Linux and is supposed to work by queuing packets based on the TOS bits if they are set. It is clearly specified in the man pages which TOS bits should be set in order for the packet to get sent to the most prioritized transmission queue. We have tried setting all of the different bits that had been specified and none of them showed any signs of improvement for the QoS, nor did any of them display behavior like what was detailed in documentation- such as transmitting from the higher prioritized queue completely before transmitting from queues with lower priority.

We noticed these things so we decided to look at another qdisc in addition to pfifo_fast called Hierarchical Token Bucket. This qdisc was only implemented on the QoS edge router, nowhere else. This works by allowing the user to set up their own queues with their own classes and filters. We set one up which matches on the TOS bits of packets, as to use our existing idea, and we made packets which had 0x14 for the TOS bits get sent to the queue with prio 1 which is the queue of the highest priority. We set up another queue which checked for the TOS bits 0x08 and put those packets in the queue with prio 3 which is the lowest of the three queues we defined. The middle queue is used as the default for when the TOS bits do not equal 0x14 or 0x08. The different queues are prioritized by having a threshold set for the max bandwidth that the queue gets. The higher priority queue obviously gets a higher bandwidth threshold. In our implementation, we have defined a maximum link bandwidth of 12000 kbps and queue 1 gets 90% of that bandwidth to be the threshold. We defined the threshold for queue 3 to be 7000 kbps less than that. During transmission, queue 1 gets about 10,800 kbps while queue 3 gets about 3,800 kbps.

For the following self studies, we have tested using both of these queuing disciplines except where specified otherwise. Early into the self studies, however, it becomes clear that the pfifo_fast qdisc is not acting effectively for managing QoS.

SelfStudy1

Description

How much are non-prioritized users negatively affected by the QoS implementation? There may be a heavy negative impact to un-prioritized users when a prioritized user sends a large file as this file transmission will cause all un-prioritized users to be slowed down. This self study will compare the time it takes to send a file from a source node on the Base Case network to the time it takes to send a file from a non-prioritized user on the Final Topology network from Fig.4. This will be evaluated with varying degrees of cross traffic.

Plan

Base Case:

Network Setup: The final topology (fig) but without any QoS modules loaded.

Test 1: Get the average time it takes for a user to send file to FTP server:

a) when it is the only one transmitting
b) while another user is sending a file
c) while two other users are transmitting as well
d) while three other users are transmitting as well
e) while all four users are sending files and when there is FTP cross traffic

Study Case:

Network Setup: The final topology with all QoS modules loaded, one unpaid user, three paid users.

Test 1: Get average time it takes for an unpaid user to send file to FTP server

a) when it is the only one transmitting
b) while a paid user is sending a file
c) while two paid users are transmitting
d) while three paid users are transmitting
e) while all four users are sending files and when there is FTP cross traffic

Results:

The file being used for FTP transmissions is 20 MB in size. Every test was run 5 times consecutively. FTP cross traffic is caused by sending 100 MB files to the iperf server in a loop

Base Case:

Test1:

User1

User2

User3

User4

18.08

35.56

32.81

50.72

50.47

54.68

73.13

70.45

68.3

70.85

134.5

129.11

119.55

126.55

All users have the same priority.
As the number of users increase the network resources are divided between each member.
Because of division of resources, as the number of users increase, the time taken to transfer the file increases proportionally.

Study Case:

Test1:

pfifo_fast

User1(paid)

User2(paid)

User3(paid)

User4(unpaid)

16.95

32.1

34.5

51.6

54.6

49.1

65.7

72.3

71.8

68.5

100.1

118.6

116.4

104.7

When the network has only one user (unpaid) is using the resource, it sends the file at a much faster rate.
When the number of paid users to unpaid users is 1 the paid gets a differential service but not by much.
When the number of paid users is increased, the unpaid users has speeds either equal to or greater than the paid users which defeats the purpose of QoS.

Hierarchical Token Bucket only on qos router

Upload cap: 12000
Min = max - 7000

User1(paid)

User2(paid)

User3(paid)

User4(unpaid)

46.9

25.5

46.4

46.1

52.9

56.8

69.8

66.4

73.2

66.7

86.8

97.6

97.5

101.1

As more paid users are added, the transfer time of the unpaid user gets closer to and eventually equal to that of the paid users.
Looks like the unpaid user's time is equal to that of the paid users when there are 3 paid users and 1 unpaid user.
The FTP cross traffic slowed down the transmission of the unpaid user since the cross traffic packets went to the unpaid queue 3.

SelfStudy2

Description

What delay changes will there be if the ratio of privileged to non-privileged users changes? Since the final topology has a 1:1 ratio of privileged to non-privileged users, what kinds of changes in delay will users experience if the ratio changes to any of the following: 0:4, 1:3, 3:1, 4:0?

*Base Case: final topology with 4 users, 2 paid, 2 unpaid

Plan:

Base:

Network setup: Final network topology (fig) with the QoS implementation loaded. There will be 2 paid and 2 unpaid users.

Test 1: Get the average delay for paid/unpaid users when a file is transmitted to the FTP server with no cross traffic.

a) for 2 unpaid/2 paid users

Study Case:

Network setup: Final network topology (fig) with the QoS implementation loaded. The types of users are varied as part of this study.

Test 1: Get the average delay for paid/unpaid users when a file is transmitted to the FTP server with no cross traffic:

a) For 0 unpaid/4 paid
b) for 4 unpaid/0 paid
c) for 1 unpaid/3 paid
d) for 3 unpaid/1 paid

Results:

Base Case:

Test 1:

pfifo_fast

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

69.9

67.2

73.0

64.4

These results do not reflect that a user being marked as paid or unpaid has any meaningful effect on which buffer a user's packets may be placed into. The values for the TOS bits that should allow a packet to be placed into a certain band (queue) were identified in the man pages of the very same Linux system as the default values for the default qdisc pfifo_fast. However, it seems that pfifo_fast is not working as documented.

Hierarchical Token Bucket only on QoS edge router

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

56.1

49.6

85.5

90.5

The paid users were transmitted much more quickly than the unpaid users because the unpaid users are sent to a queue with a much lower bandwidth ceiling than the queue for the paid users. The bandwidth is capped at about 11000 kbps for paid users and although that ceiling may not actually be reached, the unpaid users are severely limited with the ceiling of about 4000 kbps. Both queues have traffic from two users each which are competing for transmission.

Study Case:

Test1:

pfifo_fast

User1(paid)

User2(paid)

User3(paid)

User4(paid)

72.1

68.4

70.8

64.0

User1(unpaid)

User2(unpaid)

User3(unpaid)

User4(unpaid)

72.4

69.5

70.7

67.8

User1(paid)

User2(paid)

User3(paid)

User4(unpaid)

65.7

72.3

71.8

68.5

User1(paid)

User2(unpaid)

User3(unpaid)

User4(unpaid)

72.185

68.7

68.3

67.9

These results further suggest that pfifo_fast is not operating as is documented. The TOS bits have been set to any of the possible values that should allow the packet to be placed in band 0, however this never happens, as is apparent from the seemingly random times in relation to whether the node is a paid or non-paid user. Many times in these results, the unpaid user is receiving a faster transmission time than that of the paid users. Overall, the times for paid and unpaid users is collectively about the same.

Hierarchical Token Bucket only on QoS edge router

User1(paid)

User2(paid)

User3(paid)

User4(paid)

72.4

68.6

72.1

69.9

User1(unpaid)

User2(unpaid)

User3(unpaid)

User4(unpaid)

162.1

113.1

182.4

161.2

User1(paid)

User2(paid)

User3(paid)

User4(unpaid)

69.8

66.4

73.2

66.7

User1(paid)

User2(unpaid)

User3(unpaid)

User4(unpaid)

27.2

127.2

131.8

120.7

A) for this case, all packets are put into the queue with the highest bandwidth ceiling so they are transmitted at relatively equal rates. The unpaid users in the base case have much longer transmission times than any of these since those are placed in a queue with a much lower bandwidth cap. The paid users in the base case were transmitted faster than these packets because there were packets from only two nodes trying to get through that specific queue.
B) for this case, all the packets were placed in the queue which had the lowest bandwidth ceiling. The transmission times are so much slower for these files compared to that of both the paid and unpaid users in the base case because there were 4 nodes competing to transmit packets through the same queue, on top of it being the queue with the lowest bandwidth.
C) This case involved 3 paid users and one unpaid user. The unpaid user was able to have a transmission time similar to that of the paid users because the three paid users had to compete in the same queue and although that queue has a much higher bandwidth ceiling than the unpaid user's queue, the unpaid user was able to catch up.
D) The file from the single paid user was received substantially faster than that of the unpaid users. This is because it was in the high priority queue with the high bandwidth cap by itself and the other packets from the unpaid users were competing with each other in the low priority queue which had the lower bandwidth cap. The paid user in this case was able to transmit even faster than the paid users in the base case because, in this example, it was the only paid user in the queue.

SelfStudy3

Description

Does increased levels of cross traffic cause the performance gains of our QoS to diminish for prioritized users? This will compare the delays for the prioritized users and non-prioritized users as the cross traffic is varied.

Plan:

Base Case:

Network setup: Final network topology (fig) with the QoS implementation loaded. There will be 2 paid and 2 unpaid users.

Test 1: Get the average delay for paid/unpaid users when a file is transmitted to the FTP server with no cross traffic.

a) for 2 unpaid/2 paid users

Study Case:

Network setup: Final network topology (fig) with the QoS implementation loaded. There will be 2 paid and 2 unpaid users. Cross traffic will be varied as part of this study.

Test 1: Get average delay for paid/unpaid users when there is iperf cross traffic:

a) 1 flow of iperf cross traffic flowing from Cross Nodes XN1 and XN2 each to the iperf server
b) 3 flows of iperf cross traffic flowing from Cross Nodes XN1 and XN2 each to the iperf server
c) 5 flows of iperf cross traffic flowing from Cross Nodes XN1 and XN2 each to the iperf server

Test2: Get average delay for paid/unpaid users when there is FTP cross traffic

a) constant FTP transmission from Cross Nodes XN1 and XN2 each to the iperf server

Test3: Get average delay for paid/unpaid users when there is FTP and iperf cross traffic

a) constant FTP transmission and 5 iperf flows from Cross Nodes XN1 and XN2 each to the iperf server

Results:

A 5 MB file was used for all of these tests. Every test was run 5 times consecutively and the average was taken of the results.

Base Case:

pfifo_fast

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

28.8

29.8

28.4

24.2

These results further suggest that pfifo_fast is not operating as is documented. The TOS bits have been set to any of the possible values that should allow the packet to be placed in band 0, however this never happens.

Hierarchical Token Bucket

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

10.1

12.2

22.0

21.7

With traffic control, we are able to obtain differential service for paid users.

Study Case:

Test1:

pfifo_fast

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

73.7

66.2

68.3

67.1

135.9

129.2

134.1

136.8

143.9

141.8

151.5

151.0

The values appear to be random in relation to the notion that the paid users should have better transmission times than the unpaid users.

Hierarchical Token Bucket only on qos

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

25.01

27.5

38.2

33.8

63.2

59.2

81.4

70.4

102.1

99.6

116.3

119.2

The difference between the times of HTB and pfifo may have to do with the network's connection at that time cause it has been spotty. Although the paid users' queue may have a higher ceiling, if the network connectivity drops for whatever reason GENI may have it drop, then the bandwidth of the links may drop low enough so that essentially the paid and unpaid users have access to the same amount of bandwidth. As in, the usable bandwidth for the paid users may drop to be capped at the same ceiling that the unpaid users have.

Test2:

pfifo_fast

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

22.8

27.3

26.2

22.6

The FTP cross traffic had a much less impact than did the iperf flows

Hierarchical Token Bucket only on qos

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

16.38

19.5

30.4

29.1

The FTP cross traffic had a much less impact than did the iperf flows

Test3:

pfifo_fast

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

189.8

197.3

203.2

203.5

Hierarchical Token Bucket only on qos

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

231.1

230.1

282.5

246.6

HTB was still able to keep a better QoS for the paid users even in the midst of all this cross traffic. It may seem like pfifo_fast maintained a better overall throughput but this is most likely due to inconsistencies in the GENI network.

SelfStudy4

Description

Determine the effects of setting various TOS values in the iperf traffic. Since our QoS implementation relies on TOS bits to mark and prioritize packets, perhaps cross traffic with varying TOS values will have an adverse effect on the QoS.

Plan:

Base Case:

Network setup: Final network topology (fig) with the QoS implementation loaded. There will be 2 paid and 2 unpaid users.

Test 1: Get the average delay for paid/unpaid users when a file is transmitted to the FTP server with no cross traffic.

a) for 2 unpaid/2 paid users

Study Case:

Network setup: Final network topology (fig) with the QoS implementation loaded. There will be 2 paid and 2 unpaid users. Cross traffic will be varied as part of this study.

Test 1: Get delays of paid/unpaid users when iperf cross traffic has TOS bits set. There will be 3 iperf flows coming from each Cross Node to the iperf server.

a) TOS=0x08
b) TOS=0x14
c) Other TOS value for queue 1 (depending whether pfifo_fast or Hierarchical Token Bucket qdisc)
d) Other TOS value for queue 2 (depending whether pfifo_fast or Hierarchical Token Bucket qdisc)
e) Other TOS value for queue 3 (depending whether pfifo_fast or Hierarchical Token Bucket qdisc)

Results:

Base Case:

Test1: (pfifo_fast)

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

69.9

67.2

73.0

64.4

Test2: (Hierarchical Token Bucket) only on qos

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

56.1

49.6

85.5

90.5

Study Case:

Test1:

(pfifo_fast)

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

A) 0x08

96.0

94.7

98.1

96.8

B) 0x14

105.1

99.0

106.4

114.3

C) 0x12

112.3

105.6

110.2

118.1

D) 0x0

109.2

99.8

125.8

110.7

E) 0xc

108.3

117.2

126.2

124.3

The reason for setting the TOS of the cross traffic to 0x08 is to observe the effects of adding more traffic which is similar to the existing traffic and will be treated as such. This means that this traffic will be placed in all the same queues and will be treated the same by hooks. This traffic may slow down ftp traffic which is not marked for prioritization.
The reason for setting the TOS of the cross traffic to 0x14 is to observe the effects of adding more traffic which is similar to the existing traffic and will be treated as such. This means that this traffic will be placed in all the same queues and will be treated the same by hooks. This traffic may slow down traffic that is marked as prioritized.
It does not seem that the TOS bits of the cross traffic made too much of a difference for the transmission rate of the files. The latter transmissions with the latter TOS bits did take a little bit longer but that is more likely due to inconsistencies in the GENI network as behavior like this was noticed frequently and much more drastically.

(Hierarchical Token Bucket) only on qos

User1(paid)

User2(paid)

User3(unpaid)

User4(unpaid)

A) 0x08

74.6

70.4

96.6

86.4

B) 0x14

73.18

73.7

76.3

105.6

C) N/A

D) 0x0

81.6

68.5

98.1

96.5

E) N/A

The only TOS that will put packet in queue 1 is 0x14 for HTB
The only TOS that will put packet in queue 3 is 0x08 for HTB
The results do not point out for sure that some certain TOS bits in the cross traffic might affect the transmission rate of the files through FTP.

SelfStudy5

Description

How does file size affect the QoS? Does the QoS provide benefits in speed to files of all sizes compared to the topology without QoS enabled? The main comparison will be between the delays for the file transfers of the paid users vs. the non-paid users for each file size, but the base case will include the delays for the different file sizes on a non-QoS enabled topology for reference.

Plan:

Base Case:

Network Setup: The final topology (fig) but without any QoS modules loaded.

Test 1: Check delay on file transfers when 4 users are transmitting with no QoS enabled with file size:

a) 1MB
b) 5MB
c) 10MB
d) 20MB
e) 36MB
f) 113MB

Study Case:

Network Setup: The final topology (fig) with all QoS modules loaded. There will be 2 paid users and 2 non-paid users.

Test 1: Check delay on file transfers when 4 users are transmitting with QoS implemented with file size:

a) 1MB
b) 5MB
c) 10MB
d) 20MB
e) 36MB
f) 113MB

Results:

Base Case:

Test1 :

User1

User2

User3

User4

1.03

2.3

2.44

16.21

13.72

16.29

17.1

31.242

28.9

32.1

70.85

68.3

70.45

73.13

121.62

111.38

120.63

121.51

353.1

346.2

384.3

382.31

User privilege based QoS Under Dr. Rudra Datta

User privilege based QoS
Under
Dr. Rudra Datta