End to end L3 QoS using MQC

Quality of Service

4.4.a i DiffServ

General information on “L3 QoS DiffServ”:

• Stands for Differential Services
• Connectionless model
• Traffic is grouped in classes
• QoS behavior is defined by traffic’s class
• Called per-hop behavior (PHB)

4.4.a ii CoS and DSCP Mapping

General information on “L3 QoS CoS and DSCP Mapping”:

• A mapping table is used when going from L2 to L3 and vice versa
• L2 to L3: cos-to-dscp table
• L3 to L2: dscp-to-cos table

4.4.a iii Classification

General information on “L3 QoS Classification”:

• Traffic classification normally occurs at network ingress edge
• Typically, classification is done manually
• Some end devices (like VoIP phones, …) can classify their traffic before they send it out
• To accept these markings, the ports must be trusted by QoS or else they get removed
• Classification is mainly done based on three things: Markings, addressing (source/destination), Application signature (NBAR)

Configuration considerations:

• Class-Maps are used to specify the traffic which should later be marked
• Class-Maps can be created with two options:
  • match-all: all defined matches must match in order to get marked (= AND operator) (default behavior)
  • match-any: one of the defined matches must match in order to get marked (= OR operator)
• Matching can be done against several options (examples):
  • ACL
  • CoS value
  • Destination Address
  • DSCP value
  • Input Interface
  • IP Precedence value
  • Protocol type
  • Source Address
  • […]

“L3 QoS Classification” CLI configuration commands:

## Configuring a classification map
Router(config)# class-map [match-all | match-any] [NAME]

## Matching traffic based on various options
Router(config-cmap)# match [option] [arguments]

4.4.a iv Network Based Application Recognition (NBAR)

General information on “L3 QoS Network Based Application Recognition (NBAR)":

• Provides intelligent network classification
• Can recognize a wide variety of applications
• NBAR is based on deep packet inspection
• Mainly used for QoS and security purposes
• Implemented by default on most newer Cisco platforms
• NBAR database can often be updated separately to support new protocols
• Classification based on NBAR is using the match protocol command

4.4.a v Marking using IP Precedence, DSCP, CoS

// Graphic missing - Coming Soon //

General information on “L3 QoS Marking using IP Precedence, DSCP, CoS”:

• Requires CEF to be enabled
• L2 marking using CoS:
  • CoS = Class of Services
  • CoS is used for Layer 2 markings (3 bits)
  • CoS field only exists when there‘s a 802.1q header
  • CoS field is part of the 802.1q header and therefor lost at the first L3 gateway
• L3 marking using IP Precedence/DSCP:
  • Layer 3 marking uses the L3 ToS field (1 byte field)
  • Layer 3 marking (IP Precedence/DSCP) is retained end-to-end if it’s not modified/removed in between
  • IP Precedence uses the first 3 bits of the ToS field
  • IP Precedence has 8 classes (the higher the number = the more important is the traffic)
  • DSCP = Differentiated Services Codepoint
  • DSCP uses the first 6 bits of the ToS field
  • DSCP is a more granular version of IP Precedence marking
  • DSCP values CS0-CS7 provide backwards compatibility with IP Precedence by using the first 3 bits of the ToS field
  • Packets with larger DSCP CS values will be given better queueing preference
  • DSCP AF (= Assured Forwarding) defines four classes of queueing together with a drop rate probability
  • The naming scheme is AFxy, where:
    • x = queueing class (1-4, bits 1-3, the higher the better)
    • y = drop probability (1-3, bits 4-5, the lower the better)
  • DSCP EF (= Expedited Forwarding) value is used to get packets scheduled quickly and to give them low latency (decimal 46, binary 101110)
  • DSCP CSx to decimal: 8x + 2y
  • DSCP AFxy to decimal: 8x + 2y

“L3 QoS Marking” CLI configuration commands:

## Configuring a policy map
Router(config)# policy-map [NAME]

## Selecting the class-map which' traffic should be marked
Router(config-pmap)# class [CLASS-MAP-NAME]

## Setting the marking based on the matched traffic
Router(config-pmap-c)# set cos [value]

Router(config-pmap-c)# set ip precedence [value]

Router(config-pmap-c)# set ip dscp [value]

4.4.a vi Policing, shaping

Policing

General information on “L3 QoS Policing”:

• Typically applied at the ingress edge
• Can be applied inbound or outbound
• Traffic that exceeds the rate can be dropped, marked or re-marked
• Policer Terminology:
  • Metering rate = Committed Information Rate (CIR)
  • Averaging interval = Time Committed (TC)
  • Burst Committed (BC) = Amount of bits that could be sent every interval
  • Burst Excessive (BE) = Amount of bits over BC that could be sent during TC after a period of inactivity
• Policer types:
  • Single rate, two-color (one bucket, conform/exceed)
  • Single rate, three-color (two buckets, conform/exceed/violate)
  • Two rate, three-color (two buckets, conform/exceed/violate)
• Policer actions:
  • Conform: Packet size is lower or equal to the number of available BC token.
  • Exceed: Packet size is higher than the number of available BC tokens (and if BE bucket: lower or equal than the number of available BE tokens).
  • Violate: Packet size is higher than the number of available BC and BE tokens.
• Mathematical calculations:
  • Token Refill (BC): (packet arrival time - last packet arrival time) * CIR / 8
  • Token Refill (BE): (packet arrival time - last packet arrival time) * PIR / 8
• Token bucket model:
  • Depending on the policer type, there will be either one or two buckets
  • The bucket(s) will be refilled depending on the time between when the last packet arrived and the new one comes in
  • Example: If 0,1s have passed since the last packet arrived and a new one came in, the bucket will be refilled with 0,1 * CIR tokens (BC bucket) or 0,1 * PIR tokens (BE bucket).
  • 1 token = 1 byte(!) will be forwarded
  • When using two buckets, the first one will be the size of BC (refilling see above) and the second one the size of BE
  • With a single rate, three-color shaper, the second bucket (BC bucket) will be refilled through spillage from the first (BC) bucket
  • With a two rate, three-color shaper, the second bucket (BC bucket) will be refilled technically like the first one but with the value of the PIR
• Example use case:
  • PE connects to CE with Gigabit Ethernet port
  • Circuit is provisioned at 250 Mbps
  • PE applies inbound policer at the port level that transmits traffic <= 250 Mbps and drops traffic > 250 Mbps

Configuration considerations:

• Default conform-action is transmit
• Default exceed-action is drop
• Violate-action is only relevant for three-color policers

“L3 QoS Policing” CLI configuration commands:

## Configuring a single-rate, two/three color policer
Router(config-pmap-c)# police cir <cir> bc <bc> be <be>

## Configuring a two-rate policer
Router(config-pmap-c)# police cir <cir> bc <bc> pir <pir> be <be>

shaping

General information on “L3 QoS shaping”:

• Used to normalize outbound traffic flows
• Can only be applied outbound
• Smooths out traffic bursts
• Prepares traffic for ingress policing
• Traffic that exceeds the rate will be delayed or queued
• Best practice is to ask the ISP For TC, BC, BE values
• Shaper Terminology:
  • Access Rate = Physical port speed
  • Committed Information Rate (CIR) = Average guaranteed rate the shaper is targeting (in bits/s)
  • Time Committed (TC) = Interval in ms to send traffic bursts (in ms)
  • Burst Committed (BC) = Amount of bits that could be sent every interval (in bits)
  • Burst Excessive (BE) = Amount of bits over BC that could be sent during TC after a period of inactivity (in bits)
• Shaper types:
  • Average Shaper: Average guaranteed rate shaper. Uses BC and BE bucket. BE bucket gets filled with spilled tokens. BC bucket can be fully sent every TC interval. BE bucket can be sent together with the BC bucket after a period of inactivity.
  • Peak Shaper: Peak rate shaper (not guaranteed). Uses BC and BE bucket. Both buckets get fully refilled every TC. BC+BE buckets can be fully sent together every TC interval.
• Mathematical calculation:
  • TC = BC/CIR (eg. BC = 8000 bits, CIR = 64kbps … 8000/64000 = 0.125s / 125ms TC)
  • CIR = (1/TC) * BC (eg. TC = 0.125, BC = 8000 bits … (1/TC) * BC = 64kbps CIR)
  • BC = CIR * TC (eg. CIR = 64kbps, TC = 0.125 … 64kbps * 0.125 = 8000bits BC)
• Token bucket model:
  • Tokens are used for allowing the shaper to send BC (in bits) at each TC interval
  • At the beginning of each TC interval, the token bucket will be refilled with new tokens
  • The size of the token bucket is the size of the BC/BE
  • 1 token = 1 bit will be forwarded
  • When the bucket is empty, the shaper has to wait until the next TC for the token bucket(s) to be refilled and to be able to forward data again
  • BE is implemented by making the single token bucket bigger (new token bucket size = BC + BE)
  • Important: When using the shape peak command the token bucket has the size of BC + BE and will be fully refilled each interval!
• Example use case:
  • PE connects to CE with Gigabit Ethernet port
  • Circuit is provisioned at 250 Mbps
  • PE applies inbound policer at the port level that transmits traffic <= 250 Mbps and queues traffic > 250 Mbps for later transmission
  • Important: Cisco recommends a TC of 10ms for VoIP traffic to prevent jitter/delay!

“L3 QoS shaping” CLI configuration commands:

## Configuring an average-rate based shaper
Router(config-pmap-c)# shape average [value]

## Configuring a peak-rate based shaper
Router(config-pmap-c)# shape peak [value]

4.4.a vii Congestion management and avoidance

General information on “L3 QoS Congestion management and avoidance”:

• Congestion management = queueing
• Important: Queueing is only used when congestion on a link occurs (when the hardware queue is full).
• Congestion avoidance = if and when packets should be dropped
• Queueing methods (congestion management):
  • FIFO (First In First Out):
    • Default queueing method which is self explainable
    • Simplest and easiest to implement
    • Only available configuration parameter is queue-depth
  • CBWFQ (Class Based Weighted Fair Queueing):
    • Configured with the bandwidth command
    • Guarantees a minimum bandwidth per queue/class if there’s congestion on the link
    • If some queues are empty, the bandwidth is proportionally allocates to other classes
    • This means that if there’s no congestion on the link the queue/class can use more bandwidth than the configured (minimum guaranteed) bandwidth
    • Flow-based WFQ within a class can be configured with the fair-queue command
    • Important: If flow-based WFQ within a class is NOT configured, FIFO will be used for each flow within that class which could result in one flow starving out other flows' bandwidth.
  • LLQ (Low Latency Queueing):
    • Configured with the priority command
    • Looks and acts mostly like CBWFQ
    • LLQ queues will always be serviced first independent of all other queues/classes
    • When multiple priority queues are defined, all priority queue packets will be served FIFO-based
    • The configured priority value is the maximum bandwidth of the queue itself (= policing) if there’s congestion on the link
    • Just like CBWFQ, if there’s no congestion on the link the LLQ can use more bandwidth than it has configured
    • Cisco recommendation: Links with mixed traffic (voice, video, data) should never exceed 33% of the total interface bandwidth for all LLQ combined!
• Drop strategies (congestion avoidance):
  • Tail Drop:
    • Default drop behavior
    • Allows a queue to fill up until it is full and then discard any packets until there’s space again
    • Leads to TCP starvation/UDP dominance because when congestions occurs, TCP tends to temporarily reduce its window size and therefor slow down the transmission rate whereas UDP doesn’t
  • WRED (Weighted Random Early Detection):
    • Randomly drops packets in a flow if a link is approaching congestion (based on current queue and QoS)
    • Packets with a higher weight (= higher IP Precedence or DSCP value) are less likely to be dropped
    • Goal is to send individual senders into slow start instead of letting global synchronization occur
    • Not configurable on LLQs

“L3 QoS congestion management/avoidance” CLI configuration commands:

## Configuring flow-based WFQ
Router(config-pmap-c)# fair-queue

## Configuring class-based WFQ
Router(config-pmap-c)# bandwidth [arguments]

## Configuring LLQ
Router(config-pmap-c)# priority [arguments]

## Enabling WRED
Router(config-pmap-c)# random-detect [arguments]

4.4.a viii HQoS, Sub-rate Ethernet Link

HQoS

Hierarchical Quality of Service

General information on “L3 HQoS”:

• Also called nested policies
• Consists of two policy maps:
  • Child policy map: Marks all the classified traffic, assigning bandwidth, …
  • Parent policy map: Assigns overall bandwidth, …, and links to the child policy map
• Example use case:
  • PING, HTTP and SSH each need their own marking
  • PING gets a minimum gtd. bandwidth of 50%, HTTP 30% and SSH 19%
  • PING gets marked as IP Precedence 7, HTTP as IP Precedence 5 and SSH as IP Precedence 3
  • Additionally the interface bandwidth needs to be shaped to 250Mbps

Configuration steps:

1. Configure class-maps matching the protocols which later need to be marked
2. Configure a child policy-map with the classes created previously and mark the traffic, set the bandwidth, …
3. Configure a parent-policy map with class-default linking to the previously created child policy-map
4. Apply policy-map outbound on an interface

“L3 HQoS” CLI configuration commands:

## L3 HQoS example configuration based on the above use case
Router(config)# class-map match-any CLASS_PING
Router(config-cmap)# match protocol ping

Router(config)# class-map match-any CLASS_HTTP
Router(config-cmap)# match protocol http

Router(config)# class-map match-any CLASS_SSH
Router(config-cmap)# match protocol ssh

Router(config)# policy-map CHILD_MARK_PING_HTTP_SSH
Router(config-pmap)# class CLASS_PING
Router(config-pmap-c)# bandwidth percent 50
Router(config-pmap-c)# set ip precedence 7

Router(config-pmap)# class CLASS_HTTP
Router(config-pmap-c)# bandwidth percent 30
Router(config-pmap-c)# set ip precedence 5

Router(config-pmap)# class CLASS_PING
Router(config-pmap-c)# bandwidth percent 19
Router(config-pmap-c)# set ip precedence 3

Router(config)# policy-map PARENT_MARKING
Router(config-pmap)# class class-default
Router(config-pmap-c)# shape average 250m
Router(config-pmap-c)# service-policy CHILD_MARK_PING_HTTP_SSH

Sub-rate Ethernet Link

General information on “Sub-rate Ethernet Link”:

• A sub-rate ethernet link exists, when the provisioned circuit is below the speed of the physical interface
• Example: The physical interface of the CE/PE router has a bandwidth of 1G. The provisioned circuit between the PE and CE is 250Mbps. On the PE side this is most commonly implemented via policing which drops all packets exceeding the CIR. On the CE side this should be implemented via shaping or else (massive) packet drops could occur when the traffic exceeds the CIR on the PE ingress interface.