Conversation
troglobit
force-pushed
the
qos
branch
16 times, most recently
from
September 14, 2026 13:52
e0e9dbb to
43b19d0
Compare
The new upcoming QoS model needs num-traffic-class-type, added to ieee802-dot1q-types by IEEE Std 802.1Qcw-2023. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Per-interface transmission selection is rendered as tc mqprio when the driver offloads it and tc ets otherwise. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Add a qos container to every interface, with names and types from IEEE 802.1Q-2022 wherever the standard defines the concept. The defaults are a complete pipeline, trust DSCP then PCP and one traffic class per transmit queue per Table 8-5, rendered on every physical port from boot, which retires the 25-mqprio script. Ingress classification is programmed through the kernel's DCB APP table and apptrust list. Few drivers have those operations, so when the calls fail the same maps are rendered as 'tc flower' rules on the port's ingress, in trust order, setting the skb priority; on a NIC or SoC MAC that covers all traffic, on a switch the CPU path. The orders a driver accepts cannot be queried, so a table keyed by driver name reports them as a capability. The traffic class table is programmed as 'tc mqprio' with 'hw 1', else as 'tc ets' in the kernel, which numbers bands the opposite way so the map, quanta and strict count invert. A single-queue port has no queue structure to respect and gets the kernel's eight classes; the root qdisc tells where it ended up and is reported as the offload capability. Strict classes must be the highest-numbered ones, the only layout ets can express. Egress remarking uses the same DCB table via 'dcb rewr'; without it DSCP is rewritten with 'tc pedit' on egress, PCP has no software counterpart since act_vlan cannot change the priority alone. Policing and storm control are omitted for now. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
A 4xx or 5xx from rousette carries the offending node and reason in the response body, which the exception message drops. Print it before raising. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The QoS model has no test coverage. The tests run on any target: the software rendering is verified where mqprio and DCB are not offloaded, the hardware tables where they are, and DSCP classification end to end through a routed VLAN interface. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Update QoS page, which so far has covered only the Marvell hardware defaults and the hard-coded VLAN PCP mapping. Rewrite it around the new infix-if-qos.yang model, with the IEEE 802.1Q Table 8-5 factory default, the Table 34-1 layout for reserved streams and what offload capabilities mean on hardware with and without DCB support. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Figures are centered on the page, tables were left-aligned. The old .center-table class was never used; a rule on Material's table wrapper centers every table without markup. Material also gives every header cell a 5rem minimum width, so a nine column table of single digits overflows the page. Let headers size to their content. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Material's 1.4 line height leaves visible gaps between box-drawing characters in CLI output and diagrams. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
troglobit
force-pushed
the
qos
branch
5 times, most recently
from
September 18, 2026 12:35
778b006 to
469527d
Compare
The mv88e6xxx driver has no DCB ops, so on 88E6390 and 88E6393X boards
the trust order, priority maps and remarking only reached frames that
passed the CPU. Frames the switch forwarded port to port were
classified by the chip defaults and never remarked.
Add DSA support for the PCP APP selector and the DCB rewrite table, and
mv88e6xxx ops for the per-port PCP and DSCP tables, InitialPri/TagIfBoth
as trust order, the default priority, and the egress PCP and DSCP
tables. The mqprio map is pushed into the tables as queue priority, so
the traffic class table applies to forwarded frames too. The scheduler
stays at the chip default WRR.
Frames the CPU sends bypass the switch tables, so their DSCP is still
remarked by the kernel on DSA ports. Frames the CPU receives carried a
stale priority: tag_dsa read the DSA tag after stripping it, picking up
a byte of the source MAC address instead, which only showed once the
software classifier no longer overwrote it.
The renderer and the operational datastore now treat the driver as DCB
capable with all four trust orders:
dcb apptrust set dev e1 order dscp pcp
dcb app add dev e1 pcp-prio 5nd:5 dscp-prio 46:5 default-prio 2
dcb rewr add dev e2 prio-pcp 5:5nd prio-dscp 5:40
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Bridge two ports and send frames through the switch, so on a switch chip they never pass the CPU and only the hardware tables can classify and remark them. The DSCP and PCP on the wire at the egress port reveal the priority the ingress port assigned, with trust dscp on untagged frames, trust pcp with a custom map on tagged frames, and remarking off. PCP is only checked where the driver offloads remarking, there is no software path for it. Skipped on a switch port without DCB support, since forwarded frames never see the kernel's rules there. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Transmission selection was rendered as 'tc mqprio' with 'hw 1' on every port, falling back to 'tc ets' only when the driver refused it. mqprio carries the priority to traffic class map and nothing else, so a fabric that offloads both, like Microchip sparx5 and lan966x, was given the map and never the class algorithms or weights. mqprio also never marks itself offloaded, which left the operational data guessing from its presence. Render 'tc ets' everywhere. It carries the whole of transmission selection, drivers that offload it report so in the qdisc, and the software scheduler is the same qdisc. Marvell LinkStreet, which offloads the map through mqprio and nothing through ets, keeps mqprio until its driver learns ets. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Nothing in the model could hold a port below its link speed, which a
virtual port has none of, so there was no way to put the traffic
classes under load there and see the scheduler work. The design has
the node for it, one token bucket on the whole port.
Add /interfaces/interface/qos/egress/rate-limit with a rate in bits per
second and an optional burst, rendered as a 'tc tbf' root with the ets
scheduler beneath it:
tc qdisc add dev e1 root handle 1: tbf rate 10000000bit burst 12500 latency 100ms
tc qdisc add dev e1 parent 1:1 handle 2: ets bands 8 strict 8 priomap ...
An unset burst covers ten milliseconds at rate, never less than one
frame. mqprio can only be the root, so a rate limited port schedules
with ets. The operational offload list gains 'rate-limit', reported
when the driver marks the tbf offloaded. NET_SCH_TBF is enabled on all
boards.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The QoS scheduling tests need traffic the tests do not have to write themselves: sustained UDP flows with a set rate and DSCP, counted at the receiver, and single frames with an exact VLAN tag and PCP. iperf3 comes from Alpine. Alpine has no netsniff-ng package on any branch, and mausezahn needs libcli, which only exists in edge/testing, so both are built from their release tarballs like mtools already is. netsniff-ng's configure is a bash script and its install rules need GNU install, hence bash and coreutils. Image version 2.12. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The QoS tests so far check the traffic class table as the tc arguments the daemon rendered, never the queue a frame reached. A mistake of convention, the band inversion or the priority 0 and 1 swap in Table 8-5, would render as expected and queue wrong. Send ten tagged frames at each priority with mausezahn, priority taken straight from the PCP by the ingress VLAN interface so no classifier is involved, route them out the port under test, and read the scheduler's per-class counters before and after. Run for the ieee and ieee-sr presets and a custom table. The counters are read from 'tc -s class show', which counts the CPU path on every rig. infamy/qos.py collects what the QoS tests share: the IEEE tables, capabilities, the scheduler qdisc under an optional tbf root, per-class counters in 802.1Q class numbering for both ets and mqprio, and the traffic generator wrappers. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The QoS tests so far verify how the scheduler is configured, not how it behaves. Nothing shows that a strict priority class actually gets the port when two flows contend for it. Bridge two ports, trust DSCP on the ingress side, and slow the egress port so its queues fill: 100 Mbit/s where the PHY can negotiate down, a 10 Mbit/s rate limit otherwise. Two iperf3 UDP flows, EF and CS1, each offer 80 percent of what the port drains. In one class they split the port evenly. With EF in a strict class above CS1, EF passes without loss while CS1 takes the rest, and EF at 120 percent of the port leaves CS1 with less than 5 percent. Loss is counted as datagrams sent minus received, since the iperf3 server undercounts loss under tail drop. Priority 0 is pinned to the top class so the test's own control traffic survives the contest. Skipped on a switch fabric whose scheduler is not offloaded, where forwarded frames never meet the configured algorithm. The flow helpers, port slowing, and PMD type lookup live in infamy/qos.py for the weighted sharing and mixed selection tests that follow. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Nothing verifies that two enhanced transmission selection classes share the port in their configured ratio, or that the shares reach the right bands: the strict priority test only shows the top class winning. Bridge two ports as the strict priority test does, put EF and CS1 in the two lowest classes, both weighted, and offer 80 percent of the port from each so both classes always have a backlog. Equal weights must split the port evenly, which fails on its own if the shares land on the wrong bands, and 67:33 must move the split to match. Each share must land within five points of its percentage and both flows must lose frames. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The strict priority and weighted sharing tests each exercise one transmission selection algorithm. A layout with a strict class above weighted ones, the common shape for a control channel over shared bulk traffic, has no test. Bridge two ports as the other scheduling tests do, put EF in a strict class above AF21 and CS1 sharing 67:33, and run all three at once: EF at 40 percent of the port, the weighted pair at 60 percent each. EF must arrive intact, and AF21 and CS1 must both lose frames and divide what EF leaves in their configured ratio, within five points. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
…inkStreet The LinkStreet patches offload classification, remarking and the class map, but the scheduler stays at the chip's power-on weighted round robin whatever the traffic-class list says, and a rate limit shapes only what the CPU sends. Two more mv88e6xxx patches take the ets and tbf qdiscs: strict classes become the port's strict queues, weighted shares become the chip's round robin weights, and the rate limit becomes the port's egress shaper. The weights are one set per chip, so ports with weighted classes must agree on their shares; a port that differs stays in software and says so in its offload list. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The renderer kept mqprio for the one fabric that offloaded its class map through it and nothing through ets. Its driver now takes ets, so the exception can go, along with the imx8mp quirk that disabled it: every port gets the same ets rendering, and the operational offload list follows the qdisc's offloaded flag alone. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
No port renders mqprio any more. The traffic class test expects the offload capability to follow the ets qdisc's offloaded flag, and the scheduling tests skip a switch port whose driver did not take the scheduler, since the fabric forwards past a software one. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The traffic class test checks that a rate limit renders a tbf root, not that the port drains at the configured rate, and the scheduling tests lean on that limit to fill the port without proving it holds. Bridge two ports, limit the egress port to 10 Mbit/s and offer three times that from one talker: what arrives must be the limit within a fifth, once marked EF in a strict class, once marked CS1 in the lowest class, and once with both talkers at one and a half times the limit. A limit that only bites on some queues is a scheduler fault dressed up as one. Skipped on a switch fabric whose driver does not offload the limit, where forwarded frames never meet the bucket. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
WIP
Checklist
Tick relevant boxes, this PR is-a or has-a: