feat: add FreeBSD support - #2
Conversation
0.3.10 taught --device to bypass the class filter. The check reads
props.get("bdf"), but the scan loop stores the address under "slot",
the key lspci prints. The rename to bdf happens later, in
Device.from_dict. The lookup always came back empty. --device matched
nothing at all, not even NVMe devices that used to match. The command
still exited 0, so the failure was silent.
Read the slot key, and add a regression test for the bypass.
Signed-off-by: Jaeyoon Choi <j_yoon.choi@samsung.com>
device_scan, bind, and unbind read the argparse namespace directly. That coupled them to the CLI. unbind never used args at all. bind took it only to pass it along. Tests could not call the helpers without building a fake args object. Take classcode, bdf, and driver_name as plain parameters instead. main() passes the parsed values at the call sites. This keeps the upcoming Backend interface free of argparse. Behavior is unchanged. Signed-off-by: Jaeyoon Choi <j_yoon.choi@samsung.com>
Prepare for adding other platforms. The module-level scan, bind, unbind, and probe helpers move onto a LinuxBackend class behind a small abstract Backend interface. get_backend() picks the implementation for the running platform. System.drivers/limits become instance state filled in from the backend. The memlock remediation text becomes a backend hint, assembled into the same warning. The parse-time driver vocabulary splits off as KNOWN_DRIVERS, and Device.MANDATORY_KEYS becomes _LSPCI_KEYS next to the lspci parser. The tool and sysfs notes from the file header move onto LinuxBackend as its docstring. Output and behavior on Linux are unchanged. scan_devices() keeps the --device bypass of 0.3.10. Add Linux backend unit tests: lspci parsing and filtering, the --device bypass, the driver_override -> bind -> setpci sequence, and backend selection. Signed-off-by: Jaeyoon Choi <j_yoon.choi@samsung.com>
An unsupported platform, a --bind name the platform does not have, and a failing bind or unbind all ended in a Python traceback. The driver-name case gets worse once more platforms exist. A name owned by another platform would reach the device before failing. Catch the backend errors in main() and exit with a one-line error. Validate --bind against the active backend's driver_names() before touching the device. Signed-off-by: Jaeyoon Choi <j_yoon.choi@samsung.com>
Implement the Backend interface with FreeBSD's native tools: - pciconf -l enumerates devices. Both output formats are parsed: the packed chip=/card= fields of 14.3 and earlier, and the split vendor=/device= fields of 15.x and 14.4. - devctl detach / devctl set driver implement unbind and bind. - camcontrol devlist maps an nvmeX controller to its CAM disk (ndaY), so the fstat in-use check also sees users of the disk device. - kldstat reports whether nic_uio is loaded. - scan_devices() honors the --device class-filter bypass, matching the Linux backend. - pciconf -w enables bus-mastering after binding to nic_uio. A failed write is reported instead of ignored. Helpers convert between the pciX:B:S:F selector and the domain:bus:device.function bdf form. nic_uio is added to KNOWN_DRIVERS, completion, and the docs. User-facing text drops its Linux assumptions: the --bind help says driver file instead of .ko file, and the memlock warning says DMA mapping instead of VFIO_IOMMU_MAP_DMA. Unit tests cover both pciconf formats, the fstat heuristic, kldstat probing, camcontrol parsing, and the devctl bind sequence. Signed-off-by: Jaeyoon Choi <j_yoon.choi@samsung.com>
|
This brings us halfway to drop From SPDK function _configure_freebsd() {
local freebsd_bufsz=${FREEBSD_BUFSZ:-256}
if ! check_for_driver_freebsd; then
echo "DPDK drivers (contigmem and/or nic_uio) are missing, aborting" >&2
return 1
fi
configure_freebsd_pci "$@"
if [[ $SKIP_HUGE == yes ]]; then
# Do nothing as requested
return 0
fi
# If contigmem is already loaded but the HUGEMEM specified doesn't match the
# previous value, unload contigmem so that we can reload with the new value.
if kldstat -q -m contigmem; then
# contigmem may be loaded, but the kernel environment doesn't have to
# be necessarily set at this point. If it isn't, kenv will fail to
# pick up the hw. options. Handle it.
if ! contigmem_num_buffers=$(kenv hw.contigmem.num_buffers); then
contigmem_num_buffers=-1
fi 2> /dev/null
if ((contigmem_num_buffers != HUGEMEM / freebsd_bufsz)); then
kldunload contigmem.ko
fi
fi
if ! kldstat -q -m contigmem; then
kenv hw.contigmem.num_buffers=$((HUGEMEM / freebsd_bufsz))
kenv hw.contigmem.buffer_size=$((freebsd_bufsz * 1024 * 1024))
kldload contigmem.ko
fi
}@safl Do you think we should put the |
karlowich
left a comment
There was a problem hiding this comment.
Great.
I propose adding the test_linux.py in a later commit, to avoid changing the test in 3 consecutive commits.
Also see the few inline comments.
| handles: list = field(default_factory=list) | ||
|
|
||
|
|
||
| class Backend(abc.ABC): |
There was a problem hiding this comment.
I think this should be called platform instead. That puts it more in line with the doc-string: "Platform-specific PCI device-driver binding operations"
| DRIVERS: set = set() | ||
|
|
||
| def driver_names(self) -> set: | ||
| """Return the set of driver-names this backend can bind to""" |
There was a problem hiding this comment.
Let's rename backend to platform here too.
|
|
||
| class System: | ||
| DRIVERS = {"nvme", "vfio-pci", "vfio-noiommu", "uio_pci_generic"} | ||
| class LinuxBackend(Backend): |
There was a problem hiding this comment.
if we rename from backend to platform, then this class could just be named Linux
|
|
||
| def unbind(device: Device): | ||
| log.info(f"Unbinding({device.bdf}) from '{device.driver}'") | ||
| def get_backend() -> Backend: |
Yeah, separate tool. Since, the main thing about the suite of tools: |
Summary
Add FreeBSD support to devbind. A backend picked at runtime maps the same
CLI onto each platform's native tooling, so one command prepares an NVMe
device for xNVMe on both Linux and FreeBSD. Verified
end-to-end on FreeBSD 15.1 and 14.4 VMs with an emulated NVMe device, and
re-verified on Linux the same way. The branch is structured for per-commit
review; every commit passes the test suite on its own.
Details
xNVMe support FreeBSD, but preparing a device there — loading
nic_uio, runningdevctl detachanddevctl set driver, enablingbus-mastering by hand — was manual. devbind covered only Linux, with the
sysfs paths hardcoded in the tool.
The Linux code now sits behind a small
Backendinterface, and a FreeBSDimplementation joins it:
lspci, sysfspciconf -lunbind/binddevctl detach/devctl set driverlsoffstat+camcontrol(nvmeX → ndaY)vfio-pci,uio_pci_genericnic_uiosetpcipciconf -wFreeBSD has no VFIO, so
nic_uio(theuio_pci_genericanalogue) is theuserspace path there, and
iommugroupreportsNone. Naming a driver therunning platform does not have is rejected before the device is touched.
The five commits, bottom to top:
fix: the 0.3.10 --device lookup read an unmapped key and matchednothing
refactor: decouple scanning and binding from argparserefactor: move the Linux code onto aLinuxBackendclass. A puremove. Output on Linux is unchanged. Easiest to read with
--color-moved.fix: backend errors exit with a one-line message instead of atraceback
feat: the FreeBSD backend, unit tests for both platforms, and docsTwo notes for review:
pciconf -loutput differs by release. 14.3 and earlier print packedchip=/card=fields. 15.x and 14.4 print splitvendor=/device=fields. Both are parsed, and the unit fixtures cover both.
via
camcontrol devlist. Anyone holding the disk open blocksunbinding. Legacy nvd(4) disks do not attach via CAM, so the check does
not see them. The code documents this limit.