rmapi: keep kernel clients' user BAR1 mappings in one range - #1403
Open
SammyTourani wants to merge 1 commit into
Open
SammyTourani wants to merge 1 commit into
SammyTourani wants to merge 1 commit into
Conversation
Since 565.57.01, memMap_IMPL() sets BUS_MAP_FB_FLAGS_ALLOW_DISCONTIG for every mapping with bKernel == NV_FALSE. That includes a kernel client asking for an NVOS33_FLAGS_MEM_SPACE_USER mapping, which is what nvkms MapMemory(NVKMS_KAPI_MAPPING_TYPE_USER) does for nvidia-drm. When BAR1 is too fragmented for one range, kbusMapFbAperture_GM107() splits the mapping. osMapPciMemoryAreaUser() returns only the start of the first range, and rm_kernel_rmapi_op() frees the range list. nvidia-drm then ioremaps and inserts PFNs for the whole object from that address. The pages past the first range belong to other BAR1 mappings, unmapped BAR1 space or the next PCI BAR. Only allow a discontiguous mapping when it is not a kernel client's MEM_SPACE_USER mapping. Such a mapping then fails with NV_ERR_NO_MEMORY when BAR1 has no single range free. User clients still get discontiguous mappings, because nv-mmap maps every range. With static BAR1, the same flag let kbusGetStaticFbAperture_TU102() return one range per physical chunk, with the same result in nvidia-drm. For physically discontiguous memory such a mapping now fails with NV_ERR_INVALID_ARGUMENT, as a kernel mapping of it already does.
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1134
One condition in
memMap_IMPL()— only setALLOW_DISCONTIGwhen the mapping isneither a kernel client's mapping nor a MEM_SPACE_USER mapping:
Real userspace clients still get discontiguous mappings (they always have
MEM_SPACE == CLIENT and nv-mmap maps every range). nvkms's user mapping no longer
does, so on a fragmented/full BAR1 it fails cleanly with
NV_ERR_NO_MEMORYinstead of composing a mapping nvidia-drm mis-treats as contiguous. nvidia-drm
already handles that failure ("Failed to map NvKmsKapiMemory", -ENOMEM) — that is
the survivable path several reporters saw before the fatal escalation.
The single-range attempt is tried first regardless of the flag, and BAR1 mapping
reuse does not depend on it, so the only behaviour removed is the multi-range
fallback for the nvkms path. Scope: 1 file, +11/-1 (the change is one
if; therest is a comment).
Verification
No GPU or kernel build on this machine, so I verified in a userspace ASan harness
that compiles the real RM sources unchanged (
mapping_cpu.c,kern_bus_gm107.c,kern_bus_tu102.c,mapping_reuse.c,containers/map.c,os.c) and drives the realserverMap_Prologue->memMap_IMPL->kbusMapFbAperture->reusemappingdbMap->osMapPciMemoryAreaUserpath, thenreplays the tail of
rm_kernel_rmapi_op(NV04_MAP_MEMORY)verbatim and checks,page by page, what nvidia-drm (contiguous, first-range-only) vs nv-mmap
(every-range) would actually reach through the returned address. Scenario A uses
the reporter's exact BAR1 layout; F–I cover static BAR1 (Turing+ large-BAR path,
real
kbusGetStaticFbAperture_TU102).Command:
Result (real output):
Base: 6/9 (A, B nvkms-fragmented and G static-discontiguous compose cross-range
mappings). Fixed: 9/9 — those three fail cleanly with -ENOMEM; the contiguous,
reuse, and real-userspace scenarios (C, D, E, F, H, I) are unchanged and still
map every page correctly. The harness reproduces the reporter's exact
resource sanity checkrange on base.Also compiled the changed file for real x86_64 Linux with the exact flags from
make -n -C src/nvidia TARGET_OS=Linux TARGET_ARCH=x86_64(which includes-Werror-implicit-function-declaration), base and fixed: both 0 warnings / 0errors, identical compiler output, valid ELF x86-64 object.