Conversation
Adds screen-space motion vectors (per-pixel Δpixels between the current
and previous frame's projected position of the same world point). This
is the essential input for temporal upsamplers like DLSS, FSR2, and
XeSS. Without it, applications must approximate motion vectors from a
full-screen camera-only depth reprojection, which cannot represent
per-object motion and causes visible ghosting on moving geometry.
New public API:
* BN_FB_MOTION channel enum (vec2f per pixel, screen-space pixel delta)
* bnSetInstanceMotionDeltas(model, slot, BNTransform*, n) - per-instance
motion delta transform, defined as prev * inverse(curr). Optional;
when unset, only camera motion contributes.
* ANARI parameters on Camera: motion.viewProjection and
motion.previousViewProjection (both ANARI_FLOAT32_MAT4). Both must
be supplied for the motion channel to be written.
* ANARI parameter on Instance: motion.transform (ANARI_FLOAT32_MAT4,
the instance's transform on the previous frame). Optional per
instance; defaults to the current transform.
* ANARI framebuffer channel: channel.motion (ANARI_FLOAT32_VEC2).
Design choices:
* Delta form (prev * inv(curr)) rather than raw prev transform,
computed on the ANARI shim side. Barney has no device-side
per-instance current-transform array (transforms live inside the
RTC/OptiX BVH), so shading with raw prev transforms would require
inverting the current transform per pixel in the hot ray shader
path. Delta form is one CPU inverse per instance per frame in
exchange for zero inverses in the kernel.
* Screen-space pixels rather than world-space velocity: matches
DLSS/FSR2/XeSS conventions and DirectX/Vulkan raster MV output.
Zero downstream reprojection needed.
* BN_FB_MOTION added to FrameBuffer::needHitIDs() mask - the shade
kernel reads hitIDs[tid].instID to look up per-instance motion
deltas, and without this the hitIDs buffer is never populated for
motion-only channel requests.
Shading:
In shadeRays.cu, at the accumID==0 && generation==0 aux-write block:
world_hit = ray.P
prev_world = motionDeltas[instID] * world_hit (if deltas supplied)
delta_ndc = 0.5 * (project(curr_vp, world_hit) -
project(prev_vp, prev_world))
motion = delta_ndc * fbSize (in pixels)
Guarded by camera.haveMotionMatrices - falls back to no-op if the app
did not supply the motion.viewProjection matrices.
Framebuffer plumbing:
* MotionChannelTile (vec2f per pixel) parallel to AuxChannelTile
* TiledFB: alloc + linearize kernel (linearizeMotionTiles)
* FrameBuffer: linearMotionChannel staging + gatherMotionChannel +
writeMotionChannel virtuals; MPI-gather runs in finalizeFrame() so
send/recv pair correctly across all ranks
* DistFB: MPI send/recv path templated on MotionChannelTile
* LocalFB: single-node overrides
resetAccumulation() extended to zero the motion tiles on every device
at the start of each frame - the shade kernel writes only on hits, so
unwritten pixels (background) would otherwise retain stale motion
values from previous frames, causing 'ghost silhouette' artifacts.
Backward compatibility: all changes are additive. Existing apps that
don't request BN_FB_MOTION are unaffected. Existing sample apps
verified to produce identical output.
Testing: full end-to-end verified on three ANARI examples with correct
motion vectors driving NVIDIA DLSS temporal upscaling. Motion vector
corruption tests (zero / negate / exaggerate / noise) confirm DLSS
consumes the channel and its output measurably degrades on corruption.
|
i'll merge that into my rework branch asap; there's conflicts all over the place because rework branch has changed namespaces and file locations; but most of these should be easy to merge over. only concern of testing this, so once i get this into a non-conflicted form i'll ping you to run some test before final merge |
|
Sounds good, thank you! |
|
Update: done merging this in my local fork; was a lot of manual copy-n-pasting (for many files git didn't figure out the file move/renames so had to copy manually), but I'm reasonably confident I got it all - though as said I have nothing for functional correctness tests. I'll run some functional tests on my own testcases (to make sure it didn't break anything), then merge back to nvidia/main later today or tomorrow if that runs through. PS: that was a huge diff :-). At some point you need to tell us (offline) what you're actually doing with this :-) |
|
merged in via #57 (had a lot of conflicts that required local merge). Merged variant still works fine with existing test suite, but since this test suite doesn't contain anything to test this specific MR's features it may of course have broken something. If so please submit any additional diffs as new MR - this branch will only create conflicts, so I'll close it out. |
Summary
Adds a
BN_FB_MOTIONframebuffer channel producing per-pixel screen-spacemotion vectors (Δpixels between the current and previous frame's projected
position of the same world point), plus the ANARI-side parameters to drive it.
This is the input temporal upsamplers require. Without it, applications must
approximate motion vectors from a full-screen camera-only depth reprojection,
which cannot represent per-object motion and produces visible ghosting on
moving geometry.
All changes are additive: apps that never request
BN_FB_MOTIONareunaffected, and existing sample apps were verified to produce identical output.
New public API
Barney C API
BN_FB_MOTIONchannel enum —vec2fper pixel, screen-space pixel deltabnSetInstanceMotionDeltas(model, slot, BNTransform *deltas, n)—per-instance motion delta, defined as
prev * inverse(curr). Optional;when unset only camera motion contributes.
ANARI
Camera:motion.viewProjection,motion.previousViewProjection(
ANARI_FLOAT32_MAT4). Both must be supplied for the channel to be written.Instance:motion.transform(ANARI_FLOAT32_MAT4) — the instance'stransform on the previous frame. Optional; defaults to the current transform.
Frame:channel.motion(ANARI_FLOAT32_VEC2).Design choices
Delta form (
prev * inv(curr)) rather than raw previous transform.Barney has no device-side per-instance current-transform array — transforms
live inside the RTC/OptiX BVH — so shading with raw previous transforms would
require inverting the current transform per pixel in the hot ray shader. The
delta form costs one CPU inverse per instance per frame and zero inverses in
the kernel. The inversion is done on the ANARI shim side.
Screen-space pixels rather than world-space velocity.
Matches the DLSS / FSR2 / XeSS convention and DirectX/Vulkan raster MV output,
so no downstream reprojection is needed.
BN_FB_MOTIONadded toFrameBuffer::needHitIDs().The shade kernel reads
hitIDs[tid].instIDto look up per-instance deltas;without this the hitIDs buffer is never populated for motion-only requests.