Hello! I am trying to use TensorRT-RTX but I have a very interesting bug.
NOTE: TensorRT 11 is fine, works perfectly with zero problems, only TensorRT-RTX 1.5 hangs randomly. I DO want to use the RTX version because it is more interesting: Portable engines, much lower binary size are important for me. Anyway the problem is this:
A RIFE v4.6 engine built with TensorRT for RTX 1.5.0.114 hangs forever inside enqueueV3 at the input shape 1x11x576x704, but only when the GPU is allowed to go idle between consecutive enqueues! The hang is reproducible with tensorrt_rtx alone (no application code) by adding --idleTime.
The same ONNX, same shape, same idle pattern, same GPU runs good on mainline TensorRT 11.1.0.106, which is I believe this is an RTX-specific problem.
rife_v4.6_fp16.onnx.zip
NOTE: This is the fp16 version of the model which I AutoCast (but kept some "fp32 islands" with surgeries), the original FP32 model can be found here
rife_v4.6.onnx.zip
When it hangs, the GPU sits at 100% utilization but only ~39 W, with 0-1% memory utilization and one CPU core spinning at 100%.
Real inference on this engine draws 150-220 W normally. I left it in this state for 4.5 hours (well, I slept expecting it to complete in an hour 😅) with zero progress.
Environment
|
|
| GPU |
NVIDIA GeForce RTX 4070 SUPER (sm_89) |
| Driver |
610.52 |
| OS |
Windows 11 (10.0.26200) |
| TensorRT for RTX |
1.5.0.114 (reports TensorRT v10500, build b114) |
| TensorRT |
TensorRT 11.1.0.106 (v110100, build b106) |
| Model | rife_v4.6 ONNX (fp16, opset 16), from vs-mlrt external-models. 11-channel input [1,11,H,W] |
Reproduction
Build the engine (dynamic shapes, these are the ones I use):
tensorrt_rtx --onnx=rife_v4.6_fp16.onnx --saveEngine=rife_v4.6_fp16.trt \
--minShapes=input:1x11x128x128 \
--optShapes=input:1x11x1088x1920 \
--maxShapes=input:1x11x2176x3840Run it at 576x704 with an idle gap between enqueues:
# HANGS: never terminates, must be killed
tensorrt_rtx --loadEngine=rife_v4.6_fp16.trt --shapes=input:1x11x576x704 \
--iterations=800 --duration=0 --idleTime=30Expected: completes 800 iterations (roughly 1.3 ms of compute each) and reports PASSED.
Actual: stops making progress after a few hundred iterations and hangs indefinitely. The last log line is [I] Starting inference.
The same command without the idle gap passes:
# PASSES (781 qps, exit 0). The GPU never drains between enqueues.
tensorrt_rtx --loadEngine=rife_v4.6_fp16.trt --shapes=input:1x11x576x704 \
--iterations=300 --duration=0 --idleTime=0The hang is random
It happens after roughly 1 in 600 to 1500 enqueues.
Two consecutive runs of the identical command (--iterations=1500 --idleTime=30) gave HANG then ok.
A single passing run therefore proves nothing. I recommend at least 8000 enqueues per configuration when verifying.
The original FP32 version may hang MUCH later but it WILL hang, and it may also PASS sometimes. generally this is very random! But IF it happens then the GPU is stuck!
What we ruled out (all measured)
| Configuration |
Idle gap |
Result |
TRT-RTX, v4.6 @ 1x11x576x704 |
10-30 ms |
HANG (also hangs at 8000 iterations) |
TRT-RTX, v4.6 @ 1x11x576x704 |
none (back-to-back) |
ok, 781 qps |
mainline TRT 11.1, v4.6 @ 1x11x576x704, identical ONNX |
10 ms |
ok, 16,000 enqueues, twice |
TRT-RTX, v4.6, engine built with optShapes=1x11x576x704 |
10 ms |
HANG (so it is NOT a bad opt shape issue!) |
TRT-RTX, rife v4.26 (different graph, 7-channel) @ 1x7x576x704 |
10 ms |
ok |
| TRT-RTX, v4.6 @ various other shapes |
10 ms |
ok |
So the trigger is very narrow: this one input shape on this graph (of course there may be many more), plus an idle GPU between enqueues.
Neighbouring shapes (including the same height at other widths, and the same width at other heights) are good.
Additionally, from the application side (before we reduced it to trtexec), we ruled out:
- Not our memory. The TRT workspace and all application buffers were verified non-overlapping.
- Not a race with application kernels. A full
cudaStreamSynchronize immediately before enqueueV3 (empty stream, cudaSuccess) still hangs. The stuck kernel belongs to the engine.
- Not out-of-bounds access.
compute-sanitizer memcheck reports zero errors.
Idle gap is important!
This is not a synthetic condition.
In our video pipeline, RIFE enqueues are separated by an idle GPU whenever a CPU-side stage is the bottleneck (software video decode, or an x265 CPU encode applying backpressure).
Under a saturated GPU pipeline (hardware decode and encode) the same shape runs fine indefinitely, which is why this only surfaced on one code path!!
Impact
Any application feeding this engine a 576x704 input on a non-saturated GPU hangs permanently, with no error, no exception, and no timeout.
It cannot be caught or recovered from in-process!
Our workaround, avoiding the shape, is not something an application can do in general, since the "bad" shape is not predictable.
Hello! I am trying to use TensorRT-RTX but I have a very interesting bug.
NOTE: TensorRT 11 is fine, works perfectly with zero problems, only TensorRT-RTX 1.5 hangs randomly. I DO want to use the RTX version because it is more interesting: Portable engines, much lower binary size are important for me. Anyway the problem is this:
A RIFE v4.6 engine built with TensorRT for RTX 1.5.0.114 hangs forever inside
enqueueV3at the input shape1x11x576x704, but only when the GPU is allowed to go idle between consecutive enqueues! The hang is reproducible withtensorrt_rtxalone (no application code) by adding--idleTime.The same ONNX, same shape, same idle pattern, same GPU runs good on mainline TensorRT 11.1.0.106, which is I believe this is an RTX-specific problem.
rife_v4.6_fp16.onnx.zip
NOTE: This is the fp16 version of the model which I AutoCast (but kept some "fp32 islands" with surgeries), the original FP32 model can be found here
rife_v4.6.onnx.zip
When it hangs, the GPU sits at 100% utilization but only ~39 W, with 0-1% memory utilization and one CPU core spinning at 100%.
Real inference on this engine draws 150-220 W normally. I left it in this state for 4.5 hours (well, I slept expecting it to complete in an hour 😅) with zero progress.
Environment
TensorRT v10500, buildb114)v110100, buildb106)| Model |
rife_v4.6ONNX (fp16, opset 16), from vs-mlrt external-models. 11-channel input[1,11,H,W]|Reproduction
Build the engine (dynamic shapes, these are the ones I use):
tensorrt_rtx --onnx=rife_v4.6_fp16.onnx --saveEngine=rife_v4.6_fp16.trt \ --minShapes=input:1x11x128x128 \ --optShapes=input:1x11x1088x1920 \ --maxShapes=input:1x11x2176x3840Run it at
576x704with an idle gap between enqueues:# HANGS: never terminates, must be killed tensorrt_rtx --loadEngine=rife_v4.6_fp16.trt --shapes=input:1x11x576x704 \ --iterations=800 --duration=0 --idleTime=30Expected: completes 800 iterations (roughly 1.3 ms of compute each) and reports PASSED.
Actual: stops making progress after a few hundred iterations and hangs indefinitely. The last log line is
[I] Starting inference.The same command without the idle gap passes:
# PASSES (781 qps, exit 0). The GPU never drains between enqueues. tensorrt_rtx --loadEngine=rife_v4.6_fp16.trt --shapes=input:1x11x576x704 \ --iterations=300 --duration=0 --idleTime=0The hang is random
It happens after roughly 1 in 600 to 1500 enqueues.
Two consecutive runs of the identical command (
--iterations=1500 --idleTime=30) gave HANG then ok.A single passing run therefore proves nothing. I recommend at least 8000 enqueues per configuration when verifying.
The original FP32 version may hang MUCH later but it WILL hang, and it may also PASS sometimes. generally this is very random! But IF it happens then the GPU is stuck!
What we ruled out (all measured)
1x11x576x7041x11x576x7041x11x576x704, identical ONNXoptShapes=1x11x576x7041x7x576x704So the trigger is very narrow: this one input shape on this graph (of course there may be many more), plus an idle GPU between enqueues.
Neighbouring shapes (including the same height at other widths, and the same width at other heights) are good.
Additionally, from the application side (before we reduced it to
trtexec), we ruled out:cudaStreamSynchronizeimmediately beforeenqueueV3(empty stream,cudaSuccess) still hangs. The stuck kernel belongs to the engine.compute-sanitizer memcheckreports zero errors.Idle gap is important!
This is not a synthetic condition.
In our video pipeline, RIFE enqueues are separated by an idle GPU whenever a CPU-side stage is the bottleneck (software video decode, or an x265 CPU encode applying backpressure).
Under a saturated GPU pipeline (hardware decode and encode) the same shape runs fine indefinitely, which is why this only surfaced on one code path!!
Impact
Any application feeding this engine a
576x704input on a non-saturated GPU hangs permanently, with no error, no exception, and no timeout.It cannot be caught or recovered from in-process!
Our workaround, avoiding the shape, is not something an application can do in general, since the "bad" shape is not predictable.