Add Triton vs TensorRT vs ONNX Runtime comparison blog post - #33
Conversation
…frameworks Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Codex Infinity Start a task on this PR's branch by commenting:
Tasks and logs: https://codex-infinity.com |
❌ Deploy Preview for hownz failed.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b2ac644957
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| # Triton vs TensorRT vs ONNX Runtime - ML Inference Optimization Compared | ||
|
|
||
| In the [CuteDSL post](/2026/03/23/cutedsl-accelerated-ml-inference-triton-cuda-kernels/) we used OpenAI Triton to fuse GPU kernels and hit 24x speedups on time series inference. But Triton is just one tool in the inference optimization stack. TensorRT and ONNX Runtime solve overlapping but different problems. Here's when to use each. |
There was a problem hiding this comment.
Both this intro link and the duplicate footer link use /2026/03/23/cutedsl-accelerated-ml-inference-triton-cuda-kernels/, but this site is configured with permalink: :year/:month/:day/:title/ (_config.yml:17) and the referenced post is source/_posts/cutedsl.md, so Hexo will publish it at /2026/03/23/cutedsl/. As written, readers clicking through from this article will hit a 404.
Useful? React with 👍 / 👎.
|
|
||
| **OpenAI Triton** is a Python DSL for writing GPU kernels. You write blocked programs in Python that compile down to PTX/AMDGPU IR. It replaces hand-written CUDA for custom operations -- attention, normalization, activation fusions. It's what powers `torch.compile` under the hood and what [CuteDSL](https://github.com/lee101/cutedsl) uses for its fused kernels. | ||
|
|
||
| **NVIDIA TensorRT** (currently v10.16) is a graph-level inference optimizer. You feed it an ONNX graph or TorchScript model, and it performs layer fusion, kernel auto-tuning, precision calibration (FP16/INT8/FP4), and memory planning. It outputs an optimized engine file tuned for a specific GPU. Think of it as a compiler for entire neural networks. |
There was a problem hiding this comment.
Saying that TensorRT takes “an ONNX graph or TorchScript model” conflates TensorRT with Torch-TensorRT. The TensorRT API shown later in this post is the ONNX parser path; PyTorch/TorchScript compilation is documented as a separate Torch-TensorRT workflow. Readers starting from a TorchScript model will otherwise look for a direct TensorRT import path that this toolchain does not provide.
Useful? React with 👍 / 👎.
Summary
Test plan
npx hexo generatesucceeds without errors (150 files generated)/2026/03/23/triton-vs-tensorrt-vs-onnx-runtime-ml-inference/Generated with Claude Code