fix: reject malformed streamed data encoding - #16423
Merged
Rich-Harris merged 2 commits intoJul 19, 2026
Merged
Conversation
|
Install the latest version of pnpm add https://pkg.svelte.dev/@sveltejs/kit/c/3338528cdaec3a69658fc87f7b86ee026a90ce30Open in Note This PR is from a fork. A maintainer must approve approve each commit before it can be built and installed. |
🦋 Changeset detectedLatest commit: 3338528 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Author
Woohoo! You're Welcome! so pleased we could help with this one! |
Rich-Harris
pushed a commit
that referenced
this pull request
Jul 20, 2026
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to version-3, this PR will be updated.⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ `version-3` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `version-3`.⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ # Releases ## @sveltejs/kit@3.0.0-next.11 ### Major Changes - breaking: `config` exported from a universal route file takes precedence over a server one ([#16400](#16400)) - breaking: consistent special filename patterns ([#16382](#16382)) ### Minor Changes - feat: support sourcemaps in production ([#16412](#16412)) - feat: support function validators for environment variables ([#16402](#16402)) - feat: better error logging ([#16374](#16374)) ### Patch Changes - fix: don't treat callable standard schemas as function param matchers ([#16403](#16403)) - fix: reject malformed streamed data encoding ([#16423](#16423)) - fix: hide stack traces for internal errors like 404s ([#16411](#16411)) - perf: match only unpaired surrogates when escaping HTML ([#16407](#16407)) - fix: don't report empty environment variables as missing ([#16401](#16401)) - chore: clarify which hooks run during server route resolution ([#16397](#16397)) ## @sveltejs/adapter-node@6.0.0-next.5 ### Minor Changes - feat: better error logging ([#16374](#16374)) ### Patch Changes - Updated dependencies [[`5220191`](5220191), [`8cb2f7d`](8cb2f7d), [`b88c7a7`](b88c7a7), [`a6ea113`](a6ea113), [`6446f64`](6446f64), [`58f1789`](58f1789), [`09774a2`](09774a2), [`c542fdd`](c542fdd), [`aedaa27`](aedaa27), [`428ee1a`](428ee1a), [`fefb3ae`](fefb3ae)]: - @sveltejs/kit@3.0.0-next.11 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
5 tasks
Rich-Harris
pushed a commit
that referenced
this pull request
Aug 19, 2026
`read_stream` accumulates decoded text in one string, re-runs `indexOf` over the whole buffer after every transport chunk and re-slices the buffer for every emitted record. When one frame spans many chunks, every prefix is rescanned and total work grows quadratically with frame size. This parser sits behind streamed page data and remote functions (NDJSON) and `query.live` (SSE), so the cost lands on the client for large payloads. With this change only newly decoded text is searched. Already searched text accumulates in an array and is joined once per completed frame. The last `delimiter.length - 1` characters stay in the unsearched tail so a delimiter split across chunk boundaries still matches. One 2 MiB frame, Node 22 x64, median of repeated runs: | transport chunks | before | after | speedup | | ---: | ---: | ---: | ---: | | 2048 × 1 KiB | 1720.7 ms | 4.0 ms | 429× | | 512 × 4 KiB | 431.0 ms | 3.2 ms | 135× | | 128 × 16 KiB | 111.5 ms | 3.1 ms | 36× | | 32 × 64 KiB | 31.3 ms | 3.1 ms | 10× | Many small records, the common path, get slightly faster (2 MiB of 64 byte records in 16 KiB chunks, ~21 ms to ~15 ms) because the old code re-sliced the buffer once per record. Output is unchanged. The rewrite matched the previous implementation across 4000 randomized cases covering both delimiters, delimiters split between chunks, multibyte UTF-8 split between chunks and trailing unterminated records. The new unit tests also pass against the previous implementation. `read_stream` was extracted in #15957 and last changed in #16423. No open PR modifies `stream.js`, `ndjson.js` or `sse.js`. --- ### Before submitting the PR, please make sure you do the following - [ ] It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs - [x] This message body should clearly illustrate what problems it solves. - [x] Ideally, include a test that fails without this PR but passes with it. ### Tests - [x] Run the tests with `pnpm test` and lint the project with `pnpm lint` and `pnpm check` ### Changesets - [x] If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running `pnpm changeset` and following the prompts. Changesets that add features should be `minor` and those that fix bugs should be `patch`. Please prefix changeset messages with `feat:`, `fix:`, or `chore:`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related to #15511.
Summary
Thanks for SvelteKit. I really appreciate the care that goes into this project.
This PR tightens streamed
__data.jsonparsing so newline-delimited JSON data is decoded with fatal UTF-8 handling. Previously, malformed UTF-8 bytes could be decoded as replacement characters and then surface later asJSON.parsesyntax errors. With this change, malformed stream encoding is rejected at the decoding boundary instead of being parsed as corrupted JSON text.The diagnostic path that led to this patch traced the streamed response across the raw bytes -> UTF-8 text -> NDJSON record -> JSON parse boundary. That pointed at the byte-to-text transition as the useful place to fail fast, rather than changing the later JSON parsing path.
I also added regression coverage for the streamed-data parser:
I did not have a reliable end-to-end reproduction of the original timing/network conditions described in #15511. The issue notes that the full problem has been difficult to reproduce, so this PR focuses on the malformed UTF-8 / NDJSON boundary shown by the replacement-character failures in the report. Happy to adjust the scope, wording, or test shape if you would prefer a different framing.
Validation
Passed:
pnpm -F @sveltejs/kit test:unitpnpm lintafter rerunning with a larger Node heap because the default local heap hit an OOM during repo-wide ESLintpnpm checkgit diff --checkAlso run:
KIT_E2E_BROWSER=chromium pnpm run test:kitThat package-level browser/integration run reached the Chromium Playwright tests, but failed in
packages/kit/test/apps/asyncon an existing directhttp.get/ Playwright webServer readiness case withECONNREFUSED 127.0.0.1:5173. I did not change that area; the new unit regression passed in the focused and full@sveltejs/kitunit runs.Please don't delete this checklist! Before submitting the PR, please make sure you do the following:
Tests
pnpm testand lint the project withpnpm lintandpnpm checkChangesets
pnpm changesetand following the prompts. Changesets that add features should beminorand those that fix bugs should bepatch. Please prefix changeset messages withfeat:,fix:, orchore:.Edits
Disclosure: I used AI-assisted coding tools while preparing this PR. I reviewed the changes myself, tested them, and take responsibility for the implementation and any follow-up revisions needed.