Route raw non-ASCII paths when an API declares a non-ASCII route

[ericproulx]

Summary

Rack hands PATH_INFO over binary, and matching a binary string that holds non-ASCII bytes against a UTF-8 regexp raises Encoding::CompatibilityError. A route whose path holds a non-ASCII literal (get '/café/:id') compiles to such a regexp, and every request is matched against a union that holds it. So once an API declares one, every request whose path carries raw non-ASCII bytes raises — whichever route it was meant for, and whether or not the bytes are valid UTF-8. The percent-encoded spelling of the same path routes fine.

• Router#compile! records whether any compiled union has a fixed encoding.
• Only then, and only for a path that is not ASCII, Router#call routes a copy of the path tagged UTF-8 and publishes that copy as the normalized path, which the path versioner reads. PATH_INFO itself stays binary, as Rack requires.
• A path whose bytes are not UTF-8 cannot be matched against those routes at all, so it is answered as a path no route matched (404, X-Cascade: pass) instead of raising.
• Router#recognize_path gets the same treatment.
• An API without a non-ASCII route routes exactly as before; what it pays is one instance variable read per request.

Since #2940, a route that spells its path out in full is answered from a table keyed by that path, and a binary key holding non-ASCII bytes never equals the UTF-8 one. The tagged copy is what finds such a route there, so get '/café' routes too.

Behaviour

API declaring get '/café', get '/café/:id' and get '/plain/:id', PATH_INFO binary:

Path	Before	After
/café (UTF-8 bytes)	Encoding::CompatibilityError	200 from /café
/café/42 (UTF-8 bytes)	Encoding::CompatibilityError	200 from /café/:id
/plain/é (UTF-8 bytes)	Encoding::CompatibilityError	200 from /plain/:id, params[:id] == 'é'
/plain/\xFF (not UTF-8)	Encoding::CompatibilityError	404
/caf%C3%A9/42, /plain/42	200	200

An API whose routes are all ASCII routes every one of these paths as before, /plain/\xFF included (200, with the param left as invalid UTF-8).

Test plan

• Regression examples in spec/grape/router_spec.rb, with a binary PATH_INFO: the UTF-8 paths to a parameterized and a fully spelled-out non-ASCII route and to an ASCII route, the path that is not UTF-8, recognize_path, and an ASCII-only API still routing bytes that are not UTF-8.
• Mutation-checked: 5 of 6 mutations of the change fail an example. The survivor drops the next unless input guard, which is equivalent — Regexp#match(nil) returns nil, so an unguarded nil path reaches the same 404 — and is kept so a nil is never published as the normalized path.
• Full RSpec suite passes locally (2,871 examples); RuboCop clean on the changed files.
• CI green.

🤖 Generated with Claude Code

[github-actions]

Danger Report

No issues found.

View run