[3.0] Fixes broken SNI resolution without breaking the fetch guards - #9536
Closed
albertlast wants to merge 8 commits into
Closed
[3.0] Fixes broken SNI resolution without breaking the fetch guards#9536albertlast wants to merge 8 commits into
albertlast wants to merge 8 commits into
Conversation
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
The method read $url throughout, which is never assigned, so every check ran against null and the method always returned false. That is why nothing could be fetched at all: the guard rejected the URL before any connection was attempted. While in here: - An empty $allowed_schemes now means "any scheme the WebFetchApi has a handler for", which is what WebFetchApi::makeSafe() meant by it, and what the callers that pass nothing are expecting. Without this, the parameter had no sensible value for Url and Subs-Compat to pass. - A host that resolves to nothing is no longer treated as safe. An empty array trivially equals its own array_filter(), and dns_get_record() never sees names that only exist in the system's hosts file, so http://someinternalbox/ was passing the guard. - getIPs() remembers what a host resolved to for the rest of the request. WebFetchApi::$resolved_hosts used to do this. It matters for more than lookup counts now: the post-connection checks in the fetchers are only meaningful if they compare against what we resolved *before* connecting, otherwise a rebinding attack simply answers twice. - parse() drops those addresses, since the host may be about to change. - resolvesTo() compares the addresses rather than the objects. A loose comparison also weighs IP::$host, which either side may have filled in with a reverse lookup. - proxied() goes back to the cheap test it had before. It runs for every image in every post, and it only decides whether to route through proxy.php, which does its own checking. Making it depend on the resolver means a DNS hiccup silently stops proxying. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
Four of them had stopped guarding anything:
- ProxyServer::checkRequest() had the test the wrong way round, so
proxy.php refused every public image and happily fetched the private
addresses the check exists to keep it away from.
- CurlFetcher::redirect() still called WebFetchApi::makeSafe(), which is
gone, so following any 301, 302 or 307 was a fatal error rather than a
re-validated hop.
- CurlFetcher and FtpFetcher never imported SMF\IP, so the new
post-connection checks resolved SMF\WebFetch\APIs\IP and fatalled on
every request that got as far as them.
- FtpConnection::passive() no longer looked at the address the server
asked us to connect to next, which is the whole point of the check: an
FTP bounce points it at the local network. FtpFetcher does check, but
the package manager and the maintenance tools use FtpConnection
directly and never go near FtpFetcher.
Then curl gets to do this properly. Comparing curl_getinfo()['primary_ip']
after the fact only tells us where we already went. CURLOPT_RESOLVE tells
curl up front which address it is allowed to use for this host, so the URL
keeps its name and SNI, the Host header and certificate validation all
still see the real one. The check afterwards stays as a second opinion for
builds that ignore the option, but it no longer writes its failure into a
response slot that does not exist yet.
Lastly, result() returns null instead of reading index -1 when a request
was refused before anything was recorded. WebFetchApi::fetch() asks for
result('success') on that path, so every refusal emitted a warning.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
make_fetch_safe() is a 2.1 function, and Subs-Compat.php exists to keep 2.1 names callable, so removing it outright means any mod that calls it fatals. It comes back, deprecated, returning the URL unchanged when it is safe and null when it is not. Callers written as $url = make_fetch_safe($url); if ($url === null) keep working, and what they get back is now a URL that can actually be fetched, rather than one with the host replaced by an IP address. is_fetch_safe() was passing an empty array through as the allowed scheme list, which matched nothing and made it always return false. It leaves the argument off now and takes the default. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
This was referenced Aug 19, 2026
Member
Collaborator
Author
|
was only a try/way to better express a possible solution |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This is #9535 with the bugs taken out, so it is built on top of Sesquipedalian's five
commits and adds three of my own. If #9535 is merged first, only my three remain.
The approach in #9535 is the right one. Substituting a literal IP for the host was
always going to break SNI, name-based virtual hosting and certificate validation all at
once, and validating the name while pinning the connection is the correct replacement.
The patch just could not run.
Why nothing could be fetched at all —
Url::isFetchSafe()reads$urlthroughout,which is never assigned. Every check ran against null, so the method always returned
false and the guard rejected every URL before a connection was attempted. That is
sbulen's report on #9535.
Guards that had stopped guarding
ProxyServer::checkRequest()had the test inverted:|| $request->isFetchSafe(...)where the old code rejected when
makeSafe()returned null. proxy.php refused everypublic image and fetched the private addresses the check exists to keep it away from.
CurlFetcher::redirect()still calledWebFetchApi::makeSafe(), which the PR deletes.Any 301/302/307 was a fatal error.
CurlFetcherandFtpFetchernever importedSMF\IP, sonew IP(...)resolved toSMF\WebFetch\APIs\IPand fatalled on every request that reached the new check.SocketFetchergot the import; the other two did not.FtpConnection::passive()no longer looks at the address the server tells us toconnect to next, which is what the check was for — an FTP bounce points it at the
local network.
FtpFetcherdoes check, but the package manager and the maintenancetools construct
FtpConnectiondirectly in twelve places and never go nearFtpFetcher.is_fetch_safe()passed[]as the allowed scheme list, which matches nothing, so italways returned false. Under
makeSafe()an empty list meant "any scheme with ahandler";
isFetchSafe()had no such fallback, so it now takes one.Two things I did differently
curl pins the address instead of checking afterwards. Comparing
curl_getinfo()['primary_ip']after the request only tells us where we already went.CURLOPT_RESOLVEtells curl up front which address it may use for this host, so the URLkeeps its name and SNI, the Host header and certificate validation all still see the real
one, with no window between the check and the connect. The post-connection check stays as
a second opinion for builds that ignore the option.
SocketFetcher's connect-then-verifyI left alone — the only exposure there is a handshake, with no request data sent, and
stream_socket_clientwith an SSLpeer_namecontext is easy to get subtly wrong.Url::proxied()stays off the resolver.isFetchSafe()does adns_get_record(), andproxied()runs for every image in every post, so a topic with twenty images becametwenty blocking lookups. It also only decides whether to route through proxy.php, which
does its own checking, so a DNS hiccup silently stopping proxying is a bad trade. It goes
back to the cheap literal-IP and reserved-TLD test.
Smaller things
getIPs()remembers what a host resolved to for the rest of the request, asWebFetchApi::$resolved_hostsused to. This is load-bearing now, not just a lookupcount: the post-connection checks are only meaningful if they compare against what was
resolved before connecting, or a rebinding attack just answers twice.
parse()dropsthe cache, since the host may be about to change.
[] === array_filter([]), anddns_get_record()never sees names that only exist in the hosts file, sohttp://someinternalbox/was passing.resolvesTo()compares addresses rather than objects; a loose object comparison alsoweighs
IP::$host, which either side may have filled in with a reverse lookup.make_fetch_safe()comes back, deprecated. It is a 2.1 function andSubs-Compat.phpexists to keep 2.1 names callable, so removing it fatals any mod that calls it. It
returns the URL unchanged when safe and null when not, so
$url = make_fetch_safe($url); if ($url === null)still works — and what comes back can now actually be fetched.result()returns null rather than reading index-1when a request was refused beforeanything was recorded.
WebFetchApi::fetch()asks forresult('success')on that path,so every refusal emitted a warning. Pre-existing, but on the path this PR is about; say
the word and I will drop it.
Testing
php-cs-fixeris clean on all seven files. There is no test suite, so I exercised thecode directly against the URL from the issue, on
release-3.0first and then on thisbranch.
Before, on
release-3.0:That second line is the bug in one place — the host replaced by the IP.
After, on this branch:
Plus 20 checks over
isFetchSafe(),resolvesTo()and the compat shims — public host,loopback, private,
169.254.169.254,[::1], reserved TLDs, an unresolvable host, schemefiltering both ways, and that the URL comes back byte-for-byte unchanged — and 6 over
Url::proxied(). All pass.One thing I found and did not touch:
SocketFetcherreturns a 400 when it follows anhttptohttpsredirect. It does that identically on unmodifiedrelease-3.0, so it ispre-existing and unrelated, and it wants its own issue rather than a rider on this one.
Issues References (Fixes|Related|Closes)