Skip to content

[3.0] Fixes broken SNI resolution without breaking the fetch guards - #9536

Closed
albertlast wants to merge 8 commits into
SimpleMachines:release-3.0from
albertlast:3.0/sni_fetch_safe_fixes
Closed

[3.0] Fixes broken SNI resolution without breaking the fetch guards#9536
albertlast wants to merge 8 commits into
SimpleMachines:release-3.0from
albertlast:3.0/sni_fetch_safe_fixes

Conversation

@albertlast

Copy link
Copy Markdown
Collaborator

Description

This is #9535 with the bugs taken out, so it is built on top of Sesquipedalian's five
commits and adds three of my own. If #9535 is merged first, only my three remain.

The approach in #9535 is the right one. Substituting a literal IP for the host was
always going to break SNI, name-based virtual hosting and certificate validation all at
once, and validating the name while pinning the connection is the correct replacement.
The patch just could not run.

Why nothing could be fetched at allUrl::isFetchSafe() reads $url throughout,
which is never assigned. Every check ran against null, so the method always returned
false and the guard rejected every URL before a connection was attempted. That is
sbulen's report on #9535.

Guards that had stopped guarding

  • ProxyServer::checkRequest() had the test inverted: || $request->isFetchSafe(...)
    where the old code rejected when makeSafe() returned null. proxy.php refused every
    public image and fetched the private addresses the check exists to keep it away from.
  • CurlFetcher::redirect() still called WebFetchApi::makeSafe(), which the PR deletes.
    Any 301/302/307 was a fatal error.
  • CurlFetcher and FtpFetcher never imported SMF\IP, so new IP(...) resolved to
    SMF\WebFetch\APIs\IP and fatalled on every request that reached the new check.
    SocketFetcher got the import; the other two did not.
  • FtpConnection::passive() no longer looks at the address the server tells us to
    connect to next, which is what the check was for — an FTP bounce points it at the
    local network. FtpFetcher does check, but the package manager and the maintenance
    tools construct FtpConnection directly in twelve places and never go near
    FtpFetcher.
  • is_fetch_safe() passed [] as the allowed scheme list, which matches nothing, so it
    always returned false. Under makeSafe() an empty list meant "any scheme with a
    handler"; isFetchSafe() had no such fallback, so it now takes one.

Two things I did differently

curl pins the address instead of checking afterwards. Comparing
curl_getinfo()['primary_ip'] after the request only tells us where we already went.
CURLOPT_RESOLVE tells curl up front which address it may use for this host, so the URL
keeps its name and SNI, the Host header and certificate validation all still see the real
one, with no window between the check and the connect. The post-connection check stays as
a second opinion for builds that ignore the option. SocketFetcher's connect-then-verify
I left alone — the only exposure there is a handshake, with no request data sent, and
stream_socket_client with an SSL peer_name context is easy to get subtly wrong.

Url::proxied() stays off the resolver. isFetchSafe() does a dns_get_record(), and
proxied() runs for every image in every post, so a topic with twenty images became
twenty blocking lookups. It also only decides whether to route through proxy.php, which
does its own checking, so a DNS hiccup silently stopping proxying is a bad trade. It goes
back to the cheap literal-IP and reserved-TLD test.

Smaller things

  • getIPs() remembers what a host resolved to for the rest of the request, as
    WebFetchApi::$resolved_hosts used to. This is load-bearing now, not just a lookup
    count: the post-connection checks are only meaningful if they compare against what was
    resolved before connecting, or a rebinding attack just answers twice. parse() drops
    the cache, since the host may be about to change.
  • A host that resolves to nothing is no longer "safe". [] === array_filter([]), and
    dns_get_record() never sees names that only exist in the hosts file, so
    http://someinternalbox/ was passing.
  • resolvesTo() compares addresses rather than objects; a loose object comparison also
    weighs IP::$host, which either side may have filled in with a reverse lookup.
  • make_fetch_safe() comes back, deprecated. It is a 2.1 function and Subs-Compat.php
    exists to keep 2.1 names callable, so removing it fatals any mod that calls it. It
    returns the URL unchanged when safe and null when not, so $url = make_fetch_safe($url); if ($url === null) still works — and what comes back can now actually be fetched.
  • result() returns null rather than reading index -1 when a request was refused before
    anything was recorded. WebFetchApi::fetch() asks for result('success') on that path,
    so every refusal emitted a warning. Pre-existing, but on the path this PR is about; say
    the word and I will drop it.

Testing

php-cs-fixer is clean on all seven files. There is no test suite, so I exercised the
code directly against the URL from the issue, on release-3.0 first and then on this
branch.

Before, on release-3.0:

===== CurlFetcher =====
  SNI host, real fetch                       FAIL  code='0' len=0
===== SocketFetcher =====
  SNI host, real fetch                       FAIL
    'url' => 'https://172.67.70.118/smf/current-version.js?version=SMF+3.0+Alpha+4'

That second line is the bug in one place — the host replaced by the IP.

After, on this branch:

===== CurlFetcher =====
  SNI host, real fetch                       SUCCESS  code='200' len=38
  body looks like current-version.js         OK
  http://127.0.0.1/ refused                  OK
  follows http->https redirect               OK  code='403' len=5520
===== SocketFetcher =====
  SNI host, real fetch                       SUCCESS  code=200 len=38
  body looks like current-version.js         OK
  http://127.0.0.1/ refused                  OK

Plus 20 checks over isFetchSafe(), resolvesTo() and the compat shims — public host,
loopback, private, 169.254.169.254, [::1], reserved TLDs, an unresolvable host, scheme
filtering both ways, and that the URL comes back byte-for-byte unchanged — and 6 over
Url::proxied(). All pass.

One thing I found and did not touch: SocketFetcher returns a 400 when it follows an
http to https redirect. It does that identically on unmodified release-3.0, so it is
pre-existing and unrelated, and it wants its own issue rather than a rider on this one.

Issues References (Fixes|Related|Closes)

  1. Fixes [2.1|3.0]: make_fetch_safe breaks SNI required fetchs #9533
  2. Related [3.0] Fixes broken SNI resolution #9535

Sesquipedalian and others added 8 commits August 18, 2026 18:01
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
The method read $url throughout, which is never assigned, so every check
ran against null and the method always returned false. That is why
nothing could be fetched at all: the guard rejected the URL before any
connection was attempted.

While in here:

- An empty $allowed_schemes now means "any scheme the WebFetchApi has a
  handler for", which is what WebFetchApi::makeSafe() meant by it, and
  what the callers that pass nothing are expecting. Without this, the
  parameter had no sensible value for Url and Subs-Compat to pass.

- A host that resolves to nothing is no longer treated as safe. An empty
  array trivially equals its own array_filter(), and dns_get_record()
  never sees names that only exist in the system's hosts file, so
  http://someinternalbox/ was passing the guard.

- getIPs() remembers what a host resolved to for the rest of the
  request. WebFetchApi::$resolved_hosts used to do this. It matters for
  more than lookup counts now: the post-connection checks in the fetchers
  are only meaningful if they compare against what we resolved *before*
  connecting, otherwise a rebinding attack simply answers twice.

- parse() drops those addresses, since the host may be about to change.

- resolvesTo() compares the addresses rather than the objects. A loose
  comparison also weighs IP::$host, which either side may have filled in
  with a reverse lookup.

- proxied() goes back to the cheap test it had before. It runs for every
  image in every post, and it only decides whether to route through
  proxy.php, which does its own checking. Making it depend on the
  resolver means a DNS hiccup silently stops proxying.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
Four of them had stopped guarding anything:

- ProxyServer::checkRequest() had the test the wrong way round, so
  proxy.php refused every public image and happily fetched the private
  addresses the check exists to keep it away from.

- CurlFetcher::redirect() still called WebFetchApi::makeSafe(), which is
  gone, so following any 301, 302 or 307 was a fatal error rather than a
  re-validated hop.

- CurlFetcher and FtpFetcher never imported SMF\IP, so the new
  post-connection checks resolved SMF\WebFetch\APIs\IP and fatalled on
  every request that got as far as them.

- FtpConnection::passive() no longer looked at the address the server
  asked us to connect to next, which is the whole point of the check: an
  FTP bounce points it at the local network. FtpFetcher does check, but
  the package manager and the maintenance tools use FtpConnection
  directly and never go near FtpFetcher.

Then curl gets to do this properly. Comparing curl_getinfo()['primary_ip']
after the fact only tells us where we already went. CURLOPT_RESOLVE tells
curl up front which address it is allowed to use for this host, so the URL
keeps its name and SNI, the Host header and certificate validation all
still see the real one. The check afterwards stays as a second opinion for
builds that ignore the option, but it no longer writes its failure into a
response slot that does not exist yet.

Lastly, result() returns null instead of reading index -1 when a request
was refused before anything was recorded. WebFetchApi::fetch() asks for
result('success') on that path, so every refusal emitted a warning.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
make_fetch_safe() is a 2.1 function, and Subs-Compat.php exists to keep
2.1 names callable, so removing it outright means any mod that calls it
fatals. It comes back, deprecated, returning the URL unchanged when it is
safe and null when it is not. Callers written as

	$url = make_fetch_safe($url);
	if ($url === null)

keep working, and what they get back is now a URL that can actually be
fetched, rather than one with the host replaced by an IP address.

is_fetch_safe() was passing an empty array through as the allowed scheme
list, which matched nothing and made it always return false. It leaves
the argument off now and takes the default.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
@Sesquipedalian

Sesquipedalian commented Aug 19, 2026

Copy link
Copy Markdown
Member

The AI's assessment of bugs in #9535 was correct, but its proposed solutions here are problematic for several reasons. I will instead fix the issues in #9535 itself.

@albertlast

Copy link
Copy Markdown
Collaborator Author

was only a try/way to better express a possible solution

@jdarwood007 jdarwood007 added this to the 3.0 Alpha 5 milestone Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[2.1|3.0]: make_fetch_safe breaks SNI required fetchs

3 participants