feat(storage): add a static delay open request hedging strategy

[ajayky-os]

Implement TTFB Speculative Hedging with Configurable Connect Timeouts

Overview

This PR introduces a concurrent, speculative hedging architecture to the GCS C++ SDK. The primary goal is to mitigate extreme tail latencies (e.g., 20s+ stalls) caused by OS-level TCP/kernel drops during periods of high-throughput network congestion.

This architecture introduces two primary mitigation layers:

1. Time-To-First-Byte (TTFB) Hedging: Races a secondary socket if the initial HTTP `Open()` phase stalls beyond a configurable delay.
2. libcurl Native Connection Timeout in milli seconds: HttpConnectTimeoutOption (maps to CURLOPT_CONNECTTIMEOUT_MS for downloads) is added as a complementary fail-safe against black-holed TCP connects. It is applied after the stall-timeout options because libcurl treats CURLOPT_CONNECTTIMEOUT and CURLOPT_CONNECTTIMEOUT_MS as the same setting (last writer wins).

This is the first PR in a series. A follow-up adds a dynamic strategy that adapts the hedge delay to observed latency percentiles; this PR uses a fixed, configurable delay.

Architectural Highlights

1. TTFB-Exclusive Hedging (No Data Corruption)

Naive hedging of an `ObjectReadSource` stream risks severe data corruption and network exhaustion by duplicating many payload downloads.
This implementation explicitly restricts hedging to the stream's Open Phase (TTFB). Once a socket wins the initial connection race, the background thread gracefully exits, and all subsequent payload chunk reads continue sequentially on the caller's thread. This guarantees structural integrity and prevents multi-stream bandwidth DDoS.

2. Bounded Hedging Thread Pool

To prevent queue starvation and CPU exhaustion under heavy load, the `HedgingThreadPool` enforces strict gating mechanisms:

• Instantaneous Concurrency Limiter: Bounded via `MaxConcurrentHedgesOption`.
• QPS Token Bucket: Bounded via `ReadHedgeRateLimitOption`.
• Dynamic Fallback: If `ConnectionPoolSizeOption == 0` (unpooled), the pool correctly scales to `std::thread::hardware_concurrency() * 4` to match physical network capabilities.

3. Configurable Native Connect Timeout

The existing `DownloadStallTimeoutOption` maps to `CURLOPT_LOW_SPEED_TIME`, meaning aggressive timeouts would unintentionally kill healthy large-payload streams during minor jitter.
This PR introduces `HttpConnectTimeoutOption`, which maps explicitly to `CURLOPT_CONNECTTIMEOUT_MS`, allowing users to build a strict guillotine specifically for stalled TCP handshakes without threatening payload integrity.

Usage

Users can opt-in to the Hybrid Architecture via standard configuration options:
auto options = google::cloud::Options{}
.setgoogle::cloud::storage_experimental::EnableReadHedgingOption(true)
.setgoogle::cloud::storage_experimental::ReadHedgeDelayOption(std::chrono::milliseconds(500))
.setgoogle::cloud::storage_experimental::MaxConcurrentHedgesOption(15)
.setgoogle::cloud::storage_experimental::HttpConnectTimeoutOption(std::chrono::milliseconds(1000));
auto client = gcs::Client(options);

(Note: `hedge_pool_` is only allocated if `EnableReadHedgingOption` is true, enforcing the zero-overhead principle for non-hedging users).

Performance Benchmarks

We executed a 1-hour sequential testing suite (`us-central1`, 60 concurrent workers, 1MB payloads) comparing the baseline SDK against this architecture.

Baseline (Hedging Disabled):

• Throughput: 2,204,500 requests
• p50 Latency: 41.83 ms
• p99 Latency: 165.52 ms
• Max Latency: 21,292.60 ms (Severe kernel stall)

Static Hedging (Enabled):

• Throughput: 2,347,561 requests (+143,000 throughput)
• p50 Latency: 38.27 ms
• p99 Latency: 162.20 ms
• Max Latency: 5,331.10 ms (75% absolute reduction)

[gemini-code-assist]

Code Review

This pull request introduces experimental request hedging for ReadObject() streams to reduce tail latency by racing duplicate requests. It implements HedgedObjectReadSource and a dynamically-scaling HedgingThreadPool with token-bucket rate limiting and concurrency throttling. The review comments identify critical issues that must be addressed: a potential self-join deadlock when capturing std::shared_ptr<HedgingThreadPool> in the hedge task lambda, a concurrency race condition where checking and incrementing active hedges are not atomic, and performance overhead from zero-initializing the read buffer with std::vector<char> instead of std::unique_ptr<char[]>.

[v-pratap]

Is there any limit on n? if not then what if n is very large like 100 MB and it couldn't get allocated? will it be crashed without capturing the error?

[ajayky-os]

1. Added MaximumHedgeBufferOption: Added this to options.h with a default of 64 * 1024 * 1024 (64MB) and registered it in the ClientOptionList.
2. Added the Bypass Logic: In connection_impl.cc, right before deciding whether to wrap the stream in a HedgedObjectReadSource, added a check against the user's requested range. If they ask for 100MB in a single read, it safely bypasses the hedge pool and executes entirely inline:

if (request.HasOption<ReadRange>()) {
    auto const range = request.GetOption<ReadRange>().value();
    if (range.begin >= 0 && range.end >= range.begin &&
        static_cast<std::size_t>(range.end - range.begin) > max_buffer) {
        return retry_source_factory(); // Safely run inline!
    }
}

[scotthart]

Instead of adding a new Option and code that overwrites CURLOPT_CONNECTIONTIMEOUT with CURLOPT_CONNECTIONTIMEOUT_MS, what if we changed the type of TransferStallTimeoutOption to std::chrono::milliseconds, and updated all the calls to set CURLOPT_CONNECTIONTIMEOUT_MS instead of CURLOPT_CONNECTIONTIMEOUT?

[scotthart]

/gcbrun

[codecov]

Codecov Report

❌ Patch coverage is 93.20388% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.24%. Comparing base (a20125d) to head (27342aa).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
google/cloud/storage/internal/connection_impl.cc	61.29%	12 Missing ⚠️
...loud/storage/internal/hedged_object_read_source.cc	87.83%	9 Missing ⚠️
...storage/internal/hedged_object_read_source_test.cc	96.73%	5 Missing ⚠️
google/cloud/internal/curl_impl.cc	91.66%	1 Missing ⚠️
...oogle/cloud/storage/internal/hedging_thread_pool.h	98.57%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main   #16344    +/-   ##
========================================
  Coverage   92.24%   92.24%            
========================================
  Files        2227     2232     +5     
  Lines      209614   210014   +400     
========================================
+ Hits       193351   193723   +372     
- Misses      16263    16291    +28     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[rajeevpodar]

/gcbrun

[nidhiii-27]

/gcbrun

[rajeevpodar]

/gcbrun

[ajayky-os]

Instead of adding a new Option and code that overwrites CURLOPT_CONNECTIONTIMEOUT with CURLOPT_CONNECTIONTIMEOUT_MS, what if we changed the type of TransferStallTimeoutOption to std::chrono::milliseconds, and updated all the calls to set CURLOPT_CONNECTIONTIMEOUT_MS instead of CURLOPT_CONNECTIONTIMEOUT?

This will be a breaking change for people already using the CURLOPT_CONNECTIONTIMEOUT with unit assumption in seconds suddenly becomes milliseonds.

[scotthart]

/gcbrun

[ajayky-os]

Instead of adding a new Option and code that overwrites CURLOPT_CONNECTIONTIMEOUT with CURLOPT_CONNECTIONTIMEOUT_MS, what if we changed the type of TransferStallTimeoutOption to std::chrono::milliseconds, and updated all the calls to set CURLOPT_CONNECTIONTIMEOUT_MS instead of CURLOPT_CONNECTIONTIMEOUT?

Thanks for pointing out. I have updated all the calls to set CURLOPT_CONNECTIONTIMEOUT_MS instead of CURLOPT_CONNECTIONTIMEOUT. Precedence is explicit in the code (explicit option wins otherwise fall back to the stall timeout).

I hit some roadblocks in setting the existing TransferStallTimeoutOption to std::chrono::milliseconds:

• Existing TransferStallTimeoutOption configures two flags: CURLOPT_CONNECTIONTIMEOUT and CURLOPT_LOW_SPEED_TIME, we have ms alternative for CURLOPT_CONNECTIONTIMEOUT which we are using now but CURLOPT_LOW_SPEED_TIME has no milliseconds alternative. So we would have to truncate back to seconds for CURLOPT_LOW_SPEED_TIME.
• Since TransferStallTimeoutOption is public API, there are some issues with it's usage. Ex: std::chrono::seconds t = opts.get<TranferStallTimeoutOption>(); will not compile and auto n = opts.get<TranferStallTimeoutOption>().count() will start returning 30000 on place of 30 for example.