feat: add WSS ZIP cache

[AndrewBrown-USDA]

This PR introduces a caching system for Web Soil Survey ZIP archives in the downloadSSURGO() workflow, along with new utility functions for cache management. For #473

[Copilot]

Copilot review overview

🟡 Changes recommended

Critical download and cache-safety issues remain unresolved.

Review effort: Lite
Findings: 2 · 2 · 1

Open (5)

• Preserve cached archive until forced replacement download succeeds · New
• Handle partial failures in multi-area downloads · New
• Include fiscal year when retaining latest cache archives · New
• Resolve exdir only when extracting archives · New
• Document fiscal-year directory behavior for default exdir · New

What changed in this PR

Adds persistent fiscal-year-organized caching for Web Soil Survey ZIP downloads.

Changes:

• Adds cache management, listing, cleanup, and download utilities.
• Integrates caching with downloadSSURGO() and createSSURGO().
• Updates tests, documentation, exports, and release notes.

Unresolved issues include unsafe forced replacement, incomplete-download handling, cache deduplication, fallback behavior, and documentation corrections.

File	Description
vignettes/​local-ssurgo.Rmd	Documents persistent cache usage.
tests/​testthat/​test-downloadSSURGO-cache.R	Tests cache behavior and fallback scenarios.
R/​WSS_cache.R	Implements cache management and download utilities.
R/​createSSURGO.R	Integrates caching into download and database workflows.
NEWS.md	Records the new cache functionality.
NAMESPACE	Exports cache helper functions.
man/​list_WSS_cache.Rd	Documents cache inventory.
man/​downloadSSURGO.Rd	Documents updated download arguments.
man/​createSSURGO.Rd	Updates extraction argument documentation.
man/​clear_WSS_cache.Rd	Documents cache cleanup.

Files not reviewed (4)

• man/clear_WSS_cache.Rd: Generated file
• man/createSSURGO.Rd: Generated file
• man/downloadSSURGO.Rd: Generated file
• man/list_WSS_cache.Rd: Generated file

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[dylanbeaudette]

I'd like the other soilDB authors to weigh-in on the topic of accepting any genAI suggested changes.

[AndrewBrown-USDA]

I will note that having an agent review a PR is not the same as blindly accepting the proposed changes.

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved cache reliability and extraction-isolation issues include a critical risk of mixing outputs across requests.

Review effort: Lite
Findings: 1

Open (1)

• Default extraction directory mixes outputs across calls · New

Resolved since last review (5)

• Handle partial failures in multi-area downloads
• Preserve cached archive until forced replacement download succeeds
• Resolve exdir only when extracting archives
• Include fiscal year when retaining latest cache archives
• Document fiscal-year directory behavior for default exdir

Files not reviewed (4)

• man/clear_WSS_cache.Rd: Generated file
• man/createSSURGO.Rd: Generated file
• man/downloadSSURGO.Rd: Generated file
• man/list_WSS_cache.Rd: Generated file

Previously missed (4)

In code that hasn't changed since last review

Failed initial downloads leave corrupt destination files

R/​WSS_cache.R:160

For a first download (force = FALSE and no existing destination), curl_download() writes directly to destfile. If the transfer fails after creating a partial file, the error path does not remove it because tmp_destfile equals destfile; the next call then returns that corrupt ZIP at line 153 without retrying. Always download to a temporary file and rename it into place only after success, as is already done for forced replacements.

Multiple fiscal years are matched incorrectly

R/​WSS_cache.R:219

fiscal_year is normalized element-wise, but == only works correctly for a single requested year; passing multiple years (for example list_WSS_cache(fiscal_year = c(2023, 2024))) recycles the shorter vector and can select the wrong rows. Use set membership for the normalized values so all requested fiscal years are honored.

Fallback cache selection does not support LIKE predicates

R/​createSSURGO.R:166

The documented WHERE form areasymbol LIKE 'KS%' cannot use this fallback: .extract_wss_areasymbols() returns KS%, but .wss_cache_select() treats areasymbols as exact values, so cached KS### entries are never selected when the remote metadata query fails. Please make the fallback evaluate supported WHERE predicates against the cache inventory (at least SQL LIKE), or limit this path to exact area-symbol inputs and report that limitation.

Document NULL exdir behavior in createSSURGO

R/​createSSURGO.R:308

Making exdir optional changes the public behavior, but the @param exdir documentation still describes only an explicitly supplied path and does not say that NULL selects the latest fiscal-year WSS cache directory (or errors when none exists). Update the roxygen text so the generated createSSURGO help documents this new default.

[Copilot]

Copilot review overview

🟡 Changes recommended

Critical API and area-filtering issues, plus unresolved cache fallback and replacement problems, block approval.

Review effort: Lite
Findings: 2 · 1

Open (3)

• Preserve positional API when adding areasymbols · New
• Match areasymbols in directories and filenames · New
• Preserve cached archive until replacement succeeds · New

Resolved since last review (1)

• Default extraction directory mixes outputs across calls

Files not reviewed (4)

• man/clear_WSS_cache.Rd: Generated file
• man/createSSURGO.Rd: Generated file
• man/downloadSSURGO.Rd: Generated file
• man/list_WSS_cache.Rd: Generated file

[AndrewBrown-USDA]

Test failures are unrelated to this PR (fetchSCAN() failures); those seem to be persisting so I will merge this shortly and focus on some improvements to SCAN testing in another PR

[StephenRoecker-USDA]

I'd like the other soilDB authors to weigh-in on the topic of accepting any genAI suggested changes.
@dylanbeaudette

I don't have a problem with genAI changes in general. I think all the same best practices apply. Targeted changes, presumed ownership of individual contributors functions, etc.. Where it gets sticky is functions that are reused internally by packages, such as uncode, that have to work. In such cases as usual it requires sufficient testing and eyeballs.