From ec7dd36c3463d1192fded5a66e44a2fa3e2e4e2f Mon Sep 17 00:00:00 2001 From: "V. David Zvenyach" Date: Thu, 10 Sep 2026 09:02:28 -0500 Subject: [PATCH 1/2] feat(sled): state, local and education procurement MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Six methods over Tango's new `/api/sled/` namespace — `list_sled_opportunities`/`get_sled_opportunity`, `list_sled_opportunity_revisions`, `get_sled_coverage`, `list_sled_forecasts`/`get_sled_forecast` — plus both `PageStream` iterators, four `bon`-derived options builders and five `SHAPE_SLED_*` constants. Every one of the API's 27 solicitation filters and 13 forecast filters is a named field. Parity with the Python, Node and Go SLED PRs. Four behaviors are documented on the options builders because each one misleads a caller who assumes federal semantics. Leaving both `status` and `active` unset returns open solicitations only. That default is the API's, and `list_sled_opportunities` deliberately does not synthesize `status=open` — doing so would make `active = Some(false)` unreachable, since `active=false` is the complement of open. A test pins that. `status` is API-derived liveness, recomputed every fifteen minutes. The source portal's own word is served as `source_status`, frozen at last capture, and is not a liveness filter. Category scheme tagging is partial, so `naics` matches only the small tagged share and `category_code` is the escape hatch. `meta.attachment_count` can be lower than the length of `attachments`, because an auto-generated portal cover sheet is listed and flagged but excluded from the count. `active`, `has_documents` and `source_declared` are `Option` rather than `bool`, so `false` reaches the server as a filter value instead of collapsing into the default. `SHAPE_SLED_REVISIONS_MINIMAL` omits `changes` on purpose: the per-field before/after needs a Small plan, so naming it in a suggested shape would 403 a Free caller. 191 unit tests pass (16 of them new), clippy clean under `-D warnings`, `cargo fmt --check` clean. Blocked until the SLED endpoints are live on the production API. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 19 + crates/tango/src/lib.rs | 8 +- crates/tango/src/resources/mod.rs | 5 + crates/tango/src/resources/sled.rs | 917 +++++++++++++++++++++++++++++ crates/tango/src/shapes.rs | 53 ++ docs/API_REFERENCE.md | 31 + docs/WEBHOOKS.md | 1 + 7 files changed, 1031 insertions(+), 3 deletions(-) create mode 100644 crates/tango/src/resources/sled.rs diff --git a/CHANGELOG.md b/CHANGELOG.md index f0155c7..21c7b92 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,25 @@ All notable changes to the `makegov-tango` and `makegov-tango-webhooks` crates a This project follows [Semantic Versioning](https://semver.org/). +## [Unreleased] + +### `makegov-tango` + +#### Added + +- **State, local and education (SLED) procurement** (Tango API 4.25.0; parity with tango-python, tango-node and tango-go). `resources/sled.rs` adds `list_sled_opportunities` / `get_sled_opportunity`, `list_sled_opportunity_revisions`, `get_sled_coverage`, `list_sled_forecasts` / `get_sled_forecast`, plus `iterate_sled_opportunities` and `iterate_sled_forecasts`. Four `bon`-derived options builders (`ListSledOpportunitiesOptions`, `ListSledOpportunityRevisionsOptions`, `ListSledForecastsOptions`, `GetSledOptions`) name every one of the API's 27 solicitation filters and 13 forecast filters, and five `SHAPE_SLED_*` constants land in `shapes.rs`. + + Four behaviors are documented on the options builders because each misleads a caller who assumes federal semantics. **Leaving both `status` and `active` unset returns open solicitations only** — that default is the API's, and `list_sled_opportunities` deliberately does not synthesize `status=open`, since doing so would make `active = Some(false)` unreachable (pinned by a test). **`status` is Tango-derived and refreshed every fifteen minutes**; the portal's own word is served as `source_status`, is frozen at last capture, and is not a liveness filter. **Category scheme tagging is mid-migration**, so `naics` matches only the small tagged share and `category_code` is the escape hatch. And **`meta.attachment_count` can be lower than the length of `attachments`**, because an auto-generated portal cover sheet is listed and flagged `is_generated_summary` but excluded from the count. + + `active`, `has_documents` and `source_declared` are `Option` rather than `bool`, so `false` reaches the server as a filter value instead of collapsing into the default — the same reason `push_opt_bool` exists. + + `SHAPE_SLED_REVISIONS_MINIMAL` omits `changes` on purpose: the per-field before/after needs a Small plan, so naming it in a suggested shape would 403 a Free caller. `changed_fields` is in the shape and available at every plan. + +#### Documentation + +- New **State & Local — SLED** section in `docs/API_REFERENCE.md` covering all six methods and both defaults that surprise people. +- `docs/WEBHOOKS.md` alert semantics gained the date-lapse rule and its one exception. An exclusion or DIBBS solicitation reaching its date fires nothing, because open/closed is derived at query time — but `alerts.sled_opportunity.match` **does** fire on a closing, since SLED liveness is a stored column a fifteen-minute sweep writes. + ## [0.1.0] — 2026-05-15 First public release of the Tango Rust SDK. diff --git a/crates/tango/src/lib.rs b/crates/tango/src/lib.rs index f1029f5..2c79466 100644 --- a/crates/tango/src/lib.rs +++ b/crates/tango/src/lib.rs @@ -124,9 +124,11 @@ pub use shapes::{ SHAPE_GSA_ELIBRARY_CONTRACTS_MINIMAL, SHAPE_IDVS_COMPREHENSIVE, SHAPE_IDVS_MINIMAL, SHAPE_ITDASHBOARD_INVESTMENTS_COMPREHENSIVE, SHAPE_ITDASHBOARD_INVESTMENTS_MINIMAL, SHAPE_NOTICES_MINIMAL, SHAPE_OPPORTUNITIES_MINIMAL, SHAPE_ORGANIZATIONS_MINIMAL, - SHAPE_OTAS_MINIMAL, SHAPE_OTIDVS_MINIMAL, SHAPE_PROTESTS_MINIMAL, SHAPE_SUBAWARDS_MINIMAL, - SHAPE_VEHICLES_COMPREHENSIVE, SHAPE_VEHICLES_MINIMAL, SHAPE_VEHICLE_AWARDEES_MINIMAL, - SHAPE_VEHICLE_ORDERS_MINIMAL, + SHAPE_OTAS_MINIMAL, SHAPE_OTIDVS_MINIMAL, SHAPE_PROTESTS_MINIMAL, + SHAPE_SLED_FORECASTS_COMPREHENSIVE, SHAPE_SLED_FORECASTS_MINIMAL, + SHAPE_SLED_OPPORTUNITIES_COMPREHENSIVE, SHAPE_SLED_OPPORTUNITIES_MINIMAL, + SHAPE_SLED_REVISIONS_MINIMAL, SHAPE_SUBAWARDS_MINIMAL, SHAPE_VEHICLES_COMPREHENSIVE, + SHAPE_VEHICLES_MINIMAL, SHAPE_VEHICLE_AWARDEES_MINIMAL, SHAPE_VEHICLE_ORDERS_MINIMAL, }; pub use transport::RateLimitInfo; pub use version::VERSION; diff --git a/crates/tango/src/resources/mod.rs b/crates/tango/src/resources/mod.rs index d8f073f..62755df 100644 --- a/crates/tango/src/resources/mod.rs +++ b/crates/tango/src/resources/mod.rs @@ -17,6 +17,7 @@ pub(crate) mod opportunities; pub(crate) mod otas; pub(crate) mod protests; pub(crate) mod resolve_validate; +pub(crate) mod sled; pub(crate) mod subawards; pub(crate) mod vehicle_subresources; pub(crate) mod vehicles; @@ -51,6 +52,10 @@ pub use otas::{ GetOTAOptions, GetOTIDVOptions, ListOTAsOptions, ListOTIDVAwardsOptions, ListOTIDVsOptions, }; pub use protests::{GetProtestOptions, ListProtestsOptions}; +pub use sled::{ + GetSledOptions, ListSledForecastsOptions, ListSledOpportunitiesOptions, + ListSledOpportunityRevisionsOptions, +}; pub use subawards::ListSubawardsOptions; pub use vehicle_subresources::{ListVehicleAwardeesOptions, ListVehicleOrdersOptions}; pub use vehicles::{GetVehicleOptions, ListVehiclesOptions}; diff --git a/crates/tango/src/resources/sled.rs b/crates/tango/src/resources/sled.rs new file mode 100644 index 0000000..694a44e --- /dev/null +++ b/crates/tango/src/resources/sled.rs @@ -0,0 +1,917 @@ +//! `GET /api/sled/opportunities/` and `GET /api/sled/forecasts/` — state, local +//! and education (SLED) procurement. +//! +//! **Beta.** Coverage is partial and grows one jurisdiction at a time. There is +//! no national SLED feed: every jurisdiction publishes on its own portal, and +//! Tango reads them one at a time. A thin per-state result is therefore at least +//! as likely to be a portal Tango does not read as a quiet market — +//! [`Client::get_sled_coverage`] is what resolves that ambiguity. +//! +//! This data does not join to the federal data. There is no UEI, no PIID, no +//! agency-hierarchy key and no NAICS/PSC crosswalk; the `organization(*)` expand +//! here is three strings, not the federal 7-key office payload. + +use crate::client::Client; +use crate::error::{Error, Result}; +use crate::internal::{apply_pagination, push_opt, push_opt_bool}; +use crate::pagination::{FetchFn, Page, PageStream}; +use crate::resources::agencies::urlencoding; +use crate::Record; +use bon::Builder; +use std::collections::BTreeMap; +use std::sync::Arc; + +/// Options for [`Client::list_sled_opportunities`] and +/// [`Client::iterate_sled_opportunities`]. +/// +/// Two things behave unlike the federal endpoints. +/// +/// Leaving both [`status`](Self::status) and [`active`](Self::active) unset +/// returns **open solicitations only** — the API defaults the list to +/// `status=open`, because only about a fifth of the corpus is open and portals +/// drop a closed solicitation rather than restating it. Set `status` explicitly +/// to page the whole corpus. [`Client::get_sled_opportunity`] returns a +/// solicitation whatever its status. +/// +/// `status` is Tango-derived liveness, refreshed every fifteen minutes. The +/// portal's own word is served as `source_status`, is frozen at last capture, +/// and is **not** filterable — most of what it calls open already has a passed +/// deadline. +#[derive(Debug, Clone, Default, Builder, PartialEq, Eq)] +#[non_exhaustive] +pub struct ListSledOpportunitiesOptions { + /// 1-based page number. + #[builder(into)] + pub page: Option, + /// Page size (server caps at 100). + #[builder(into)] + pub limit: Option, + /// Keyset cursor. + #[builder(into)] + pub cursor: Option, + /// Comma-separated field selector. Use + /// [`SHAPE_SLED_OPPORTUNITIES_MINIMAL`](crate::SHAPE_SLED_OPPORTUNITIES_MINIMAL) + /// or roll your own. + #[builder(into)] + pub shape: Option, + /// Collapse nested objects into dot-separated keys. + #[builder(default)] + pub flat: bool, + /// When [`flat`](Self::flat) is also true, flatten list-valued fields. + #[builder(default)] + pub flat_lists: bool, + + /// Two-letter state or territory code. Multi-value: `"TX|OK"`. + #[builder(into)] + pub state: Option, + /// Level of government: `"state"`, `"local"`, `"education"`, or + /// `"unknown"` for the aggregator rows that cannot tell state from local. + #[builder(into)] + pub jurisdiction: Option, + /// `"open"`, `"closed"`, `"awarded"`, `"cancelled"` or `"unknown"`. With + /// [`active`](Self::active) also unset the API returns open only; + /// `"unknown"` (standing rosters, dateless RFIs) is hidden by that default + /// and reachable with `"open|unknown"`. + #[builder(into)] + pub status: Option, + /// Sugar for federal-shaped callers: `true` is `status=open`, `false` is its + /// complement (so it includes `unknown`). `Option` so that `false` is + /// a real filter value rather than an absent one. + #[builder(into)] + pub active: Option, + /// Substring match on the buyer's published text (min 2 characters). No code + /// resolution behind it — state agencies have no entry in the federal + /// organization tree. + #[builder(into)] + pub agency: Option, + /// The number a human would quote. Null on roughly a third of the corpus, + /// where the portal publishes none. + #[builder(into)] + pub solicitation_number: Option, + /// `"rfp"`, `"ifb"`, `"rfq"`, `"rfi"`, `"itb"`, `"sole_source"`, `"grant"` + /// or `"other"`. `"null"` (the portal states no type) is a distinct answer + /// from `"other"` (a type the vocabulary does not recognize). + #[builder(into)] + pub solicitation_type: Option, + /// Whether the solicitation advertises at least one document. + #[builder(into)] + pub has_documents: Option, + /// Kind of the most recent substantive revision: `"deadline_change"`, + /// `"status_change"`, `"documents_added"`, `"documents_removed"`, + /// `"documents_replaced"`, `"title_change"` or `"content_change"`. + #[builder(into)] + pub revision_kind: Option, + + /// Exact match within the `naics` category scheme only. Thin on purpose: + /// scheme tagging is mid-migration, so only a small share of entries are + /// tagged NAICS. Prefer [`category_code`](Self::category_code) unless you + /// need scheme precision. + #[builder(into)] + pub naics: Option, + /// Exact match within the `nigp` scheme, the most widely tagged of the four. + #[builder(into)] + pub nigp: Option, + /// Exact match within the `unspsc` scheme. + #[builder(into)] + pub unspsc: Option, + /// Exact match within the `text` scheme, where the code is the portal's own + /// human label. + #[builder(into)] + pub category: Option, + /// Match a code under ANY scheme, including the untagged pre-migration + /// strings. The escape hatch when a scheme-specific filter returns less than + /// you expected. + #[builder(into)] + pub category_code: Option, + + /// Lower bound on `posted_date` (ISO `YYYY-MM-DD`, inclusive). + #[builder(into)] + pub posted_after: Option, + /// Upper bound on `posted_date` (inclusive). + #[builder(into)] + pub posted_before: Option, + /// Lower bound on `response_deadline` (inclusive). + #[builder(into)] + pub response_deadline_after: Option, + /// Upper bound on `response_deadline` (inclusive). + #[builder(into)] + pub response_deadline_before: Option, + /// Lower bound on when Tango FIRST OBSERVED the solicitation. The polling + /// primitive: everything new since your last call. + #[builder(into)] + pub first_seen_after: Option, + /// Upper bound on first observation (inclusive). + #[builder(into)] + pub first_seen_before: Option, + /// Lower bound on when Tango OBSERVED the last substantive change. A scrape + /// date, not an amendment date: a state portal restates one page in place + /// and publishes no amendment date, so the resolution is that state's crawl + /// cadence. The API exposes no upper bound on this field. + #[builder(into)] + pub change_seen_after: Option, + /// Lower bound on when the Tango row last changed (inclusive). + #[builder(into)] + pub modified_after: Option, + /// Upper bound on when the Tango row last changed (inclusive). + #[builder(into)] + pub modified_before: Option, + + /// Support filter identifying the source portal's platform family. In no + /// response shape, and not a stable value. + #[builder(into)] + pub platform: Option, + /// Support filter — the portal's own identifier. + #[builder(into)] + pub native_id: Option, + /// Support filter — Tango's opaque lake key. At most 500 values. + #[builder(into)] + pub external_id: Option, + + /// Ranked full-text search over title, agency, identifiers, category labels + /// and description, widened by the solicitations whose ATTACHMENT text + /// matched (min 2 characters). A row that matched on its description gains a + /// `snippet` field; a title-or-agency match carries none. + #[builder(into)] + pub search: Option, + /// One of `rank`, `response_deadline`, `posted_date`, `first_seen_at`, + /// `last_seen_at`, `last_change_seen_at`, `modified`. `rank` requires a + /// non-empty [`search`](Self::search). + #[builder(into)] + pub ordering: Option, + + /// Escape hatch for filter keys not yet first-classed on this struct. + #[builder(default)] + pub extra: BTreeMap, +} + +impl ListSledOpportunitiesOptions { + pub(crate) fn to_query(&self) -> Vec<(String, String)> { + let mut q = Vec::new(); + apply_pagination( + &mut q, + self.page, + self.limit, + self.cursor.as_deref(), + self.shape.as_deref(), + self.flat, + self.flat_lists, + ); + push_opt(&mut q, "state", self.state.as_deref()); + push_opt(&mut q, "jurisdiction", self.jurisdiction.as_deref()); + push_opt(&mut q, "status", self.status.as_deref()); + push_opt_bool(&mut q, "active", self.active); + push_opt(&mut q, "agency", self.agency.as_deref()); + push_opt( + &mut q, + "solicitation_number", + self.solicitation_number.as_deref(), + ); + push_opt( + &mut q, + "solicitation_type", + self.solicitation_type.as_deref(), + ); + push_opt_bool(&mut q, "has_documents", self.has_documents); + push_opt(&mut q, "revision_kind", self.revision_kind.as_deref()); + push_opt(&mut q, "naics", self.naics.as_deref()); + push_opt(&mut q, "nigp", self.nigp.as_deref()); + push_opt(&mut q, "unspsc", self.unspsc.as_deref()); + push_opt(&mut q, "category", self.category.as_deref()); + push_opt(&mut q, "category_code", self.category_code.as_deref()); + push_opt(&mut q, "posted_after", self.posted_after.as_deref()); + push_opt(&mut q, "posted_before", self.posted_before.as_deref()); + push_opt( + &mut q, + "response_deadline_after", + self.response_deadline_after.as_deref(), + ); + push_opt( + &mut q, + "response_deadline_before", + self.response_deadline_before.as_deref(), + ); + push_opt(&mut q, "first_seen_after", self.first_seen_after.as_deref()); + push_opt( + &mut q, + "first_seen_before", + self.first_seen_before.as_deref(), + ); + push_opt( + &mut q, + "change_seen_after", + self.change_seen_after.as_deref(), + ); + push_opt(&mut q, "modified_after", self.modified_after.as_deref()); + push_opt(&mut q, "modified_before", self.modified_before.as_deref()); + push_opt(&mut q, "platform", self.platform.as_deref()); + push_opt(&mut q, "native_id", self.native_id.as_deref()); + push_opt(&mut q, "external_id", self.external_id.as_deref()); + push_opt(&mut q, "search", self.search.as_deref()); + push_opt(&mut q, "ordering", self.ordering.as_deref()); + for (k, v) in &self.extra { + if !v.is_empty() { + q.push((k.clone(), v.clone())); + } + } + q + } +} + +/// Options for [`Client::list_sled_opportunity_revisions`], the nested +/// `/revisions/` route. +#[derive(Debug, Clone, Default, Builder, PartialEq, Eq)] +#[non_exhaustive] +pub struct ListSledOpportunityRevisionsOptions { + /// 1-based page number. + #[builder(into)] + pub page: Option, + /// Page size (server caps at 100). + #[builder(into)] + pub limit: Option, + /// Keyset cursor. + #[builder(into)] + pub cursor: Option, + /// Comma-separated field selector. Use + /// [`SHAPE_SLED_REVISIONS_MINIMAL`](crate::SHAPE_SLED_REVISIONS_MINIMAL), + /// which omits the plan-gated `changes` leaf. + #[builder(into)] + pub shape: Option, + /// Collapse nested objects into dot-separated keys. + #[builder(default)] + pub flat: bool, + /// When [`flat`](Self::flat) is also true, flatten list-valued fields. + #[builder(default)] + pub flat_lists: bool, + + /// A revision kind, plus `"enrichment"` — which the `revisions(*)` expand + /// excludes and this route serves. Multi-value: use `|`. + #[builder(into)] + pub kind: Option, + /// Whether a portal's own amendment marker moved at this emission. True on + /// about 5% of revisions; everything else is Tango inferring the change from + /// the diff. + #[builder(into)] + pub source_declared: Option, + /// Lower bound on `observed_at` (ISO `YYYY-MM-DD`, inclusive). + #[builder(into)] + pub observed_after: Option, + /// Upper bound on `observed_at` (inclusive). + #[builder(into)] + pub observed_before: Option, + + /// Escape hatch for filter keys not yet first-classed on this struct. + #[builder(default)] + pub extra: BTreeMap, +} + +impl ListSledOpportunityRevisionsOptions { + pub(crate) fn to_query(&self) -> Vec<(String, String)> { + let mut q = Vec::new(); + apply_pagination( + &mut q, + self.page, + self.limit, + self.cursor.as_deref(), + self.shape.as_deref(), + self.flat, + self.flat_lists, + ); + push_opt(&mut q, "kind", self.kind.as_deref()); + push_opt_bool(&mut q, "source_declared", self.source_declared); + push_opt(&mut q, "observed_after", self.observed_after.as_deref()); + push_opt(&mut q, "observed_before", self.observed_before.as_deref()); + for (k, v) in &self.extra { + if !v.is_empty() { + q.push((k.clone(), v.clone())); + } + } + q + } +} + +/// Options for [`Client::list_sled_forecasts`] and +/// [`Client::iterate_sled_forecasts`]. +/// +/// Forecasts carry no liveness at all — there is no deadline to have passed, so +/// there is no `status` field, no `active` field, and no open-only default. +/// Currency is the caller's call from `estimated_advertisement_date`, which is +/// the START of the published quarter rather than a posting date. +#[derive(Debug, Clone, Default, Builder, PartialEq, Eq)] +#[non_exhaustive] +pub struct ListSledForecastsOptions { + /// 1-based page number. + #[builder(into)] + pub page: Option, + /// Page size (server caps at 100). + #[builder(into)] + pub limit: Option, + /// Keyset cursor. + #[builder(into)] + pub cursor: Option, + /// Comma-separated field selector. Use + /// [`SHAPE_SLED_FORECASTS_MINIMAL`](crate::SHAPE_SLED_FORECASTS_MINIMAL) + /// or roll your own. + #[builder(into)] + pub shape: Option, + /// Collapse nested objects into dot-separated keys. + #[builder(default)] + pub flat: bool, + /// When [`flat`](Self::flat) is also true, flatten list-valued fields. + #[builder(default)] + pub flat_lists: bool, + + /// Two-letter state code. Multi-value: use `|`. + #[builder(into)] + pub state: Option, + /// Substring match on the buyer's published text (min 2 characters). + #[builder(into)] + pub agency: Option, + /// The portal's own category word, passed through verbatim. + #[builder(into)] + pub procurement_category: Option, + /// The portal's own method word, passed through verbatim. + #[builder(into)] + pub procurement_method: Option, + /// The number the state expects to award under, where it publishes one in + /// advance. + #[builder(into)] + pub contract_number: Option, + /// Substring match on the incumbent vendor's name as published. NOT resolved + /// to a Tango entity. + #[builder(into)] + pub incumbent_name: Option, + /// Lower bound on the estimated advertisement date (inclusive). Remember the + /// date is a QUARTER START, not a posting date. + #[builder(into)] + pub advertisement_after: Option, + /// Upper bound on the estimated advertisement date (inclusive). + #[builder(into)] + pub advertisement_before: Option, + /// Lower bound on when Tango first observed the forecast (inclusive). + #[builder(into)] + pub first_seen_after: Option, + /// Upper bound on first observation (inclusive). + #[builder(into)] + pub first_seen_before: Option, + /// Lower bound on when the Tango row last changed (inclusive). + #[builder(into)] + pub modified_after: Option, + /// Upper bound on when the Tango row last changed (inclusive). + #[builder(into)] + pub modified_before: Option, + /// Ranked full-text search over title, agency and description (min 2 + /// characters). + #[builder(into)] + pub search: Option, + /// One of `rank`, `estimated_advertisement_date`, `first_seen_at`, + /// `last_seen_at`, `modified`. `rank` requires a non-empty + /// [`search`](Self::search). + #[builder(into)] + pub ordering: Option, + + /// Escape hatch for filter keys not yet first-classed on this struct. + #[builder(default)] + pub extra: BTreeMap, +} + +impl ListSledForecastsOptions { + pub(crate) fn to_query(&self) -> Vec<(String, String)> { + let mut q = Vec::new(); + apply_pagination( + &mut q, + self.page, + self.limit, + self.cursor.as_deref(), + self.shape.as_deref(), + self.flat, + self.flat_lists, + ); + push_opt(&mut q, "state", self.state.as_deref()); + push_opt(&mut q, "agency", self.agency.as_deref()); + push_opt( + &mut q, + "procurement_category", + self.procurement_category.as_deref(), + ); + push_opt( + &mut q, + "procurement_method", + self.procurement_method.as_deref(), + ); + push_opt(&mut q, "contract_number", self.contract_number.as_deref()); + push_opt(&mut q, "incumbent_name", self.incumbent_name.as_deref()); + push_opt( + &mut q, + "advertisement_after", + self.advertisement_after.as_deref(), + ); + push_opt( + &mut q, + "advertisement_before", + self.advertisement_before.as_deref(), + ); + push_opt(&mut q, "first_seen_after", self.first_seen_after.as_deref()); + push_opt( + &mut q, + "first_seen_before", + self.first_seen_before.as_deref(), + ); + push_opt(&mut q, "modified_after", self.modified_after.as_deref()); + push_opt(&mut q, "modified_before", self.modified_before.as_deref()); + push_opt(&mut q, "search", self.search.as_deref()); + push_opt(&mut q, "ordering", self.ordering.as_deref()); + for (k, v) in &self.extra { + if !v.is_empty() { + q.push((k.clone(), v.clone())); + } + } + q + } +} + +/// Options for [`Client::get_sled_opportunity`] and +/// [`Client::get_sled_forecast`]. +#[derive(Debug, Clone, Default, Builder, PartialEq, Eq)] +#[non_exhaustive] +pub struct GetSledOptions { + /// Shape selector. When empty, the server returns its default detail shape. + #[builder(into)] + pub shape: Option, + /// Flatten nested objects into dot-separated keys. + #[builder(default)] + pub flat: bool, + /// When `flat=true`, also flatten list-valued nested fields. + #[builder(default)] + pub flat_lists: bool, +} + +impl GetSledOptions { + pub(crate) fn to_query(&self) -> Vec<(String, String)> { + let mut q = Vec::new(); + push_opt(&mut q, "shape", self.shape.as_deref()); + if self.flat { + q.push(("flat".into(), "true".into())); + } + if self.flat_lists { + q.push(("flat_lists".into(), "true".into())); + } + q + } +} + +impl Client { + /// `GET /api/sled/opportunities/` — one page of state, local and education + /// solicitations. + /// + /// Leaving both `status` and `active` unset returns open solicitations only. + /// That default is the API's, and this method deliberately does not + /// synthesize one: sending `status=open` here would make `active = false` + /// unreachable, since it is the complement of open rather than an + /// independent value. + pub async fn list_sled_opportunities( + &self, + opts: ListSledOpportunitiesOptions, + ) -> Result> { + let q = opts.to_query(); + let bytes = self.get_bytes("/api/sled/opportunities/", &q).await?; + Page::decode(&bytes) + } + + /// `GET /api/sled/opportunities/{opportunity_id}/` — one solicitation, + /// whatever its status. The open-only default applies to the list endpoint, + /// not here. + pub async fn get_sled_opportunity( + &self, + opportunity_id: &str, + opts: Option, + ) -> Result { + if opportunity_id.is_empty() { + return Err(Error::Validation { + message: "get_sled_opportunity: opportunity_id is required".into(), + response: None, + }); + } + let q = opts.unwrap_or_default().to_query(); + let path = format!("/api/sled/opportunities/{}/", urlencoding(opportunity_id)); + self.get_json::(&path, &q).await + } + + /// `GET /api/sled/opportunities/{opportunity_id}/revisions/` — one + /// solicitation's observed revision history. + /// + /// `observed_at` is the scrape that saw the change, not the date the agency + /// made it: no state portal emits amendment notices, so `kind` is Tango's + /// inference from the diff on about 95% of revisions, resolution is that + /// state's crawl cadence, and history starts when Tango began reading the + /// jurisdiction rather than when the solicitation was posted. + /// + /// Unlike the `revisions(*)` expand, this route serves `enrichment` rows — + /// Tango's own detail fetch filling in coverage rather than an agency + /// amendment. Pass `kind = "enrichment"` for only those. The per-field + /// before/after (`changes`) requires a Small plan; `changed_fields` names + /// what moved at every plan. + pub async fn list_sled_opportunity_revisions( + &self, + opportunity_id: &str, + opts: ListSledOpportunityRevisionsOptions, + ) -> Result> { + if opportunity_id.is_empty() { + return Err(Error::Validation { + message: "list_sled_opportunity_revisions: opportunity_id is required".into(), + response: None, + }); + } + let q = opts.to_query(); + let path = format!( + "/api/sled/opportunities/{}/revisions/", + urlencoding(opportunity_id) + ); + let bytes = self.get_bytes(&path, &q).await?; + Page::decode(&bytes) + } + + /// `GET /api/sled/opportunities/coverage/` — the per-state coverage rollup. + /// + /// Returns corpus totals plus one row per jurisdiction: the total, the count + /// in each of the five statuses, the jurisdiction levels present, and when a + /// solicitation there last changed. It answers one question — whether a thin + /// result for a state is a thin market or a portal Tango does not read. + /// + /// Every state row carries all five status buckets whether or not they have + /// rows, so a total and two buckets never invite subtraction. Takes no + /// parameters and is neither shaped nor paginated. + pub async fn get_sled_coverage(&self) -> Result { + self.get_json::("/api/sled/opportunities/coverage/", &[]) + .await + } + + /// `GET /api/sled/forecasts/` — one page of planned state procurements. + pub async fn list_sled_forecasts( + &self, + opts: ListSledForecastsOptions, + ) -> Result> { + let q = opts.to_query(); + let bytes = self.get_bytes("/api/sled/forecasts/", &q).await?; + Page::decode(&bytes) + } + + /// `GET /api/sled/forecasts/{forecast_id}/` — one forecast. + pub async fn get_sled_forecast( + &self, + forecast_id: &str, + opts: Option, + ) -> Result { + if forecast_id.is_empty() { + return Err(Error::Validation { + message: "get_sled_forecast: forecast_id is required".into(), + response: None, + }); + } + let q = opts.unwrap_or_default().to_query(); + let path = format!("/api/sled/forecasts/{}/", urlencoding(forecast_id)); + self.get_json::(&path, &q).await + } + + /// Stream every SLED solicitation matching `opts`. + pub fn iterate_sled_opportunities( + &self, + opts: ListSledOpportunitiesOptions, + ) -> PageStream { + let opts = Arc::new(opts); + let fetch: FetchFn = Box::new(move |client, page, cursor| { + let mut next = (*opts).clone(); + next.page = page; + next.cursor = cursor; + Box::pin(async move { client.list_sled_opportunities(next).await }) + }); + PageStream::new(self.clone(), fetch) + } + + /// Stream every SLED forecast matching `opts`. + pub fn iterate_sled_forecasts(&self, opts: ListSledForecastsOptions) -> PageStream { + let opts = Arc::new(opts); + let fetch: FetchFn = Box::new(move |client, page, cursor| { + let mut next = (*opts).clone(); + next.page = page; + next.cursor = cursor; + Box::pin(async move { client.list_sled_forecasts(next).await }) + }); + PageStream::new(self.clone(), fetch) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn get_q(q: &[(String, String)], k: &str) -> Option { + q.iter().find(|(kk, _)| kk == k).map(|(_, v)| v.clone()) + } + + #[test] + fn list_sled_opportunities_all_filters_emit() { + let opts = ListSledOpportunitiesOptions::builder() + .state("TX|OK") + .jurisdiction("local|education") + .status("open|unknown") + .active(true) + .agency("Texas Commission") + .solicitation_number("RFP-2026-001") + .solicitation_type("rfp") + .has_documents(true) + .revision_kind("deadline_change") + .naics("541620") + .nigp("962-47") + .unspsc("77101500") + .category("Environmental Services") + .category_code("541620") + .posted_after("2026-01-01") + .posted_before("2026-12-31") + .response_deadline_after("2026-09-01") + .response_deadline_before("2026-10-01") + .first_seen_after("2026-09-01") + .first_seen_before("2026-09-30") + .change_seen_after("2026-09-05") + .modified_after("2026-09-01") + .modified_before("2026-09-30") + .platform("custom") + .native_id("abc-123") + .external_id("tx:custom:abc-123") + .search("environmental mitigation") + .ordering("response_deadline") + .build(); + let q = opts.to_query(); + for (key, want) in [ + ("state", "TX|OK"), + ("jurisdiction", "local|education"), + ("status", "open|unknown"), + ("active", "true"), + ("agency", "Texas Commission"), + ("solicitation_number", "RFP-2026-001"), + ("solicitation_type", "rfp"), + ("has_documents", "true"), + ("revision_kind", "deadline_change"), + ("naics", "541620"), + ("nigp", "962-47"), + ("unspsc", "77101500"), + ("category", "Environmental Services"), + ("category_code", "541620"), + ("posted_after", "2026-01-01"), + ("posted_before", "2026-12-31"), + ("response_deadline_after", "2026-09-01"), + ("response_deadline_before", "2026-10-01"), + ("first_seen_after", "2026-09-01"), + ("first_seen_before", "2026-09-30"), + ("change_seen_after", "2026-09-05"), + ("modified_after", "2026-09-01"), + ("modified_before", "2026-09-30"), + ("platform", "custom"), + ("native_id", "abc-123"), + ("external_id", "tx:custom:abc-123"), + ("search", "environmental mitigation"), + ("ordering", "response_deadline"), + ] { + assert_eq!(get_q(&q, key).as_deref(), Some(want), "param {key}"); + } + } + + /// The open-only default is the API's. Synthesizing `status=open` here would + /// make `active = false` unreachable, since it is the complement of open. + #[test] + fn no_liveness_filter_is_sent_when_none_requested() { + let opts = ListSledOpportunitiesOptions::builder().state("TX").build(); + let q = opts.to_query(); + assert_eq!(get_q(&q, "state").as_deref(), Some("TX")); + assert_eq!(get_q(&q, "status"), None); + assert_eq!(get_q(&q, "active"), None); + } + + #[test] + fn active_false_is_sent_not_dropped() { + let opts = ListSledOpportunitiesOptions::builder() + .active(false) + .build(); + assert_eq!(get_q(&opts.to_query(), "active").as_deref(), Some("false")); + } + + #[test] + fn has_documents_false_is_sent_not_dropped() { + let opts = ListSledOpportunitiesOptions::builder() + .has_documents(false) + .build(); + assert_eq!( + get_q(&opts.to_query(), "has_documents").as_deref(), + Some("false") + ); + } + + /// `status` is Tango-derived liveness; the portal's frozen word is served but + /// not filterable. + #[test] + fn source_status_is_not_a_filter() { + let opts = ListSledOpportunitiesOptions::builder() + .status("closed") + .build(); + let q = opts.to_query(); + assert_eq!(get_q(&q, "status").as_deref(), Some("closed")); + assert_eq!(get_q(&q, "source_status"), None); + } + + #[test] + fn list_sled_opportunities_zero_value_omitted() { + let opts = ListSledOpportunitiesOptions::builder().build(); + let q = opts.to_query(); + assert!(q.is_empty(), "expected empty query, got {q:?}"); + } + + #[test] + fn list_sled_opportunities_extra_emits() { + let mut extra = BTreeMap::new(); + extra.insert("verbose".to_string(), "true".to_string()); + let opts = ListSledOpportunitiesOptions::builder().extra(extra).build(); + assert!(opts.to_query().contains(&("verbose".into(), "true".into()))); + } + + #[test] + fn revision_filters_emit() { + let opts = ListSledOpportunityRevisionsOptions::builder() + .kind("deadline_change") + .source_declared(true) + .observed_after("2026-09-01") + .observed_before("2026-09-30") + .build(); + let q = opts.to_query(); + assert_eq!(get_q(&q, "kind").as_deref(), Some("deadline_change")); + assert_eq!(get_q(&q, "source_declared").as_deref(), Some("true")); + assert_eq!(get_q(&q, "observed_after").as_deref(), Some("2026-09-01")); + assert_eq!(get_q(&q, "observed_before").as_deref(), Some("2026-09-30")); + } + + /// The `revisions(*)` expand excludes enrichment rows; this route is how a + /// caller reaches them. + #[test] + fn revision_kind_reaches_enrichment() { + let opts = ListSledOpportunityRevisionsOptions::builder() + .kind("enrichment") + .build(); + assert_eq!( + get_q(&opts.to_query(), "kind").as_deref(), + Some("enrichment") + ); + } + + /// `changes` needs a Small plan, so naming it in the suggested revision shape + /// would 403 a Free caller on a field they never asked to gate. + #[test] + fn revisions_shape_omits_plan_gated_changes() { + let shape = crate::SHAPE_SLED_REVISIONS_MINIMAL; + assert!( + !shape.split(',').any(|f| f == "changes"), + "SHAPE_SLED_REVISIONS_MINIMAL must not name the Small-gated `changes` leaf: {shape}" + ); + assert!( + shape.contains("changed_fields"), + "SHAPE_SLED_REVISIONS_MINIMAL should carry `changed_fields`, which every plan can read: {shape}" + ); + } + + #[test] + fn list_sled_forecasts_filters_emit() { + let opts = ListSledForecastsOptions::builder() + .state("MD") + .agency("Department of Transportation") + .procurement_category("Services") + .procurement_method("Competitive Sealed Proposals") + .contract_number("K-2026-001") + .incumbent_name("Acme") + .advertisement_after("2026-10-01") + .advertisement_before("2027-03-31") + .first_seen_after("2026-09-01") + .first_seen_before("2026-09-30") + .modified_after("2026-09-01") + .modified_before("2026-09-30") + .search("data center") + .ordering("estimated_advertisement_date") + .build(); + let q = opts.to_query(); + assert_eq!(get_q(&q, "state").as_deref(), Some("MD")); + assert_eq!( + get_q(&q, "procurement_method").as_deref(), + Some("Competitive Sealed Proposals") + ); + assert_eq!( + get_q(&q, "advertisement_after").as_deref(), + Some("2026-10-01") + ); + assert_eq!(get_q(&q, "search").as_deref(), Some("data center")); + // A forecast has no deadline, so there is no liveness filter to send. + assert_eq!(get_q(&q, "status"), None); + assert_eq!(get_q(&q, "active"), None); + } + + #[test] + fn get_sled_options_emit() { + let opts = GetSledOptions::builder() + .shape(crate::SHAPE_SLED_OPPORTUNITIES_COMPREHENSIVE) + .flat(true) + .flat_lists(true) + .build(); + let q = opts.to_query(); + assert_eq!( + get_q(&q, "shape").as_deref(), + Some(crate::SHAPE_SLED_OPPORTUNITIES_COMPREHENSIVE) + ); + assert_eq!(get_q(&q, "flat").as_deref(), Some("true")); + assert_eq!(get_q(&q, "flat_lists").as_deref(), Some("true")); + } + + #[test] + fn cursor_wins_over_page() { + let opts = ListSledOpportunitiesOptions::builder() + .page(2u32) + .cursor("xyz".to_string()) + .build(); + let q = opts.to_query(); + assert_eq!(get_q(&q, "cursor").as_deref(), Some("xyz")); + assert_eq!(get_q(&q, "page"), None); + } + + #[tokio::test] + async fn get_sled_opportunity_validates_empty_id() { + let client = Client::builder().api_key("x").build().expect("client"); + let err = client.get_sled_opportunity("", None).await.unwrap_err(); + match err { + Error::Validation { message, .. } => { + assert!(message.contains("opportunity_id is required")); + } + other => panic!("expected Validation, got {other:?}"), + } + } + + #[tokio::test] + async fn list_sled_opportunity_revisions_validates_empty_id() { + let client = Client::builder().api_key("x").build().expect("client"); + let err = client + .list_sled_opportunity_revisions("", ListSledOpportunityRevisionsOptions::default()) + .await + .unwrap_err(); + match err { + Error::Validation { message, .. } => { + assert!(message.contains("opportunity_id is required")); + } + other => panic!("expected Validation, got {other:?}"), + } + } + + #[tokio::test] + async fn get_sled_forecast_validates_empty_id() { + let client = Client::builder().api_key("x").build().expect("client"); + let err = client.get_sled_forecast("", None).await.unwrap_err(); + match err { + Error::Validation { message, .. } => { + assert!(message.contains("forecast_id is required")); + } + other => panic!("expected Validation, got {other:?}"), + } + } +} diff --git a/crates/tango/src/shapes.rs b/crates/tango/src/shapes.rs index a86414b..e4d924f 100644 --- a/crates/tango/src/shapes.rs +++ b/crates/tango/src/shapes.rs @@ -134,3 +134,56 @@ pub const SHAPE_ITDASHBOARD_INVESTMENTS_COMPREHENSIVE: &str = concat!( "investment_title,type_of_investment,part_of_it_portfolio,", "updated_time,url", ); + +/// Suggested list shape for +/// [`Client::list_sled_opportunities`](crate::Client::list_sled_opportunities). +/// +/// `description` is detail-only on the API — its median is around 550 +/// characters and its tail runs past 120,000 — so it is deliberately absent +/// here. Name it explicitly, or pass `verbose=true` via `extra`. +pub const SHAPE_SLED_OPPORTUNITIES_MINIMAL: &str = concat!( + "opportunity_id,solicitation_number,solicitation_type,title,state,", + "jurisdiction,agency,status,status_reason,posted_date,response_deadline,", + "source_url,has_documents,first_seen_at,last_change_seen_at", +); + +/// Suggested detail shape for +/// [`Client::get_sled_opportunity`](crate::Client::get_sled_opportunity). +pub const SHAPE_SLED_OPPORTUNITIES_COMPREHENSIVE: &str = concat!( + "opportunity_id,solicitation_number,solicitation_type,", + "solicitation_type_source,title,description,state,jurisdiction,agency,", + "status,status_reason,status_computed_at,source_status,source_url,", + "posted_date,response_deadline,response_deadline_original,", + "bid_opening_date,bid_opening_raw,category_codes,has_documents,", + "first_seen_at,last_seen_at,last_change_seen_at,", + "organization(*),contact(*),meta(*),attachments(*),revisions(*)", +); + +/// Suggested shape for +/// [`Client::list_sled_opportunity_revisions`](crate::Client::list_sled_opportunity_revisions). +/// +/// `changes` — the per-field before and after — is omitted on purpose: it needs +/// a Small plan, so naming it in a default shape would 403 a Free caller on a +/// field they never asked to gate. `changed_fields` names what moved at every +/// plan. +pub const SHAPE_SLED_REVISIONS_MINIMAL: &str = + "observed_at,sequence,kind,changed_fields,source_declared"; + +/// Suggested list shape for +/// [`Client::list_sled_forecasts`](crate::Client::list_sled_forecasts). +pub const SHAPE_SLED_FORECASTS_MINIMAL: &str = concat!( + "forecast_id,state,agency,title,estimated_advertisement_date,", + "estimated_advertisement_raw,procurement_category,procurement_method,", + "contract_number,incumbent_name,source_url,estimated_value(*)", +); + +/// Suggested detail shape for +/// [`Client::get_sled_forecast`](crate::Client::get_sled_forecast). +pub const SHAPE_SLED_FORECASTS_COMPREHENSIVE: &str = concat!( + "forecast_id,state,agency,title,description,", + "estimated_advertisement_date,estimated_advertisement_raw,", + "procurement_category,procurement_method,contract_term,contract_number,", + "incumbent_name,mbe_dbe_goal,delivery_location,source_url,source_status,", + "has_documents,first_seen_at,last_seen_at,", + "organization(*),contact(*),estimated_value(*)", +); diff --git a/docs/API_REFERENCE.md b/docs/API_REFERENCE.md index 1db1958..543038a 100644 --- a/docs/API_REFERENCE.md +++ b/docs/API_REFERENCE.md @@ -126,6 +126,37 @@ Options: `ListGsaElibraryContractsOptions`, `GetGsaElibraryContractOptions`. Options: `ListProtestsOptions`, `GetProtestOptions`. No `ordering` (server rejects it for this resource). +### State & Local — SLED (`sled.rs`) — **Beta** + +State, local and education procurement: solicitations that never appear on SAM.gov because they were never federal. Coverage is partial and grows one jurisdiction at a time. + +| Method | Endpoint | Returns | +| ------ | -------- | ------- | +| `list_sled_opportunities(opts)` / `iterate_sled_opportunities(opts)` | `GET /api/sled/opportunities/` | `Page` / `PageStream` | +| `get_sled_opportunity(id, opts)` | `GET /api/sled/opportunities/{opportunity_id}/` | `Record` | +| `list_sled_opportunity_revisions(id, opts)` | `GET /api/sled/opportunities/{opportunity_id}/revisions/` | `Page` | +| `get_sled_coverage()` | `GET /api/sled/opportunities/coverage/` | `Record` | +| `list_sled_forecasts(opts)` / `iterate_sled_forecasts(opts)` | `GET /api/sled/forecasts/` | `Page` / `PageStream` | +| `get_sled_forecast(id, opts)` | `GET /api/sled/forecasts/{forecast_id}/` | `Record` | + +Options: `ListSledOpportunitiesOptions`, `ListSledOpportunityRevisionsOptions`, `ListSledForecastsOptions`, `GetSledOptions`. Shapes: `SHAPE_SLED_OPPORTUNITIES_MINIMAL` / `_COMPREHENSIVE`, `SHAPE_SLED_REVISIONS_MINIMAL`, `SHAPE_SLED_FORECASTS_MINIMAL` / `_COMPREHENSIVE`. + +**This data does not join to the federal data.** No UEI, no PIID, no agency-hierarchy key and no NAICS/PSC crosswalk; the `organization(*)` expand here is three strings, not the federal 7-key office payload. + +**Leaving both `status` and `active` unset returns open solicitations only.** Only about a fifth of the corpus is open, and a portal drops a closed solicitation rather than restating it, so the API defaults the list to `status=open`. Set `status` explicitly to page the whole corpus; `status = "open|unknown"` also reaches the standing rosters and dateless RFIs that `unknown` covers. `get_sled_opportunity` returns a solicitation whatever its status. + +`list_sled_opportunities` deliberately does **not** synthesize `status=open` client-side: doing so would make `active = Some(false)` unreachable, since `active=false` is the complement of open rather than an independent value. `active`, `has_documents` and `source_declared` are `Option` for the same reason — `false` has to be distinguishable from unset. + +**`status` is Tango's answer, not the portal's.** Derived from the portal's word, the deadline and the clock, and refreshed every fifteen minutes. The portal's own word is served as `source_status`, is frozen at last capture, and is **not** filterable — most of what it calls open already has a passed deadline. + +**Category scheme tagging is mid-migration**, so `naics` matches only the small tagged share. Use `category_code` to match a code under any scheme, including the untagged pre-migration strings. + +**`meta.attachment_count` can be lower than the length of `attachments`.** Some portals auto-generate a cover sheet alongside the real documents; it is listed and flagged `is_generated_summary` but excluded from the count and from `has_documents`. The count answers "does this record hold its solicitation package"; the array answers "what files exist". Attachment bodies are never served as a field. + +`get_sled_coverage` takes no parameters and is neither shaped nor paginated. **Call it before treating a per-state count as market size** — a thin result for a state is at least as likely to be a portal Tango does not read as a quiet market, and every state row carries all five status buckets whether or not they have rows. + +Forecasts carry **no liveness at all**: no deadline to have passed, so no `status`, no `active`, and no open-only default. `estimated_advertisement_date` is the start of the published quarter rather than a posting date, and `estimated_value(min,max,raw)` is parsed from a free-text award band — a band naming one number is a floor, so `max` is null. + ### IT Dashboard (`itdashboard.rs`) | Method | Endpoint | Returns | diff --git a/docs/WEBHOOKS.md b/docs/WEBHOOKS.md index 2f28752..7c7aa53 100644 --- a/docs/WEBHOOKS.md +++ b/docs/WEBHOOKS.md @@ -202,6 +202,7 @@ Client-side validation rejects empty `name` or `callback_url` on `create_webhook - `filters` must be a non-empty JSON object. Client-side validation rejects `null` or `{}`. - `endpoint` is required when the account has multiple endpoints; single-endpoint accounts can omit it (the server auto-resolves). - `query_type` and `filters` are **read-only after creation**; `update_webhook_alert` only allows `name`, `frequency`, `cron_expression`, `is_active`. +- **A record merely reaching its date fires nothing — except `sled_opportunity`.** An exclusion passing its termination date, or a DIBBS solicitation passing its close date, emits no event: open/closed and in-force are derived at query time, so no stored row changes. `sled_opportunity` is the one exception — a state solicitation's liveness is a stored `status` column Tango recomputes every fifteen minutes rather than deriving per request, so a deadline passing **is** a write and `alerts.sled_opportunity.match` can follow it. There is no `sled_forecast` query type (a forecast has no deadline, so nothing transitions), and SLED revisions and attachments are not separately alertable: subscribe to `sled_opportunity` and filter on `change_seen_after` or `revision_kind`. ### Example: create and test an endpoint From ade9f12f6e9383968e31744b629794254258171c Mon Sep 17 00:00:00 2001 From: "V. David Zvenyach" Date: Thu, 10 Sep 2026 09:41:23 -0500 Subject: [PATCH 2/2] docs(sled): say that the document body is a paid, opt-in leaf MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `attachments(extracted_text)` serves a SLED document body on a Small plan or above from API 4.25.1. This SDK returns `Record`, so the leaf needs no type change — what it needed was saying so, in the three places a caller looks. The leaf must be named: no `SHAPE_SLED_*` constant includes it and `attachments(*)` does not carry it, because the API resolves the body only for a caller who asked. A constant naming it would make every detail fetch pay for a document nobody wanted to read, so `suggested_shapes_do_not_name_the_paid_document_body` pins both against it. The key is absent rather than null whenever the text is not being served — below Small it is withheld and named in `meta.upgrade_hints`. A contested document never returns text at any plan, because its stored bytes disagree with what the record advertised. Searching document text and reading it stay separate: `search` matches inside attachment text on every plan and returns no fragment of it. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 2 ++ crates/tango/src/resources/sled.rs | 33 ++++++++++++++++++++++++++++++ crates/tango/src/shapes.rs | 9 ++++++++ docs/API_REFERENCE.md | 4 +++- 4 files changed, 47 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 21c7b92..05d345c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,8 @@ This project follows [Semantic Versioning](https://semver.org/). #### Added +- **`attachments(extracted_text)` — SLED document bodies on the Small plan and above** (Tango API 4.25.1; parity with tango-python, tango-node and tango-go). This SDK returns `Record`, so the leaf needs no type change — what it needed was saying so. Documented on `get_sled_opportunity`, on `SHAPE_SLED_OPPORTUNITIES_COMPREHENSIVE` and in `docs/API_REFERENCE.md`: the leaf must be **named** (no `SHAPE_SLED_*` constant includes it, and `attachments(*)` does not carry it, because the API resolves the body only for a caller who asked); the **key is absent rather than null** when the text is not being served; and a **contested document never returns text at any plan**. `suggested_shapes_do_not_name_the_paid_document_body` pins the constants. Searching document text stays ungated on every plan and returns no fragment of it. + - **State, local and education (SLED) procurement** (Tango API 4.25.0; parity with tango-python, tango-node and tango-go). `resources/sled.rs` adds `list_sled_opportunities` / `get_sled_opportunity`, `list_sled_opportunity_revisions`, `get_sled_coverage`, `list_sled_forecasts` / `get_sled_forecast`, plus `iterate_sled_opportunities` and `iterate_sled_forecasts`. Four `bon`-derived options builders (`ListSledOpportunitiesOptions`, `ListSledOpportunityRevisionsOptions`, `ListSledForecastsOptions`, `GetSledOptions`) name every one of the API's 27 solicitation filters and 13 forecast filters, and five `SHAPE_SLED_*` constants land in `shapes.rs`. Four behaviors are documented on the options builders because each misleads a caller who assumes federal semantics. **Leaving both `status` and `active` unset returns open solicitations only** — that default is the API's, and `list_sled_opportunities` deliberately does not synthesize `status=open`, since doing so would make `active = Some(false)` unreachable (pinned by a test). **`status` is Tango-derived and refreshed every fifteen minutes**; the portal's own word is served as `source_status`, is frozen at last capture, and is not a liveness filter. **Category scheme tagging is mid-migration**, so `naics` matches only the small tagged share and `category_code` is the escape hatch. And **`meta.attachment_count` can be lower than the length of `attachments`**, because an auto-generated portal cover sheet is listed and flagged `is_generated_summary` but excluded from the count. diff --git a/crates/tango/src/resources/sled.rs b/crates/tango/src/resources/sled.rs index 694a44e..1f80827 100644 --- a/crates/tango/src/resources/sled.rs +++ b/crates/tango/src/resources/sled.rs @@ -520,6 +520,18 @@ impl Client { /// `GET /api/sled/opportunities/{opportunity_id}/` — one solicitation, /// whatever its status. The open-only default applies to the list endpoint, /// not here. + /// + /// A document's body is available as the `attachments(extracted_text)` leaf + /// on a Small plan or above, from API version 4.25.1. It must be NAMED — no + /// suggested shape includes it and `attachments(*)` does not carry it — and + /// its key is ABSENT rather than null whenever the text is not being served: + /// below Small (where it is withheld and named in `meta.upgrade_hints`), on a + /// contested document, or where it could not be resolved. A contested + /// document never returns text at any plan, because its stored bytes + /// disagree with what the record advertised. + /// + /// Searching document text and reading it are separate: `search` matches + /// inside attachment text on every plan and returns no fragment of it. pub async fn get_sled_opportunity( &self, opportunity_id: &str, @@ -815,6 +827,27 @@ mod tests { ); } + /// The API resolves the document body only for a caller who names it, so a + /// suggested shape naming it would make every detail fetch pay for it. + #[test] + fn suggested_shapes_do_not_name_the_paid_document_body() { + for (name, shape) in [ + ( + "SHAPE_SLED_OPPORTUNITIES_MINIMAL", + crate::SHAPE_SLED_OPPORTUNITIES_MINIMAL, + ), + ( + "SHAPE_SLED_OPPORTUNITIES_COMPREHENSIVE", + crate::SHAPE_SLED_OPPORTUNITIES_COMPREHENSIVE, + ), + ] { + assert!( + !shape.contains("extracted_text"), + "{name} must not name the Small-gated `extracted_text` leaf: {shape}" + ); + } + } + #[test] fn list_sled_forecasts_filters_emit() { let opts = ListSledForecastsOptions::builder() diff --git a/crates/tango/src/shapes.rs b/crates/tango/src/shapes.rs index e4d924f..dfeb91f 100644 --- a/crates/tango/src/shapes.rs +++ b/crates/tango/src/shapes.rs @@ -149,6 +149,15 @@ pub const SHAPE_SLED_OPPORTUNITIES_MINIMAL: &str = concat!( /// Suggested detail shape for /// [`Client::get_sled_opportunity`](crate::Client::get_sled_opportunity). +/// +/// Deliberately does not name `attachments(extracted_text)`. The document body +/// needs a Small plan and the API resolves it only for a caller who names the +/// leaf, so a default shape carrying it would make every detail fetch pay for a +/// document nobody asked to read. Ask for it explicitly instead: +/// +/// ```text +/// opportunity_id,attachments(name,size_bytes,extracted_text) +/// ``` pub const SHAPE_SLED_OPPORTUNITIES_COMPREHENSIVE: &str = concat!( "opportunity_id,solicitation_number,solicitation_type,", "solicitation_type_source,title,description,state,jurisdiction,agency,", diff --git a/docs/API_REFERENCE.md b/docs/API_REFERENCE.md index 543038a..df6e256 100644 --- a/docs/API_REFERENCE.md +++ b/docs/API_REFERENCE.md @@ -151,7 +151,9 @@ Options: `ListSledOpportunitiesOptions`, `ListSledOpportunityRevisionsOptions`, **Category scheme tagging is mid-migration**, so `naics` matches only the small tagged share. Use `category_code` to match a code under any scheme, including the untagged pre-migration strings. -**`meta.attachment_count` can be lower than the length of `attachments`.** Some portals auto-generate a cover sheet alongside the real documents; it is listed and flagged `is_generated_summary` but excluded from the count and from `has_documents`. The count answers "does this record hold its solicitation package"; the array answers "what files exist". Attachment bodies are never served as a field. +**`meta.attachment_count` can be lower than the length of `attachments`.** Some portals auto-generate a cover sheet alongside the real documents; it is listed and flagged `is_generated_summary` but excluded from the count and from `has_documents`. The count answers "does this record hold its solicitation package"; the array answers "what files exist". + +**The document body is `attachments(extracted_text)`, on a Small plan or above** (API 4.25.1+). It must be **named** — no `SHAPE_SLED_*` constant includes it and `attachments(*)` does not carry it, because the API resolves the body only for a caller who asked. Its key is **absent rather than null** whenever the text is not being served: below Small (withheld and named in `meta.upgrade_hints`), on a contested document, or where it could not be resolved. A **contested document never returns text at any plan**, because its stored bytes disagree with what the record advertised. Searching document text and reading it are separate — `search` matches inside attachment text on every plan and returns no fragment of it. `get_sled_coverage` takes no parameters and is neither shaped nor paginated. **Call it before treating a per-state count as market size** — a thin result for a state is at least as likely to be a portal Tango does not read as a quiet market, and every state row carries all five status buckets whether or not they have rows.