From 79f45c9a1e4de7bacb7129cf52727315f6d4265c Mon Sep 17 00:00:00 2001 From: MrAlders0n Date: Mon, 5 Oct 2026 09:01:17 -0400 Subject: [PATCH] Document shared API keys, access scopes and staged repeater enforcement --- docs/api-keys.md | 62 ++++++++++++++++++++++++++++++++++ docs/channels-api.md | 62 ++++++++++++---------------------- docs/coverage-api.md | 16 ++++++--- docs/scopes-api.md | 63 ++++++++++++----------------------- docs/zones-api.md | 79 +++++++++++++------------------------------- mkdocs.yml | 1 + 6 files changed, 141 insertions(+), 142 deletions(-) create mode 100644 docs/api-keys.md diff --git a/docs/api-keys.md b/docs/api-keys.md new file mode 100644 index 0000000..b71b07c --- /dev/null +++ b/docs/api-keys.md @@ -0,0 +1,62 @@ +# API keys and access + +MeshMapper uses the same **Coverage API key** for Coverage and the read APIs below. Existing keys keep their values, geographic assignments and Coverage quota. There is no need to regenerate a key for the new endpoints. + +!!! info "Server v1.5.117 rollout" + Zones, boundaries, scopes and channels require keys when the updated server endpoints are deployed. Send your key now as part of the migration. Repeater authentication has a separate switch, off by default until app 1.4.1 is shipped and the update is forced. This page describes the new contract, not a confirmation that every region has deployed it. + +## Endpoints and key types + +| API | Credential | Geographic access | +| --- | --- | --- | +| [Coverage](coverage-api.md), `coverage.php` | Coverage key in `X-API-Key` or legacy `?key=` | Assigned region, group, selected regions or GLOBAL | +| [Region directory](zones-api.md), `get_zones.php` | Coverage key in `X-API-Key` | Authorized regions within the requested country | +| [Boundaries](zones-api.md#region-boundary), `get_geojson.php` | Coverage key in `X-API-Key` | Requested region or fully authorized group | +| [Scopes](scopes-api.md), `get_scopes.php` | Coverage key in `X-API-Key` | Requested region or fully authorized group | +| [Channels](channels-api.md), `get_channels.php` | Coverage key in `X-API-Key` | Requested region or fully authorized group | +| Repeaters, `get_repeaters.php` | Scoped Coverage key or existing mobile App key in `X-API-Key`; optional during transition | Coverage keys stay within their assignments; App keys support the app's cross-region repeater reads | + +Only Coverage accepts a query-string key. App keys do not grant access to Coverage, zones, boundaries, scopes or channels. The mobile app can send its existing App key for repeaters before connecting a radio or starting a session. + +## Generate or reuse a key + +In your region or group's admin panel, open **User Settings > API Access**. Generate a key with a description, or show your existing key. Each administrator can have one self-service key per region or group. Regeneration invalidates the old value immediately. + +Group keys cover the group's current enabled member IATAs, including membership changes without issuing a new key. A key for just one member cannot fetch a whole group's boundaries, scopes or channels unless it covers every enabled member. Repeater exports are filtered to the Coverage key's authorized members, even where a member URL normally expands to its group. + +Contact the MeshMapper team for custom integrations, selected-region or GLOBAL access, endpoint permissions, or larger limits. Master admins can allow only the endpoints an integration needs. GLOBAL permits geographic access across regions; it does not remove required parameters or turn every endpoint into an all-region feed. For example, `get_zones.php` still requires `country`. + +## Send the key + +Keep integration keys in your backend configuration, outside source control and visitor-facing JavaScript. For example, with `MESHMAPPER_API_KEY` set in your environment: + +```bash +curl --fail-with-body --compressed \ + -H "X-API-Key: $MESHMAPPER_API_KEY" \ + 'https://yow.meshmapper.net/get_scopes.php' +``` + +Send the header on every request, including conditional requests. CORS preflights allow `X-API-Key` and `If-None-Match`. + +## Limits and caching + +The five read APIs share a default budget of **1,000 admitted requests per UTC day and 30 per fixed minute per Coverage key**. Custom keys may have different limits. Coverage retains its separate daily quota, normally 100 calls for self-service keys. Read calls do not consume Coverage calls. + +Every admitted read counts, including a `304` and a downstream data error. Authentication and authorization failures do not consume the read budget. `OPTIONS` preflights do not count. Existing IP burst protection also applies; App repeater reads do not put all phones behind one shared daily key budget. + +Authenticated reads replace the previous once-per-target IP interval. Key quota exhaustion returns `503 key_rate_limited` with `Retry-After`; wait at least that long before retrying. The daily allowance resets at midnight UTC. A zones/boundaries/scopes/channels IP burst rejection returns `503 slow_down`; repeaters may return `429 rate_limited`. Coverage retains its own documented `429` responses. + +Protected responses use `Cache-Control: private, no-store`. Keep your backend's parsed data and any ETag separately for each credential and geographic scope, and discard data when access changes. Where supported, `If-None-Match` saves bandwidth but not quota. Do not expose keys in shared cache URLs. Coverage's server-side grid cache is unchanged. + +## Read API errors + +| Status | Error | Action | +| --- | --- | --- | +| 401 | `missing_key`, `invalid_key` | Supply a current key in `X-API-Key`. | +| 403 | `wrong_key_type` | Use a Coverage key; App keys only support repeaters. | +| 403 | `api_not_allowed` | Ask the key owner to review endpoint permissions. | +| 403 | `no_region`, `region_not_allowed` | Check the key's region/group assignment. | +| 503 | `key_rate_limited` | Wait for `Retry-After`. | +| 503 | `auth_unavailable` | Authentication storage is temporarily unavailable; respect `Retry-After`. | + +Endpoint-specific errors remain documented on each API page. Coverage keeps its existing missing/invalid-key errors and adds `403 api_not_allowed` and `503 auth_unavailable`. diff --git a/docs/channels-api.md b/docs/channels-api.md index 9af4bd6..b7acfe0 100644 --- a/docs/channels-api.md +++ b/docs/channels-api.md @@ -1,6 +1,6 @@ # Channels API -The Channels API lists the hashtag channels a region's admins have set up for it. That's the same list the MeshMapper app listens on for passive RX in that region. Use it to show a region's local channels, or to set them up in your own client, without scraping the map. No key is needed. +The Channels API lists the hashtag channels a region's admins have set up for it. That's the same list the MeshMapper app listens on for passive RX in that region. Use it to show a region's local channels, or to set them up in your own client, without scraping the map. Send your Coverage API key in the `X-API-Key` header. ``` GET https://yow.meshmapper.net/get_channels.php @@ -11,11 +11,11 @@ GET https://yow.meshmapper.net/get_channels.php There are no parameters. -!!! warning "One call per day; keep calling early and you're banned" - Each region site's answer can be fetched **once every 23.5 hours per IP address**. A second request to the same region site inside that window is refused with `429`, and **the third refused request within the window bans your IP address from all of meshmapper.net for a day**. Every repeat doubles the ban, up to 30 days. See [Call limits](#call-limits). Accessing anything that isn't a published API, or scraping pages for data, is not allowed; see the warning on the [Coverage API](coverage-api.md) page. +## Authentication -!!! warning "Call this from your server, not your visitors' browsers" - Fetch this API from your own backend, store the result, and serve your own copy to visitors. Don't call it from client-side JavaScript in a visitor's browser: everyone behind the same home router or mobile carrier shares one public IP address, so one visitor's fetch uses the call and the next visitor's fetch bans that whole IP address from all of meshmapper.net, including every MeshMapper app user on that network. +Send your existing **Coverage API key** in the `X-API-Key` header. No new key is needed. See [API keys and access](api-keys.md) for generation, group permissions, custom GLOBAL integrations and the v1.5.117 rollout. + +Fetch from your backend and serve your own parsed data to visitors. Keep the key out of client-side JavaScript. ## Response @@ -69,51 +69,33 @@ print(k.hex(), hashlib.sha256(k).hexdigest()[:2]) # 7871ec72b45617696c35c970bd ## Call limits -- **One call per region site every 23.5 hours per IP address.** A group site counts as its own site, separate from its members. -- **An early call is refused with `429`; the third refusal within the window bans your IP address.** Your IP address is banned from all of meshmapper.net (the map included, and every device behind that IP address) for 1 day. Each repeat doubles the ban: 2 days, 4 days, 8 days, up to 30 days. -- **At most 30 calls a minute from one IP address, across all these APIs together.** A call over that gets `503` with `{"error":"slow_down","retry_after":N}`. It doesn't use your call and never counts toward a ban: wait `Retry-After` seconds and carry on. Pausing 2 seconds between calls keeps you under it. -- **Only certain JSON errors hand your call back.** A `404` `zone_not_found`, a `503` `unavailable`, or a `503` `slow_down` (after waiting `Retry-After`) doesn't use your call, so retrying after one of those is safe. Anything else, a cut-off or unparseable body, a timeout on your side, an HTML `5xx` error page, a connection reset, may already have used your call: keep your previous copy and wait for your next scheduled run instead of retrying. -- **Only `GET` counts.** A browser's CORS preflight (`OPTIONS`) doesn't. `HEAD` and other methods get `405` and don't count either. -- **Don't rely on the clock, track your last call.** Store the time of your last served call for each region site and skip the call if it was less than 23.5 hours ago. Schedule in UTC: a local-time cron job gets a 23-hour day at the DST change, which is enough to trip the limit. -- **Use a generous client timeout, 60 to 120 seconds.** A short timeout on your side can cut the connection before the server finishes, and that counts against you as an "anything else" error above, not a safe-to-retry one. -- **While testing, don't open the URL twice.** Save the response to a file once and work from the file; every reload after the first is refused and counts toward a ban. +These reads use the key's shared read budget, normally **1,000 requests per UTC day and 30 per fixed minute**, independently of Coverage's daily quota. Authenticated reads no longer use the old once-per-target IP interval. IP burst protection still applies. Every admitted request counts, including `304` responses and downstream failures. See [limits and caching](api-keys.md#limits-and-caching). ## Caching -Sends `Cache-Control: public, max-age=84600` (23.5 hours) and a strong `ETag`, so a standard HTTP cache won't ask again before your next call is allowed. On your next daily call, send the `ETag` back in `If-None-Match` to get `304 Not Modified` with no body when nothing changed. `generated_at` isn't part of the `ETag`. A `304` uses your call for that window, just like a `200`. - -`ETag` and `Retry-After` are readable from browser JavaScript (`Access-Control-Expose-Headers`), for server-side tools and your own debugging, not so you can embed this call directly in a page your visitors load; see the warning above. - -Responses are gzip-compressed. +Responses use `Cache-Control: private, no-store`. Keep your backend's parsed data and ETag separately by key and scope. Send both `X-API-Key` and `If-None-Match` on conditional requests; a `304` saves bandwidth but still counts. Responses support gzip compression. ## Errors -Errors are JSON: `{"error": ""}`. A `429` also carries `retry_after`. +Authentication, permissions and quota errors are listed in [Read API errors](api-keys.md#read-api-errors). -| Status | `error` | Meaning | +| Status | Error | Meaning | | --- | --- | --- | -| 404 | `zone_not_found` | The region is unknown, pending or turned off. | -| 405 | `method_not_allowed` | Only `GET` (and the `OPTIONS` preflight) are answered. | -| 429 | `rate_limited` | This region site was already fetched from your IP address in the last 23.5 hours. `Retry-After` and `retry_after` give the seconds left. Three of these within the window ban your IP address; see [Call limits](#call-limits). | -| 503 | `unavailable` | Temporary server problem. Try again later; it doesn't use your call. | -| 503 | `slow_down` | More than 30 calls in the last minute from your IP address, across all these APIs. `Retry-After` and `retry_after` give the seconds left. It doesn't use your call and never counts toward a ban. | +| 404 | `zone_not_found` | The region is unknown, pending or disabled. | +| 405 | `method_not_allowed` | Use GET or an OPTIONS preflight. | +| 503 | `unavailable` | Temporary data problem; retain your previous copy and retry later. An admitted request still consumes quota. | +| 503 | `slow_down` | IP burst protection; wait for Retry-After. | + +A group request requires access to every enabled member. ## Example -Run this once a day from a scheduled job on your own server, never in a loop and never from a browser. It tracks the last served call so an early rerun skips instead of risking a ban: +With `MESHMAPPER_API_KEY` set in your backend environment: -```python -import json, os, time, httpx - -STATE = "meshmapper-state.json" -last = json.load(open(STATE)) if os.path.exists(STATE) else {} - -url = "https://yow.meshmapper.net/get_channels.php" -if time.time() - last.get(url, 0) >= 23.5 * 3600: - r = httpx.get(url, timeout=120) - if r.status_code == 200: - last[url] = time.time() - json.dump(r.json(), open("yow-channels.json", "w")) - # anything else: keep your previous copy, don't retry here -json.dump(last, open(STATE, "w")) +```bash +curl --fail-with-body --compressed \ + -H "X-API-Key: $MESHMAPPER_API_KEY" \ + 'https://yow.meshmapper.net/get_channels.php' ``` + +Schedule polling to fit your key budget, retain the last successful data on failure, and respect `Retry-After`. diff --git a/docs/coverage-api.md b/docs/coverage-api.md index f84d512..7fd1188 100644 --- a/docs/coverage-api.md +++ b/docs/coverage-api.md @@ -6,6 +6,10 @@ The Coverage API provides programmatic access to MeshMapper coverage grid-square Access requires a **Coverage** API key. Each key is scoped to a specific region, a multiregion group, or a set of adjacent regions (see [Multi-Region Keys](#multi-region-keys)). Self-service keys have a daily limit of 100 requests; keys issued by the MeshMapper team may carry a different limit (the 429 body's `limit` field shows yours). +This same key also works for zones, boundaries, scopes, channels and repeaters, subject to endpoint permissions. Existing keys do not need regeneration. Group keys follow current member IATAs. See [API keys and access](api-keys.md) for the endpoint table, shared read limits and repeater rollout. + +Send `X-API-Key: YOUR_API_KEY` on requests, or keep using the existing `?key=` URLs for Coverage. + ### Generating a Key Regional administrators can generate their own API key directly from the admin panel: @@ -16,7 +20,7 @@ Regional administrators can generate their own API key directly from the admin p 4. Enter a description/reason for the key (mandatory) 5. Click **Generate API Key** -Each administrator is limited to **one API key per region**. The key is automatically scoped to your region with a fixed rate limit of 100 requests per day. If you need to replace your key, use the **Regenerate** button — this invalidates the old key immediately. +Each administrator is limited to **one API key per region or group**. The key is automatically scoped to that region or group with a fixed rate limit of 100 requests per day. If you need to replace your key, use the **Regenerate** button — this invalidates the old key immediately. !!! warning "Unauthorized Access" MeshMapper utilizes API keys and rate limits to protect server resources and prevent access to data that regions do not wish to have shared externally. As such, accessing unauthorized API's, scraping for data, etc., is strictly prohibited and will result in action taken to protect the server and data (which may include IP or origin bans, removal of a region, etc.). The MeshMapper team is happy to review requests for data not provided in the API's below. @@ -31,7 +35,7 @@ GET https://meshmapper.net/coverage.php?key=YOUR_API_KEY | Parameter | Required | Description | | --- | --- | --- | -| `key` | Yes | Your Coverage API key. | +| `key` | Unless using X-API-Key | Your Coverage API key. Existing query-key URLs remain supported. | | `include` | No | Comma-separated list of optional sections to add to the response. Currently supports `repeaters` (e.g. `?include=repeaters`) — see [Repeater Fields](#repeater-fields). | | `fresh` | No | `fresh=1` skips the 15-minute server cache and builds the grid now. Single-region and group keys only; multi-region and global keys return HTTP 400 `fresh_not_supported`. | | `f_*` | No | Filter the pings that go into the grid before it is built: by radio configuration, date, power or antenna. See [Filtering](#filtering). | @@ -254,7 +258,7 @@ Filters are not available on [multi-region](#multi-region-keys) or [global](#glo The API is built for efficient, low-frequency polling. Coverage data does not change second-to-second, so please poll sparingly. - **Compression.** Responses are gzip-compressed. Send `Accept-Encoding: gzip` (most HTTP clients do this automatically) to receive compressed data — payloads are much smaller. -- **Server-side cache.** Responses carry `Cache-Control: public, max-age=900` and are cached for up to **15 minutes**. Polling more often than that returns identical data (and still counts toward your daily limit), so a poll interval of **15 minutes or longer is recommended**. `generated_at` tells you when the cached data was built. Filtered responses (see [Filtering](#filtering)) are built on every request and are not cached on the server. On single-region and group keys, `fresh=1` skips the cache and rebuilds now. +- **Server-side cache.** Grid data is cached on the server for up to **15 minutes**. HTTP responses use `Cache-Control: private, no-store` to prevent shared caching of protected data. Polling more often than that returns identical data (and still counts toward your daily limit), so a poll interval of **15 minutes or longer is recommended**. `generated_at` tells you when the cached data was built. Filtered responses (see [Filtering](#filtering)) are built on every request and are not cached on the server. On single-region and group keys, `fresh=1` skips the cache and rebuilds now. - **Conditional requests.** Each response includes an `ETag` (and `Last-Modified`). Send the `ETag` value back in an `If-None-Match` header; if nothing has changed since, you'll get a **`304 Not Modified`** with an empty body, saving you the download. ```bash @@ -299,6 +303,8 @@ A separate per-IP throttle protects against bursts: about 30 requests in a rolli | 400 | `fresh_not_supported` | `fresh=1` on a multi-region or global key. | | 400 | `too_many_regions` | A multi-region key with more than 6 member regions. | | 401 | `invalid_key` | API key not found, or not a Coverage key (message: `Invalid or non-Coverage API key`). | +| 403 | `api_not_allowed` | Coverage access is disabled for this key. | +| 503 | `auth_unavailable` | Authentication policy storage is temporarily unavailable; respect Retry-After. | | 403 | `no_region` | No region assigned to this key, or the key's region has been deleted. | | 429 | `rate_limit_exceeded` | Daily request limit reached. | | 429 | `rate_limited` | Too many requests in a short period (per-IP throttle). | @@ -315,7 +321,7 @@ If you have a Coverage API key assigned to your admin account, you can view your A Coverage key can be scoped to a **set of up to 6 regions** (for example `PDX,SEA,YVR`) instead of a single region. The response merges every member region's coverage into **one grid** — the same payload shape as a single-region response — so it suits integrations that render adjacent regions as one continuous map. -Multi-region keys are not self-service: like [global keys](#global-coverage-feed) and keys with a custom limit, they are issued by the MeshMapper team (Master administrators) on request (the admin-panel self-service flow only creates single-region keys). Adjacent regions are the intended use — the merged grid serves them as one map. +Multi-region keys are not self-service: like [global keys](#global-coverage-feed) and keys with a custom limit, they are issued by the MeshMapper team (Master administrators) on request (the admin-panel self-service flow creates keys for its current region or group). Adjacent regions are the intended use — the merged grid serves them as one map. !!! info "Multi-region keys vs. Multiregion Groups" A [Multiregion Group](multiregions.md) merges regions *inside* MeshMapper — shared map, leaderboards, collision detection, and admin panel. A multi-region **key** changes nothing about the regions themselves; it only merges their coverage data in this API's response. If a group already exists, a key can simply be scoped to the group's code instead. A multi-region key is for sets of regions that aren't (and shouldn't become) a group. @@ -432,7 +438,7 @@ There is **no pagination or cursor**, by design: the response is served as a sin The global response aggregates the entire fleet, so it is cached more aggressively than regional responses: -- The server cache lasts **6 hours** (`Cache-Control: public, max-age=21600`). Polling more often returns identical data; **once or twice a day is the intended usage**. +- The server cache lasts **6 hours**. HTTP responses use `Cache-Control: private, no-store`. Polling more often returns identical data; **once or twice a day is the intended usage**. - A request that arrives after the cache has expired triggers a rebuild. The response **streams region-by-region while it builds** — data starts flowing immediately, but the complete download can take a minute. Configure a generous *total* timeout in your HTTP client (the connection is never idle, so per-read timeouts are fine at their defaults). All other requests are served instantly from cache. - `ETag` / `If-None-Match` conditional requests work exactly as for regional keys, and a `304 Not Modified` is by far the cheapest way to poll. The response that triggers a rebuild carries no `ETag`; the next (cached) response does. diff --git a/docs/scopes-api.md b/docs/scopes-api.md index 2f1a7d4..3d4b91b 100644 --- a/docs/scopes-api.md +++ b/docs/scopes-api.md @@ -1,6 +1,6 @@ # Scopes API -The Scopes API lists the Mesh Scopes a region's repeaters carry, with a repeater count per scope. Use it to show which scopes are active in a region without scraping the map. No key is needed. +The Scopes API lists the Mesh Scopes a region's repeaters carry, with a repeater count per scope. Use it to show which scopes are active in a region without scraping the map. Send your Coverage API key in the `X-API-Key` header. ``` GET https://yow.meshmapper.net/get_scopes.php @@ -11,11 +11,11 @@ GET https://yow.meshmapper.net/get_scopes.php There are no parameters. -!!! warning "One call per hour; keep calling early and you're banned" - Each region site's answer can be fetched **once every 55 minutes per IP address**. A second request to the same region site inside that window is refused with `429`, and **the third refused request within the window bans your IP address from all of meshmapper.net for an hour**. Every repeat doubles the ban, up to a week. See [Call limits](#call-limits). Accessing anything that isn't a published API, or scraping pages for data, is not allowed; see the warning on the [Coverage API](coverage-api.md) page. +## Authentication -!!! warning "Call this from your server, not your visitors' browsers" - Fetch this API from your own backend, store the result, and serve your own copy to visitors. Don't call it from client-side JavaScript in a visitor's browser: everyone behind the same home router or mobile carrier shares one public IP address, so one visitor's fetch uses the call and the next visitor's fetch bans that whole IP address from all of meshmapper.net, including every MeshMapper app user on that network. +Send your existing **Coverage API key** in the `X-API-Key` header. No new key is needed. See [API keys and access](api-keys.md) for generation, group permissions, custom GLOBAL integrations and the v1.5.117 rollout. + +Fetch from your backend and serve your own parsed data to visitors. Keep the key out of client-side JavaScript. ## Response @@ -59,52 +59,33 @@ Names and counts only. There is no per-repeater list, no repeater ID, and no key ## Call limits -- **One call per region site every 55 minutes per IP address.** A group site counts as its own site, separate from its members. -- **An early call is refused with `429`; the third refusal within the window bans your IP address.** Your IP address is banned from all of meshmapper.net (the map included, and every device behind that IP address) for 1 hour. Each repeat doubles the ban: 2 hours, 4 hours, 8 hours, up to a week. -- **At most 30 calls a minute from one IP address, across all these APIs together.** A call over that gets `503` with `{"error":"slow_down","retry_after":N}`. It doesn't use your call and never counts toward a ban: wait `Retry-After` seconds and carry on. Pausing 2 seconds between calls keeps you under it. -- **Only certain JSON errors hand your call back.** A `404` `zone_not_found`, a `503` `unavailable`, or a `503` `slow_down` (after waiting `Retry-After`) doesn't use your call, so retrying after one of those is safe. Anything else, a cut-off or unparseable body, a timeout on your side, an HTML `5xx` error page, a connection reset, may already have used your call: keep your previous copy and wait for your next scheduled run instead of retrying. -- **Only `GET` counts.** A browser's CORS preflight (`OPTIONS`) doesn't. `HEAD` and other methods get `405` and don't count either. -- **Don't rely on the clock, track your last call.** Store the time of your last served call for each region site and skip the call if it was less than 55 minutes ago. Schedule in UTC: a local-time cron's hour mapping shifts at the DST change, which can land two runs closer together than you expect. -- **Use a generous client timeout, 60 to 120 seconds.** A short timeout on your side can cut the connection before the server finishes, and that counts against you as an "anything else" error above, not a safe-to-retry one. -- **While testing, don't open the URL twice.** Save the response to a file once and work from the file; every reload after the first is refused and counts toward a ban. +These reads use the key's shared read budget, normally **1,000 requests per UTC day and 30 per fixed minute**, independently of Coverage's daily quota. Authenticated reads no longer use the old once-per-target IP interval. IP burst protection still applies. Every admitted request counts, including `304` responses and downstream failures. See [limits and caching](api-keys.md#limits-and-caching). ## Caching -Sends `Cache-Control: public, max-age=3300` (55 minutes) and a strong `ETag`, so a standard HTTP cache won't ask again before your next call is allowed. On your next call, send the `ETag` back in `If-None-Match` to get `304 Not Modified` with no body when nothing changed. `generated_at` isn't part of the `ETag`. A `304` uses your call for that window, just like a `200`. - -`ETag` and `Retry-After` are readable from browser JavaScript (`Access-Control-Expose-Headers`), for server-side tools and your own debugging, not so you can embed this call directly in a page your visitors load; see the warning above. - -Responses are gzip-compressed. +Responses use `Cache-Control: private, no-store`. Keep your backend's parsed data and ETag separately by key and scope. Send both `X-API-Key` and `If-None-Match` on conditional requests; a `304` saves bandwidth but still counts. Responses support gzip compression. ## Errors -Errors are JSON: `{"error": ""}`. A `429` also carries `retry_after`. +Authentication, permissions and quota errors are listed in [Read API errors](api-keys.md#read-api-errors). -| Status | `error` | Meaning | +| Status | Error | Meaning | | --- | --- | --- | -| 404 | `zone_not_found` | The region is unknown, pending or turned off. | -| 405 | `method_not_allowed` | Only `GET` (and the `OPTIONS` preflight) are answered. | -| 429 | `rate_limited` | This region site was already fetched from your IP address in the last 55 minutes. `Retry-After` and `retry_after` give the seconds left. Three of these within the window ban your IP address; see [Call limits](#call-limits). | -| 503 | `unavailable` | Temporary server problem. Try again later; it doesn't use your call. | -| 503 | `slow_down` | More than 30 calls in the last minute from your IP address, across all these APIs. `Retry-After` and `retry_after` give the seconds left. It doesn't use your call and never counts toward a ban. | +| 404 | `zone_not_found` | The region is unknown, pending or disabled. | +| 405 | `method_not_allowed` | Use GET or an OPTIONS preflight. | +| 503 | `unavailable` | Temporary data problem; retain your previous copy and retry later. An admitted request still consumes quota. | +| 503 | `slow_down` | IP burst protection; wait for Retry-After. | -## Example +A group request requires access to every enabled member. -Run this at most once an hour from a scheduled job on your own server, never in a loop and never from a browser. It tracks the last served call so an early rerun skips instead of risking a ban: - -```python -import json, os, time, httpx +## Example -STATE = "meshmapper-state.json" -last = json.load(open(STATE)) if os.path.exists(STATE) else {} +With `MESHMAPPER_API_KEY` set in your backend environment: -url = "https://yow.meshmapper.net/get_scopes.php" -if time.time() - last.get(url, 0) >= 55 * 60: - r = httpx.get(url, timeout=120) - if r.status_code == 200: - last[url] = time.time() - for s in r.json()["scopes"]: - print(s["name"], s["repeaters"]) - # anything else: keep your previous copy, don't retry here -json.dump(last, open(STATE, "w")) +```bash +curl --fail-with-body --compressed \ + -H "X-API-Key: $MESHMAPPER_API_KEY" \ + 'https://yow.meshmapper.net/get_scopes.php' ``` + +Schedule polling to fit your key budget, retain the last successful data on failure, and respect `Retry-After`. diff --git a/docs/zones-api.md b/docs/zones-api.md index 46be1dd..ce4125c 100644 --- a/docs/zones-api.md +++ b/docs/zones-api.md @@ -1,6 +1,6 @@ # Zones API -The Zones API lists MeshMapper regions and serves their boundaries as GeoJSON. Use it to show MeshMapper regions on your own map, link to them, or keep a local copy of their outlines. No key is needed. +The Zones API lists MeshMapper regions and serves their boundaries as GeoJSON. Use it to show MeshMapper regions on your own map, link to them, or keep a local copy of their outlines. Send your Coverage API key in the `X-API-Key` header. Two endpoints work together: @@ -9,11 +9,11 @@ Two endpoints work together: Call the first one, then fetch `url + "get_geojson.php"` for each region you want. -!!! warning "One call per day; keep calling early and you're banned" - Each answer can be fetched **once every 23.5 hours per IP address**: `get_zones.php` once per country, and `get_geojson.php` once per region site. A second request for the same country or region inside that window is refused with `429`, and **the third refused request within the window bans your IP address from all of meshmapper.net for a day**. Every repeat doubles the ban, up to 30 days. See [Call limits](#call-limits). Accessing anything that isn't a published API, or scraping pages for data, is not allowed; see the warning on the [Coverage API](coverage-api.md) page. +## Authentication -!!! warning "Call this from your server, not your visitors' browsers" - Fetch this API from your own backend, store the result, and serve your own copy to visitors. Don't call it from client-side JavaScript in a visitor's browser: everyone behind the same home router or mobile carrier shares one public IP address, so one visitor's fetch uses the call and the next visitor's fetch bans that whole IP address from all of meshmapper.net, including every MeshMapper app user on that network. +Send your existing **Coverage API key** in the `X-API-Key` header. No new key is needed. See [API keys and access](api-keys.md) for generation, group permissions, custom GLOBAL integrations and the v1.5.117 rollout. + +Fetch from your backend and serve your own parsed data to visitors. Keep the key out of client-side JavaScript. ## List regions @@ -119,66 +119,33 @@ Returns a GeoJSON `FeatureCollection` ([RFC 7946](https://datatracker.ietf.org/d ## Call limits -- **One call per target every 23.5 hours per IP address.** For `get_zones.php` the target is the country (`?country=CA` and `?country=US` are separate calls). For `get_geojson.php` it's the region site, and a group site counts as its own site, separate from its members. Fetching every region of a country in one run is fine: each region is its own call. -- **An early call for the same target is refused with `429`; the third refusal within the window bans your IP address.** Your IP address is banned from all of meshmapper.net (the map included, and every device behind that IP address) for 1 day. Each repeat doubles the ban: 2 days, 4 days, 8 days, up to 30 days. -- **At most 30 calls a minute from one IP address, across all these APIs together.** A call over that gets `503` with `{"error":"slow_down","retry_after":N}`. It doesn't use your call and never counts toward a ban: wait `Retry-After` seconds and carry on. Pausing 2 seconds between calls keeps you under it. -- **Only certain JSON errors hand your call back.** A `400` (missing or bad `country` on `get_zones.php`), a `404` `zone_not_found`, a `503` `unavailable`, or a `503` `slow_down` (after waiting `Retry-After`) doesn't use your call, so retrying after one of those is safe. Anything else, a cut-off or unparseable body, a timeout on your side, an HTML `5xx` error page, a connection reset, may already have used your call: keep your previous copy and wait for your next scheduled run instead of retrying. -- **Only `GET` counts.** A browser's CORS preflight (`OPTIONS`) doesn't. `HEAD` and other methods get `405` and don't count either. -- **Don't rely on the clock, track your last call.** Store the time of your last served call for each target (country or region) and skip the call if it was less than 23.5 hours ago. Schedule in UTC: a local-time cron job gets a 23-hour day at the DST change, which is enough to trip the limit. -- **Use a generous client timeout, 60 to 120 seconds.** A short timeout on your side can cut the connection before the server finishes, and that counts against you as an "anything else" error above, not a safe-to-retry one. -- **While testing, don't open a URL twice.** Save the response to a file once and work from the file; every reload after the first is refused and counts toward a ban. +These reads use the key's shared read budget, normally **1,000 requests per UTC day and 30 per fixed minute**, independently of Coverage's daily quota. Authenticated reads no longer use the old once-per-target IP interval. IP burst protection still applies. Every admitted request counts, including `304` responses and downstream failures. See [limits and caching](api-keys.md#limits-and-caching). ## Caching -Both endpoints send `Cache-Control: public, max-age=84600` (23.5 hours) and an `ETag`, so a standard HTTP cache won't ask again before your next call is allowed. On your next daily call, send the `ETag` back in `If-None-Match`; you'll get `304 Not Modified` with no body if nothing changed. `generated_at` changes on every response and isn't part of the `ETag`. A `304` uses your call for that window, just like a `200`. - -`ETag` and `Retry-After` are readable from browser JavaScript (`Access-Control-Expose-Headers`), for server-side tools and your own debugging, not so you can embed this call directly in a page your visitors load; see the warning above. - -Responses are gzip-compressed. +Responses use `Cache-Control: private, no-store`. Keep your backend's parsed data and ETag separately by key and scope. Send both `X-API-Key` and `If-None-Match` on conditional requests; a `304` saves bandwidth but still counts. Responses support gzip compression. ## Errors -Errors are JSON: `{"error": ""}`. A `429` also carries `retry_after`. +Authentication, permissions and quota errors are listed in [Read API errors](api-keys.md#read-api-errors). -| Status | `error` | Meaning | +| Status | Error | Meaning | | --- | --- | --- | -| 400 | `country_required` | `get_zones.php` was called without `country`. | -| 400 | `invalid_country` | `country` isn't two letters. | -| 404 | `zone_not_found` | The region is unknown, pending or turned off. | -| 405 | `method_not_allowed` | Only `GET` (and the `OPTIONS` preflight) are answered. | -| 429 | `rate_limited` | This country or region was already fetched from your IP address in the last 23.5 hours. `Retry-After` and `retry_after` give the seconds left. Three of these within the window ban your IP address; see [Call limits](#call-limits). | -| 503 | `unavailable` | Temporary server problem. Try again later; it doesn't use your call. | -| 503 | `slow_down` | More than 30 calls in the last minute from your IP address, across all these APIs. `Retry-After` and `retry_after` give the seconds left. It doesn't use your call and never counts toward a ban. | +| 404 | `zone_not_found` | The region is unknown, pending or disabled. | +| 405 | `method_not_allowed` | Use GET or an OPTIONS preflight. | +| 503 | `unavailable` | Temporary data problem; retain your previous copy and retry later. An admitted request still consumes quota. | +| 503 | `slow_down` | IP burst protection; wait for Retry-After. | + +The directory also returns `400 country_required` or `400 invalid_country` for a missing or malformed country. Results include only regions authorized by your key; group member lists are filtered to the same access scope. The sample above illustrates a key authorized for all listed regions. A group boundary request requires access to every enabled member. ## Example -Run this from a scheduled job on your own server, never in a loop and never from a browser. It fetches every Canadian region with a boundary into one GeoJSON file (one call to `get_zones.php`, then one call per region), tracking the last served call per URL so an early rerun skips instead of risking a ban: - -```python -import json, os, time, httpx - -STATE = "meshmapper-state.json" -last = json.load(open(STATE)) if os.path.exists(STATE) else {} - -def fetch(url, **params): - key = str(httpx.URL(url, params=params)) # the target: for get_zones this includes ?country=.. - if time.time() - last.get(key, 0) < 23.5 * 3600: - return None # already used this call recently - r = httpx.get(url, params=params, timeout=120) - if r.status_code != 200: - return None # keep the previous copy, don't retry here - last[key] = time.time() - return r.json() - -zones = fetch("https://meshmapper.net/get_zones.php", country="CA") -features = [] -if zones: - for z in zones["zones"]: - if z["has_boundary"]: - fc = fetch(z["url"] + "get_geojson.php") - if fc: - features.extend(fc["features"]) - time.sleep(2) # stay under the 30-calls-a-minute cross-API cap - json.dump({"type": "FeatureCollection", "features": features}, open("meshmapper-ca.geojson", "w")) -json.dump(last, open(STATE, "w")) +With `MESHMAPPER_API_KEY` set in your backend environment: + +```bash +curl --fail-with-body --compressed \ + -H "X-API-Key: $MESHMAPPER_API_KEY" \ + 'https://meshmapper.net/get_zones.php?country=CA' ``` + +Schedule polling to fit your key budget, retain the last successful data on failure, and respect `Retry-After`. diff --git a/mkdocs.yml b/mkdocs.yml index 8df4545..5b08fc5 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -48,6 +48,7 @@ nav: - Admin Portal: admins.md - Region Boundaries: region_boundaries.md - Webhooks: webhooks.md + - API Keys and Access: api-keys.md - Coverage API: coverage-api.md - Zones API: zones-api.md - Scopes API: scopes-api.md