Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 17 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ working wiring.
| --- | --- | --- | --- |
| `tinycortex` | TinyCortex, in-process | embedded | 3 (mandatory) via `provider`; all 18 via `TinycortexProvider` |
| `supermemory` | Supermemory, hosted | external | 3 (mandatory) |
| `mem0` | Mem0, self-hosted | external | 3 (mandatory) |
| `mem0` | Mem0, hosted (`cloud`) or self-hosted | external | 3 (mandatory) |
Comment thread
coderabbitai[bot] marked this conversation as resolved.
| `cognee` | Cognee, hosted or self-hosted | external | 3 (mandatory) |
| `memory-git` | add-on: git-backed diff snapshots | — | requires `tinycortex` |
| *(none)* | `NullMemoryProvider` | null | contract + registry only, 40 crates |
Expand Down Expand Up @@ -177,10 +177,10 @@ that skips enforcement is the entire reason the policy layer exists.
## Remote engines

The `tinymemory-remote` crate supports the managed and self-hosted native APIs
of Supermemory and Cognee, plus self-hosted Mem0. Each adapter stores
TinyMemory's key, category, session, and provenance in backend metadata (or a
Cognee raw-data envelope), so exact CRUD and portability survive the seam while
recall remains engine-native. Provider-facing dataset names, container tags,
of Supermemory, Cognee, and Mem0. Each adapter stores TinyMemory's key,
category, session, and provenance in backend metadata (or a Cognee raw-data
envelope), so exact CRUD and portability survive the seam while recall remains
engine-native. Provider-facing dataset names, container tags,
and filenames are bounded stable hashes, so every namespace and key accepted by
the TinyMemory contract remains valid on the remote API.

Expand All @@ -196,19 +196,23 @@ Managed APIs have explicit constructors so their authentication cannot be
confused with a self-hosted token:

```rust
use tinymemory_remote::{CogneeMemory, SupermemoryMemory};
use tinymemory_remote::{CogneeMemory, Mem0Memory, SupermemoryMemory};

let cognee = CogneeMemory::cloud("cognee-api-key")?;
let supermemory = SupermemoryMemory::cloud("sm_...")?;
// Cognee Cloud issues a per-tenant base URL (the API-key dashboard shows it);
// there is no shared endpoint, so its constructor takes one.
let cognee = CogneeMemory::api("https://tenant-<uuid>.aws.cognee.ai", "cognee-api-key")?;

// Cognee also issues tenant-specific API origins.
let tenant = CogneeMemory::api("https://tenant.example.cognee.ai", "api-key")?;
# Ok::<_, anyhow::Error>((cognee, supermemory, tenant))
// Supermemory and Mem0 both serve one hosted origin, so theirs take only a key.
let supermemory = SupermemoryMemory::cloud("sm_...")?;
let mem0 = Mem0Memory::cloud("m0-...")?;
# Ok::<_, anyhow::Error>((cognee, supermemory, mem0))
```

Cognee Cloud uses `X-Api-Key`; authenticated self-hosted Cognee uses a bearer
access token. Supermemory uses bearer API keys for both deployment modes. All
constructors redact credentials from `Debug` output and transport errors.
access token. Supermemory uses bearer API keys for both deployment modes. Mem0's
hosted platform uses `Authorization: Token`, and self-hosted Mem0 uses
`X-API-Key`. All constructors redact credentials from `Debug` output, from
transport errors, and from the request's own header rendering.

All three advertise the mandatory Core, Recall, and Portability families. The
live Docker harness and conformance command are documented in
Expand Down
30 changes: 14 additions & 16 deletions adapters/remote/src/cognee.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,6 @@ use crate::common::{stable_id, Dialect, HttpClient, RemoteMemory, StoredEntry};
/// Stable driver id used by configuration and status output.
pub use tinymemory_api::drivers::COGNEE_DRIVER_ID;

/// Default base URL for Cognee's managed API.
pub const COGNEE_API_ENDPOINT: &str = "https://api.cognee.ai";

/// A Cognee managed or self-hosted service exposed through TinyMemory's contract.
#[derive(Debug)]
pub struct CogneeMemory {
Expand Down Expand Up @@ -51,10 +48,20 @@ impl CogneeMemory {
})
}

/// Connect to a Cognee managed API using `X-Api-Key` authentication.
/// Connect to Cognee Cloud using `X-Api-Key` authentication.
///
/// `endpoint` is **your tenant's** base URL, which Cognee Cloud issues per
/// account and prints on the API-key dashboard — it looks like
/// `https://tenant-<uuid>.aws.cognee.ai`. There is deliberately no shared
/// default: this crate carried a `COGNEE_API_ENDPOINT` pointing at
/// `api.cognee.ai`, and that host answers no TLS handshake at all (its DNS
/// record resolves, nothing listens), so every "just use the default"
/// caller met a confusing transport error instead of a working client.
/// The tenant URL is the only address that exists.
///
/// This accepts a custom endpoint because Cognee Cloud may issue a
/// tenant-specific base URL. Use [`Self::cloud`] for the shared default.
/// The tenant and user ids the dashboard shows alongside the URL are not
/// needed here: the tenant is identified by the hostname, and the API's
/// only security scheme is this key.
///
/// # Errors
///
Expand All @@ -70,15 +77,6 @@ impl CogneeMemory {
}),
})
}

/// Connect to Cognee's shared managed API endpoint.
///
/// # Errors
///
/// Returns an error when `api_key` is blank.
pub fn cloud(api_key: &str) -> anyhow::Result<Self> {
Self::api(COGNEE_API_ENDPOINT, api_key)
}
}

#[async_trait]
Expand Down Expand Up @@ -183,7 +181,7 @@ impl CogneeDialect {
async fn datasets(&self) -> anyhow::Result<Vec<Dataset>> {
let response: Value = self
.client
.json(Method::GET, "api/v1/datasets", None)
.json(Method::GET, "api/v1/datasets/", None)
.await?;
Ok(response
.as_array()
Expand Down
5 changes: 4 additions & 1 deletion adapters/remote/src/cognee_test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,10 @@ fn cognee_remote_names_are_bounded_and_safe_for_arbitrary_contract_keys() {
async fn native_cognee_round_trips_the_tinymemory_contract() {
let state = AppState::default();
let app = Router::new()
.route("/api/v1/datasets", get(datasets))
// The real API serves the collection at the slashed form and 307s the
// bare one; the adapter now asks for `/api/v1/datasets/` directly, so
// the double must answer there or it stops mirroring the service.
.route("/api/v1/datasets/", get(datasets))
.route("/api/v1/datasets/{dataset}/data", get(data))
.route("/api/v1/datasets/{dataset}/data/{data}/raw", get(raw))
.route("/api/v1/datasets/{dataset}/data/{data}", delete(remove))
Expand Down
141 changes: 133 additions & 8 deletions adapters/remote/src/common.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ use std::collections::BTreeMap;

use anyhow::{bail, Context};
use async_trait::async_trait;
use reqwest::header::{HeaderValue, AUTHORIZATION};
use reqwest::{Method, RequestBuilder, StatusCode, Url};
use serde::{de::DeserializeOwned, Deserialize, Serialize};
use sha2::{Digest, Sha256};
Expand All @@ -28,6 +29,13 @@ enum Auth {
None,
Bearer(String),
ApiKey(String),
/// `Authorization: Token <key>` — Mem0's hosted platform.
///
/// Distinct from [`Auth::Bearer`] on the wire *and* in behaviour:
/// api.mem0.ai routes a `Bearer` credential into its JWT verifier and
/// answers `token_not_valid`, so sending the wrong one of the two reports
/// a failure in the wrong subsystem.
Token(String),
}

impl std::fmt::Debug for HttpClient {
Expand Down Expand Up @@ -90,6 +98,27 @@ async fn read_capped(response: reqwest::Response, path: &str) -> anyhow::Result<
Ok(body)
}

/// Wraps a credential in a header value that will not be printed back out.
///
/// `RequestBuilder::bearer_auth` marks its `Authorization` value sensitive on
/// the caller's behalf; `RequestBuilder::header` handed a plain string does
/// not. So the two schemes that have no such helper -- `X-API-Key` and
/// `Authorization: Token` -- would otherwise carry a live API key through
/// every `Debug` rendering of the request and through any middleware that
/// formats headers. The flag is set here instead.
///
/// Parsing up front is the second half of the same fix: a credential holding a
/// newline or another byte no header may carry becomes an error at the call
/// site, naming the credential, rather than a deferred failure inside `send`
/// that reads as a transport fault. The parse error carries no value, so the
/// credential does not reach the message either.
fn credential_header(value: &str) -> anyhow::Result<HeaderValue> {
let mut header =
HeaderValue::from_str(value).context("credential is not a valid HTTP header value")?;
header.set_sensitive(true);
Ok(header)
}

impl HttpClient {
/// Builds a client that optionally authenticates with a bearer token.
pub(crate) fn bearer(endpoint: &str, credential: Option<&str>) -> anyhow::Result<Self> {
Expand All @@ -99,6 +128,14 @@ impl HttpClient {
)
}

/// A client authenticating with `Authorization: Token <key>`.
pub(crate) fn token(endpoint: &str, credential: Option<&str>) -> anyhow::Result<Self> {
Self::new(
endpoint,
credential.map_or(Auth::None, |value| Auth::Token(value.into())),
)
}

/// Builds a client that optionally authenticates with `X-API-Key`.
pub(crate) fn api_key(endpoint: &str, credential: Option<&str>) -> anyhow::Result<Self> {
Self::new(
Expand Down Expand Up @@ -136,7 +173,10 @@ impl HttpClient {
Ok(match &self.auth {
Auth::None => request,
Auth::Bearer(token) => request.bearer_auth(token),
Auth::ApiKey(key) => request.header("X-API-Key", key),
Auth::ApiKey(key) => request.header("X-API-Key", credential_header(key)?),
Auth::Token(key) => {
request.header(AUTHORIZATION, credential_header(&format!("Token {key}"))?)
}
})
}

Expand Down Expand Up @@ -180,23 +220,39 @@ impl HttpClient {
/// rejected credential specifically, because "HTTP 401" three layers deep
/// in an anyhow chain reads as "the engine is down" and sends the operator
/// to the wrong runbook.
fn status_error(&self, path: &str, status: reqwest::StatusCode) -> anyhow::Error {
fn status_error(&self, path: &str, status: reqwest::StatusCode, body: &str) -> anyhow::Error {
let host = self.endpoint.host_str().unwrap_or("<endpoint>");
// Hosted engines explain a rejection in the response body — mem0
// answers `{"detail": "..."}`, cognee likewise — and discarding it
// turned "this one field is invalid" into a bare status code that
// said only that something, somewhere, was wrong. Truncated because
// an error body is not a payload budget, and only ever an error
// body: success responses never reach here.
let detail = body.trim();
let detail = if detail.is_empty() {
String::new()
} else {
let mut shown: String = detail.chars().take(300).collect();
if detail.chars().count() > 300 {
shown.push('…');
}
format!(" — {shown}")
};
match status.as_u16() {
401 | 403 => {
let hint = match &self.auth {
Auth::ApiKey(_) => "check the API key",
Auth::ApiKey(_) | Auth::Token(_) => "check the API key",
Auth::Bearer(_) => "check the bearer token",
Auth::None => {
"the endpoint requires credentials this client was not configured with"
}
};
anyhow::anyhow!(
"memory API {path} on {host}: the configured credential was rejected \
(HTTP {status}) — {hint}"
(HTTP {status}) — {hint}{detail}"
)
}
_ => anyhow::anyhow!("memory API {path} on {host} returned HTTP {status}"),
_ => anyhow::anyhow!("memory API {path} on {host} returned HTTP {status}{detail}"),
}
}

Expand All @@ -216,7 +272,8 @@ impl HttpClient {
.map_err(|error| self.transport_error(error))?;
let status = response.status();
if !status.is_success() {
return Err(self.status_error(path, status));
let body = response.text().await.unwrap_or_default();
return Err(self.status_error(path, status, &body));
}
let body = read_capped(response, path).await?;
serde_json::from_slice(&body)
Expand All @@ -232,7 +289,8 @@ impl HttpClient {
.map_err(|error| self.transport_error(error))?;
let status = response.status();
if !status.is_success() {
return Err(self.status_error(path, status));
let body = response.text().await.unwrap_or_default();
return Err(self.status_error(path, status, &body));
}
let body = read_capped(response, path).await?;
String::from_utf8(body).context("memory API response was not valid UTF-8")
Expand All @@ -255,7 +313,8 @@ impl HttpClient {
.map_err(|error| self.transport_error(error))?;
let status = response.status();
if !status.is_success() {
return Err(self.status_error(path, status));
let body = response.text().await.unwrap_or_default();
return Err(self.status_error(path, status, &body));
}
Ok(status)
}
Expand Down Expand Up @@ -587,6 +646,72 @@ fn classify_transport(is_timeout: bool, is_connect: bool, chain: &str) -> &'stat
}
}

#[cfg(test)]
mod credential_header_tests {
#![allow(clippy::expect_used, clippy::panic)]

use super::{credential_header, Auth, HttpClient};

/// The point of the helper. `reqwest` only redacts a header value whose
/// sensitive flag is set, and `RequestBuilder::header` handed a plain
/// string leaves it clear -- which is how an API key ends up rendered in
/// full by anything that formats the request.
#[test]
fn a_credential_header_is_marked_sensitive() {
let header = credential_header("Token m0-secret").expect("a plain key is a valid header");
assert!(header.is_sensitive());
}

/// The value still has to be the credential; marking it sensitive must not
/// change what goes on the wire.
#[test]
fn marking_it_sensitive_does_not_change_the_value() {
let header = credential_header("Token m0-secret").expect("valid");
assert_eq!(header.as_bytes(), b"Token m0-secret");
}

/// A credential carrying a newline cannot be a header. Rejecting it here
/// names the credential; letting it through defers the failure into `send`,
/// where it reads as a transport fault.
#[test]
fn a_credential_that_cannot_be_a_header_is_refused_by_name() {
let error = credential_header("key\r\nX-Injected: 1").expect_err("must not be accepted");
assert!(format!("{error}").contains("credential"), "got: {error}");
}

/// And the refusal must not print the credential it refused.
#[test]
fn the_refusal_does_not_echo_the_credential() {
let error =
credential_header("supersecret\nX-Injected: 1").expect_err("must not be accepted");
let rendered = format!("{error:?}");
assert!(!rendered.contains("supersecret"), "leaked: {rendered}");
}

/// Both credential-bearing schemes go through the helper, so both reach
/// the wire redacted. `Auth::Bearer` is covered by `reqwest`'s own
/// `bearer_auth`, which sets the flag itself.
#[test]
fn both_manual_schemes_send_a_sensitive_authorization_value() {
for auth in [
Auth::ApiKey("cg-secret".into()),
Auth::Token("m0-secret".into()),
] {
let client = HttpClient::new("https://example.test", auth).expect("valid endpoint");
let request = client
.request(reqwest::Method::GET, "v1/thing")
.expect("a plain key builds")
.build()
.expect("request builds");
let sensitive = request
.headers()
.values()
.any(reqwest::header::HeaderValue::is_sensitive);
assert!(sensitive, "no sensitive header on {:?}", request.headers());
}
}
}

#[cfg(test)]
mod transport_tests {
use super::classify_transport;
Expand Down
5 changes: 4 additions & 1 deletion adapters/remote/src/conformance_test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -485,7 +485,10 @@ async fn cg_recall(State(sets): State<Datasets>, Json(body): Json<Value>) -> Jso
async fn cognee_backend() -> String {
let sets: Datasets = Arc::new(Mutex::new(BTreeMap::new()));
let app = Router::new()
.route("/api/v1/datasets", get(cg_datasets))
// The real API serves the collection at the slashed form and 307s the
// bare one; the adapter now asks for `/api/v1/datasets/` directly, so
// the double must answer there or it stops mirroring the service.
.route("/api/v1/datasets/", get(cg_datasets))
.route("/api/v1/datasets/{dataset}/data", get(cg_data))
.route("/api/v1/datasets/{dataset}/data/{data_id}/raw", get(cg_raw))
.route(
Expand Down
Loading
Loading