Merge branch 'main' into fix/fetch-schema-validate-before-write

Author: LukasGold

.gitignore

@@ -69,5 +69,8 @@ playground
 /osw_files/
 */accounts.pwd.yaml
 /accounts.pwd.yaml
-.ign
-.claude
+
+# Local folders
+.ign/
+.claude/
+graphify-out/

CHANGELOG.md

@@ -3,7 +3,7 @@ message: "If you use this software, please cite it as below."
 type: software
 title: "OpenSemanticWorld Python Package (osw-python)"
 # Managed by python-semantic-release (version_variables). Do not edit by hand.
-version: 2.3.1
+version: 2.6.0
 abstract: "Python toolset for data processing, queries, wikicode generation and page manipulation within OpenSemanticLab: work with OpenSemanticLab instances as typed pydantic entities, query with semantic search, generate models from wiki schemas, and write changes back."
 repository-code: "https://github.com/OpenSemanticLab/osw-python"
 url: "https://opensemanticlab.github.io/osw-python/"

CITATION.cff

@@ -0,0 +1,2 @@
+osw-python
+Copyright 2022-2026 Simon Stier and contributors

LICENSE.txt

@@ -22,7 +22,7 @@ pip install osw
 ```
 
 Optional extras (`osw[wikitext]`, `osw[DB]`, `osw[S3]`, `osw[dataimport]`,
-`osw[UI]`, `osw[all]`) are described in the
+`osw[UI]`, `osw[mcp]`, `osw[all]`) are described in the
 [Get Started guide](https://opensemanticlab.github.io/osw-python/get-started/).
 
 ## Quickstart
@@ -39,6 +39,26 @@ More runnable scripts live in [examples/](examples/), and the
 [Basics tutorial](docs/tutorials/basics.ipynb) walks through the
 OpenSemanticLab data model.
 
+## Tools
+
+Installing `osw` also installs an `osw` command line client, and the
+`osw[mcp]` extra adds an MCP server that exposes a live instance to agent
+clients such as Claude Code:
+
+```bash
+osw search ask '[[Category:Item]]' --limit 5
+```
+
+Commands, tools and their configuration are described in the
+[Tools guide](https://opensemanticlab.github.io/osw-python/tools/).
+
+## Logging
+
+osw reports what it is doing on the `osw` logger at INFO by default. Levels,
+the `OSW_LOG_LEVEL` environment variable and how to collect the records in your
+own logging setup are described in the
+[Get Started guide](https://opensemanticlab.github.io/osw-python/get-started/#logging).
+
 ## Contributing
 
 Contributions are welcome, see [CONTRIBUTING.md](CONTRIBUTING.md).

NOTICE

@@ -30,6 +30,7 @@
 | `osw[S3]` | Interact with S3 stores per S3FileController |
 | `osw[dataimport]` | Additional tools to import data |
 | `osw[UI]` | To use a helper UI to work with entity slots |
+| `osw[mcp]` | [MCP server](tools/mcp.md) for agent clients |
 | `osw[all]` | All of the above |
 
 Install multiple extras with `pip install osw[opt1,opt2]`.
@@ -64,6 +65,69 @@ Credentials are resolved from the environment variables `OSW_USERNAME` /
 credentials file, or via an interactive prompt - and are held in memory
 only, never written to disk; see [Authentication](api/auth.md).
 
+## Logging
+
+osw reports what it is doing through the standard `logging` module, on the
+`osw` logger, at INFO by default:
+
+```python
+import osw
+
+osw.set_log_level("WARNING")  # see less
+osw.set_log_level("DEBUG")    # see more
+osw.disable_logging()         # detach the handler osw attached
+```
+
+Problems that osw can work around are WARNING records on the same logger, not
+Python `warnings`. A truncated query result and a page that does not exist are
+examples. A `warnings` filter or the `-W` option therefore has no effect on
+them. A message that repeats is also written out every time, where the warnings
+machinery would have shown it once. Use `set_log_level` to control them.
+
+The records go to `stderr`, which leaves `stdout` free for your program's own
+output. That matters for anything speaking a protocol over `stdout`, such as an
+MCP stdio server. Pass `osw.enable_logging(stream=...)` to send them elsewhere.
+
+Set `OSW_LOG_LEVEL` to a level name, a level number, or `OFF` to choose the
+level before the package is imported. `OFF` silences osw everywhere, including
+in your own handlers.
+
+### Collecting osw's records in your application
+
+Configure logging the way you normally would and osw's records arrive there,
+once:
+
+```python
+import logging
+import osw
+
+logging.basicConfig(level=logging.INFO, filename="app.log")
+```
+
+The `osw` logger propagates at all times, so the records reach your handlers
+whatever else happens. osw's own handler notices that something above it is
+listening, detaches itself so nothing is written twice, and gives back the
+level it had picked, so your level applies from then on. It makes no difference
+whether you configure logging before or after importing osw.
+
+A level you asked for is kept across that hand-over, so `set_log_level("DEBUG")`
+or `OSW_LOG_LEVEL=DEBUG` is how you pull osw's debug records into an aggregated
+setup while the rest of your application stays quieter.
+
+One case osw cannot detect is a handler added to the `osw` logger itself, since
+that is indistinguishable from one of its own. Call `disable_logging()` first if
+you do that.
+
+### Output from parallel batches
+
+Several osw calls process their input in parallel. What such a task prints, as
+opposed to logs, is collected while the batch runs and replayed afterwards, so
+that concurrent writing cannot garble the progress bar. That text arrives on the
+`osw.parallel.output` logger, one record per line, at INFO. Give that logger its
+own level or handler to keep the output of third-party code apart from osw's own
+records. Most osw calls replay it only when you pass `debug=True`; `copy_pages`
+always replays.
+
 ## Examples and tutorials
 
 - Runnable scripts in

README.md

@@ -0,0 +1,74 @@
+# CLI
+
+## Quick start
+
+Both adapters need an instance and credentials. The quickest start is a
+gitignored `.env` file in your project root:
+
+```dotenv
+OSW_DOMAIN=wiki-dev.open-semantic-lab.org
+OSW_USERNAME=your-user
+OSW_PASSWORD=your-password
+```
+
+The CLI searches upward from the working directory for it, so `osw status`
+now reports the instance, the username (whether it comes from `OSW_USERNAME`
+or a credential file), and connection state. The MCP server takes its
+settings from the `env` block of its registration instead, see
+[Registering a server](mcp.md#registering-a-server). Every variable is listed
+under [Configuration](configuration.md).
+
+## Command line
+
+```bash
+osw status
+osw search ask '[[Category:Item]]' --limit 5
+osw entity get 'Item:OSW1234...' --json | jq .
+osw file cat 'File:Example.csv'                            # inline text
+osw file download 'File:Example.csv' --target-dir ./tmp    # to disk
+```
+
+Commands are grouped by subject:
+
+| Group | Commands |
+| --- | --- |
+| `entity` | `get`, `put`, `export`, `delete` |
+| `file` | `info`, `cat`, `write`, `download`, `upload` |
+| `search` | `ask`, `titles`, `content`, `entities`, `sparql` |
+| `slot` | `list`, `get`, `set` |
+| `schema` | `get` |
+| `instances` | `list`, `status` |
+| `ledger` | `path` |
+| top level | `status` |
+
+`osw search entities` finds pages in the wiki that are instances of a
+category, while `osw instances` is about the OSL servers this process can
+connect to.
+
+`osw instances list` lists the iris the process can connect to: the
+env-configured domain plus every entry of a configured credential file.
+
+`osw instances status` reports the same instances in more detail. For each
+one it prints the iri, whether it is the active one, the username that would
+be used, and whether a connection succeeded. It never prints passwords. The
+instances are contacted one after another and a single attempt has no
+timeout, so an unreachable instance delays the command until its connection
+attempt gives up.
+
+Global options apply to every command. They belong to the `osw` command
+itself, so they come before the subcommand, the same way `git` and `docker`
+options do: `osw --instance <iri> status`, not `osw status --instance <iri>`.
+Typing them after the subcommand now produces an error that names the correct
+form.
+
+- `--instance IRI` picks the instance. Optional: it is only required when
+  `OSW_DOMAIN` is not set and the configured credential file holds more than
+  one iri.
+- `--json` / `-j` writes machine-readable JSON to stdout and keeps osw's own
+  progress output on stderr, so it pipes cleanly into `jq`.
+- `--read-only` refuses write operations.
+- `--verbose` / `-v` shows full tracebacks instead of a one-line message, and
+  adds the env-file line to the source report described under
+  [Where settings come from](configuration.md#where-settings-come-from).
+
+Failures exit non-zero with a short message on stderr and no traceback.

docs/get-started.md

@@ -0,0 +1,125 @@
+# Configuration
+
+Both adapters share the settings below.
+
+## Where settings come from
+
+Settings are read from the process environment. A `.env` file fills that
+environment; a real environment variable wins over the same name in a file.
+
+**Env file**
+
+| `OSW_ENV_FILE` | CLI | MCP server |
+| --- | --- | --- |
+| set | loads that file, searches nowhere | loads that file, searches nowhere |
+| unset | searches upward from the working directory | searches nowhere |
+
+**Credential file.** The first step that produces a file wins:
+
+1. `OSW_CRED_FILEPATH` or an alias, set in the environment or the env file.
+   The run fails if this file has no entry for `OSW_DOMAIN`. That check is
+   skipped when `OSW_USERNAME` and `OSW_PASSWORD` are both set.
+2. CLI only: `accounts.pwd.yaml` in the working directory. Parent directories
+   are not searched. This step is skipped when `OSW_USERNAME` or
+   `OSW_PASSWORD` is set. If the file has no entry for `OSW_DOMAIN` it is
+   ignored and the run continues.
+3. No credential file.
+
+**Source report.** Both adapters write to stderr before connecting. The first
+line is labelled `credential file` when a file was found:
+
+- `<path> (from the OSW_CRED_FILEPATH environment variable)`
+- `<path> (from OSW_CRED_FILEPATH in the env file)`
+- `<path> (accounts.pwd.yaml found in the working directory)`
+- `<path> (accounts.pwd.yaml found in the working directory, ignored: no entry for domain '<domain>')`
+
+and `credentials` when none was:
+
+- `OSW_USERNAME/OSW_PASSWORD (from the environment)`
+- `OSW_USERNAME/OSW_PASSWORD (from the env file)`
+- `not configured (set OSW_CRED_FILEPATH, or OSW_USERNAME/OSW_PASSWORD)`
+
+The second line is labelled `env file`. Which lines appear depends on the
+adapter:
+
+- **CLI**: the first line only. `--verbose`, or a command that fails, adds
+  the second.
+- **MCP server**: neither, since its sources are fixed in the server entry.
+  `OSW_VERBOSE=true` prints both, and a failed start prints both regardless.
+
+A verbose run of the CLI prints:
+
+```text
+[osw] credential file: /home/me/project/accounts.pwd.yaml (accounts.pwd.yaml found in the working directory)
+[osw] env file       : /home/me/project/.env (found from the working directory upward)
+```
+
+The prefix names the adapter that printed the line: `[osw]` for the CLI,
+`[osw-mcp]` for the MCP server. This holds for every message the two share,
+not only these two lines.
+
+## Where messages go
+
+The source report above is printed directly, because the adapter's own verbose
+flag decides whether it appears, not the log level.
+
+Every other message the adapters produce goes to the `osw` logger, together
+with the records of the library itself. A failed connection check and an
+unreadable provenance ledger are reported that way. `OSW_LOG_LEVEL` sets how
+much of it appears, and an application that configures logging itself takes the
+records over. See [Logging](../get-started.md#logging).
+
+Both kinds of message are written to stderr, never to stdout. The MCP server
+speaks JSON-RPC over stdout, and the CLI writes its `--json` output there, so
+stdout has to stay free.
+
+## Credentials
+
+Keep credentials in a gitignored file. They are read once per process, into that
+process only, and never written back to disk. Set either `OSW_USERNAME` and
+`OSW_PASSWORD`, or `OSW_CRED_FILEPATH`.
+
+A credential file uses the YAML format osw's `CredentialManager` reads, keyed
+by iri (default file name: `accounts.pwd.yaml`):
+
+```yaml
+wiki-dev.open-semantic-lab.org:
+  username: your-user
+  password: your-password
+```
+
+A credential file may hold several iris. The CLI selects one automatically if it
+is the only one, and otherwise requires `osw --instance <iri>`. The MCP server
+never selects one, see [One server per instance](mcp.md).
+
+## Variable reference
+
+The canonical variable names are `OSW_*`. Older `OSW_MCP_*` and `OSL_*` names
+stay accepted so existing deployments keep working, and the first name that is
+set wins:
+
+| Canonical | Also accepted | Meaning |
+| --- | --- | --- |
+| `OSW_DOMAIN` | `OSL_DOMAIN` | Instance to connect to. A bare host (`wiki.example.org`) or a full URL (`https://wiki.example.org/w/`); the host is taken from either, and a value no host can be read from (`https://`, `/w/index.php`) is rejected at startup |
+| `OSW_USERNAME` | `OSL_USERNAME` | Login user |
+| `OSW_PASSWORD` | `OSL_PASSWORD` | Login password |
+| `OSW_CRED_FILEPATH` | `OSW_MCP_CRED_FILEPATH`, `OSL_CRED_FILEPATH` | YAML credential file, keyed by iri (falls back to `accounts.pwd.yaml` in the working directory, CLI only) |
+| `OSW_ENV_FILE` | `OSW_MCP_ENV_FILE` | `.env` file to load |
+| `OSW_READ_ONLY` | `OSW_MCP_READ_ONLY` | `true` refuses every write |
+| `OSW_SPARQL_ENDPOINT` | | Endpoint for `sparql` queries |
+| `OSW_STATE_DIR` | `OSW_MCP_STATE_DIR` | Where the provenance ledger is kept. Must be an absolute path; a leading `~` is expanded |
+| `OSW_MAX_RESULTS` | `OSW_MCP_MAX_RESULTS` | Default result cap (100) |
+| `OSW_MAX_CHARS` | `OSW_MCP_MAX_CHARS` | Result size cap in characters (100000) |
+| `OSW_VERBOSE` | `OSW_MCP_VERBOSE` | `true` prints the configuration source report |
+
+## Windows paths in a `.env` file
+
+Quote them with single quotes, or leave them unquoted. A double-quoted value is
+escape-decoded, so `\a` in a path silently becomes a BEL byte that renders as
+nothing:
+
+```dotenv
+OSW_CRED_FILEPATH='C:\Users\me\accounts.pwd.yaml'   # ok
+OSW_CRED_FILEPATH=C:\Users\me\accounts.pwd.yaml     # ok
+OSW_CRED_FILEPATH="C:\Users\me\accounts.pwd.yaml"   # broken: \a is eaten
+```

docs/tools/cli.md

@@ -0,0 +1,41 @@
+# Tools
+
+Besides the Python API, osw ships two adapters that talk to a live instance:
+the `osw` command line client, and an MCP server for agent clients such as
+Claude Code. Both run the same operations from one shared, SDK-free core
+(`osw.service`), so a command and its matching tool behave identically. They
+differ in exactly one way: only the CLI accepts filesystem paths.
+
+## Setup
+
+Install one of the two; the second includes the first:
+
+```bash
+uv tool install osw             # the `osw` command
+uv tool install "osw[mcp]"      # the same, plus the `osw-mcp` server
+```
+
+<details markdown="1">
+<summary>Other ways to install</summary>
+
+```bash
+pip install "osw[mcp]"          # into the active environment
+uv add "osw[mcp]"               # as a dependency of the current uv project
+uvx --from "osw[mcp]" osw-mcp   # run the server without installing it
+```
+
+`uvx` is what the registration examples further down use, so the server needs
+no install of its own.
+
+</details>
+
+`osw[mcp]` is also part of `osw[all]`. The other extras are listed in the
+[Get Started guide](../get-started.md#optional-extras).
+
+## In this section
+
+| Page | Contents |
+| ---- | -------- |
+| [CLI](cli.md) | The `.env` quick start, the full command reference, and the global flags |
+| [MCP server](mcp.md) | The tool surface, the no-filesystem-access and one-server-per-instance rules, and how to register the server with a client such as Claude Code |
+| [Configuration](configuration.md) | What both adapters share: where credentials and settings come from, and the full environment-variable reference |