Skip to content

refactor(agents): extract OpenClaw from BenchFlow core - #1093

Open
kywch wants to merge 6 commits into
mainfrom
refactor/openclaw-extraction
Open

refactor(agents): extract OpenClaw from BenchFlow core#1093
kywch wants to merge 6 commits into
mainfrom
refactor/openclaw-extraction

Conversation

@kywch

@kywch kywch commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Closes #1090: #1090

Summary

  • Moves OpenClaw shim, manifest, config, and tests to agents PR #72.
  • Deletes the 973-line built-in shim and OpenClaw-specific production policy.
  • Adds generic lazy manifest loading with local-checkout override support.
  • Makes unknown bare agent IDs fail closed; explicit commands remain supported.
  • Preserves catalog collisions, diagnostics, listing, and atomic publication.

Agents PR #72 must merge first.

Validation

  • Ruff and ty pass.
  • BenchFlow agent tests: 82 passed, 3 skipped.
  • Paired manifest parity: 12 passed.
  • Agents contract/OpenClaw tests: 98 passed.
  • Full suite: 5,952 passed; 3 host-sensitive failures reproduced on main.
  • Production Python: net -873 LOC.

@kywch
kywch deployed to pypi-internal-preview September 3, 2026 20:26 — with GitHub Actions Active
@bingran-you
bingran-you marked this pull request as ready for review September 3, 2026 20:59
@bingran-you
bingran-you requested a review from xdotli September 3, 2026 20:59
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-03T21:03:36.845766Z c3e770f Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@bingran-you
bingran-you deployed to pypi-internal-preview September 3, 2026 21:00 — with GitHub Actions Active
@bingran-you

Copy link
Copy Markdown
Collaborator

Thermo-nuclear review of exact head c3e770f.

The central ownership change is a net simplification: the 973-line OpenClaw implementation and its feature-specific registry policy leave core, while generic manifest selection/loading remains in focused modules. I found and repaired one cross-feature regression: the new fail-closed launch boundary also tried to resolve the non-agent task-runtime as an ACP launch during default setup. Setup now models agentless modes once and skips both web policy and launch lookup; a regression test exercises real default rollout planes.

Validation on the exact tree: 5,911 tests passed with 87 skipped and 7 deselected; ty and Ruff passed; the paired agents catalog gate passed 28 tests; local list/show resolved OpenClaw from the external manifest. The supplied DeepSeek key was first validated with a direct HTTP 200 call. A complete Docker hello-world run then installed the manifest-owned OpenClaw shim, produced two proxied HTTP 200 model exchanges with provider-reported usage, emitted one completed write tool call, scored reward 1, and recorded no agent or verifier error.

Merge gate: the behavior and structure are acceptable, but agents #71 and #72 must land first, this PR must be refreshed against their immutable merged source, exact-head CI must finish green, and independent human approval is still required before squash merge.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

Devin Review

Comment thread src/benchflow/agents/registry.py
Comment thread src/benchflow/agents/remote_manifests.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c3e770fa05

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/benchflow/agents/registry.py
Comment thread src/benchflow/agents/remote_manifests.py Outdated
@kywch
kywch deployed to pypi-internal-preview September 3, 2026 21:54 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extract OpenClaw shim from BenchFlow core into benchflow-ai/agents

2 participants