Skip to content

Request: AppBench evaluation licence, official harness, and grading route #1

Description

@arvindcr4

Request: AppBench evaluation licence, official harness, and grading route

Hello AfterQuery team,

I am preparing a reproducible academic evaluation using the public
AfterQuery/App-Bench dataset pinned at revision
de80d5bcd404adee5307311571e512b5c37e6112. The dataset currently exposes
only AppBench vExternal.csv: its dataset card is null and the revision has no
licence, terms, or notice file.

Could you point me to the official channel for the following, or provide them
here if public?

  1. Written permission or a licence for evaluation use and publication of
    aggregate results.
  2. The official deployment/reset runner or a provider-hosted evaluation path,
    including artifact and side-effect verification.
  3. The required two-grader protocol, re-adjudication rules, and whether a
    provider-run grader service is available (including terms and price).
  4. The authoritative held-out/contamination policy for the six public tasks.

I will not substitute a local protocol, appoint graders, or report a benchmark
score until these requirements have immutable evidence. I am not requesting
credentials or initiating a paid evaluation through this issue.

The same request is open in the dataset discussion:
https://huggingface.co/datasets/AfterQuery/App-Bench/discussions/2

Thank you.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions