From 455ff2b1a77ee6e1ab59dfddc919d8f3fd74847e Mon Sep 17 00:00:00 2001 From: Levi Waldron Date: Sun, 13 Sep 2026 11:23:20 -0400 Subject: [PATCH 1/5] docs: streamline contribution onboarding flow Simplify README and CONTRIBUTING to make onboarding easier for new contributors. Shift focus to scientific correctness over syntax, point to template and issues, and clarify AUTHORING. --- AUTHORING.md | 121 +++++++++++++++++------------------------------- CONTRIBUTING.md | 74 +++++++++++------------------ README.md | 8 ++-- 3 files changed, 75 insertions(+), 128 deletions(-) diff --git a/AUTHORING.md b/AUTHORING.md index 6d1ee2e..c00e364 100644 --- a/AUTHORING.md +++ b/AUTHORING.md @@ -1,5 +1,7 @@ # Writing a protocol +Your primary focus should be **clarity, absence of ambiguity, scientific correctness, and accurate citation**. CI will validate formatting, but cannot validate the science for you. + [`CONTRIBUTING.md`](CONTRIBUTING.md) covers mechanics: where files go, how to bump a version, how to record a review. [`PROTOCOL_STANDARD.md`](https://github.com/waldronlab/agent-protocol-standard/blob/main/PROTOCOL_STANDARD.md) defines the format. This file covers what neither can check for you — what goes in a protocol, and @@ -9,21 +11,14 @@ where one protocol ends and the next starts. Write what to do and why, precisely enough that two people, or two agents working in two languages, get the same answer. Not an R script with comments. If a step only makes sense once you've read an -implementation, it isn't done. - -That's harder than it sounds, and it's the point. Code hides decisions in library defaults: which -linkage, which tie-breaker, which denominator, what happens to a missing value. Prose drags each one -into the open where someone can argue with it. Most of the work is finding the decisions your -reference implementation was making silently. - -Copy [`independent-filtering-variance`](protocols/independent-filtering-variance/protocol.md) for tone -and level of detail. +implementation, more detail is needed. -## Cite the method's origin, not its users +Note that code hides decisions such as default settings in library defaults, but such defaults can +change with package versions or between different implementations of a method. A protocol should specify +the procedure is enough detail not to be affected by such implementation differences. -An atomic protocol carries exactly one `method_citation`: the primary literature where the method was -first published. Not the paper you took the analysis from. Not the paper that made it popular. The one -that proposed it. This is important to ensuring accuracy of citation of original literature. +See [`independent-filtering-variance`](protocols/independent-filtering-variance/protocol.md) for an +example of appropriate tone and level of detail. ## The four citation and DOI fields @@ -36,72 +31,57 @@ Each name says what it identifies. `*_citation` points at other work; `*_doi` id | `artifact_doi` | **this document**, as a citable artifact | a DOI minted for it, e.g. from protocols.io | | `collection_doi` | the **repository or collection** housing it | a Zenodo record | -The first two get confused, because both can point at a paper containing the method. The difference: -`method_citation` is the method in general, `protocol_citation` is precise usage — the parameter values, -thresholds and choices this protocol fixes. +Protocols carry one `method_citation` and zero or one `protocol_citation`: -So one method can be the basis of several protocols. Random forest classification is one method with one -origin, but two published parameterizations are different procedures that give different answers from the -same data. Each is its own protocol. They share a `method_citation` and differ in `protocol_citation`. +- Atomic protocols can have method_citation and nothing else. Use protocol_citation only when a paper really does describe the procedure as you've written it, which is in some way distinguished from the general method. +- Composite protocols automatically inherit the `method_citation` from the protocols they comprise, provide one only if a primary publication describes the workflow as you've defined it. +- When to use `protocol_citation`: If a paper describes running a specific sequence of existing methods (e.g., a published pipeline or a specific analysis workflow). There would be no `protocol_citation` only if you are writing a first definition of the protocol. +`method_citation` is the method in general, `protocol_citation` is precise usage: the parameter values, +thresholds and choices this protocol specifies. Be sure to cite the primary literature that proposed the method +or defined the protocol, not a later paper that used the method or protocol. -Worth remembering, because it's what stops the one-method-one-citation rule from forcing unlike procedures -into one document. If two candidate protocols run the same method with different parameters from different -sources, the rule isn't telling you to merge them. Siblings are fine. +One method can be the basis of several protocols; for example, Random Forest classification is a method with one +origin, with many published parameterizations that give different answers from the +same data. Each is its own protocol; they share a `method_citation` but differ in `protocol_citation`. If you're unsure which field a DOI belongs in, ask whether it would still be right if you rewrote the steps. A method's origin survives a rewrite, so that's `method_citation`. A paper describing this -procedure doesn't, because it would no longer be the procedure described — `protocol_citation`. - -Example: "We employed random forest classification (`method_citation`) as implemented by Pasolli et al. 2016. (`protocol_citation`)." - -Most composites have no `method_citation` — they sequence existing protocols, propose nothing new, and -inherit their constituents' citations. A paper describing that kind of pipeline is a `protocol_citation`. -But a string of methods can be published as a method in its own right, and then `method_citation` is right: -a two-stage hierarchical meta-analysis is a method, and so is `humann4-database-build`. Ask whether the -paper proposes the combination as an approach people would cite, or just describes running the steps. - -Atomic protocols can have `method_citation` and nothing else. Use `protocol_citation` only when a -paper really does describe the procedure as you've written it, which is in some way distinguished from the general method. +procedure doesn't, so that's `protocol_citation`. ## If one paper didn't propose everything the protocol does, it's more than one protocol Examples from previous reviews of protocol proposals: -- **Microbe set enrichment.** Looked like one protocol with a method parameter. ORA, PADOG and CBEA have - three citations, three null hypotheses, and three different input types — a thresholded list, a labelled - abundance matrix, a compositional matrix. Three protocols, plus a composite that benchmarks them. -- **Prevalence filtering and CLR.** Drafted as one preprocessing protocol. Downstream analyses need to run - with the transformation and without it, so splitting made CLR an optional slot instead of a mandatory step. +- **Benchmarking of microbe set enrichment methods.** Looked like one protocol as described in the BugSigDB publication. +However it involves three different atomic enrichment protocols, and a composite benchmarking protocol. +- **Centered Log Ratio transformation (CLR) followed by filtering on microbial prevalence.** Seen multiple times in preprocessing workflows, but these are distinct protocols because they have different citations and can be used independently of each other. -The pattern: a bundled protocol may feel atomic when one paper happened to do all of it at once, but that's a fact -about the paper, not about the methods. +## When to stop splitting up steps into new protocols -## When to stop splitting +Splitting the steps of a protocol into more protocols can unnecessarily increase complexity. +Split a step of a protocol out into its own protocol only if **all three** are yes: -An atomic protocol is a composable unit that's useful as a whole in building analyses. Do not split further than necessary. Split only if all three are yes: - -1. **Would two different analyses use this on its own?** If it only ever shows up inside one bigger thing, +1. **Would different analyses use the step on their own?** If a step only ever shows up this protocol, splitting buys no reuse. -2. **Is there a plausible substitute?** Split where real analyses vary. CLR has - substitutes — arcsin-square-root, or no transformation — so the transformation is a unit. "Divide by the - geometric mean" has no substitute inside CLR, so it isn't. +2. **Is there a plausible substitute for the step?** Split where real analyses vary. CLR is a good atomic +protocol because it has substitutes: arcsin-square-root, log-transformation, or no transformation. +"Divide by the geometric mean"has no substitute inside CLR, so this does not motivate splitting this step +into its own protocol. 3. **Does it have a name people use?** A named method is citable; a step inside one isn't. Usually why the naming question and the citation question give the same answer. Another way to think about it is: in a methods section, would you say "we did X", ie name the procedure? If so, it's a protocol. Or would you say, "we did X as part of Y". If so, X is likely a step in protocol Y. -For example, "Abundances were -CLR-transformed" is one clause and one protocol. Nobody writes "we divided by the geometric mean and took -logarithms", because naming CLR already covers it. CLR can be considered an atomic protocol with two steps. - -Some cases are arguable. Make the call and say why in `## Notes`. +Some cases are arguable. Geometric mean _does_ have other uses, but for microbiome researchers I (Levi) +can only think of its use inside CLR, and it wouldn't be much burden to redefine it elsewhere if needed, so I +wouldn't split it out. Make the call and say why in `## Notes`. ## Alternatives inside one protocol, sometimes A protocol can offer a choice when the alternatives come from the same source, take the same inputs, -produce the same kind of output, and mean the same thing. Then it's a tuning knob, not a different method. +produce the same kind of output, and mean the same thing. Then it's a tuning knob, not a different protocol. -`independent-filtering-variance` is the precedent: `filter: variance | mean`, both from the same paper, one +[independent-filtering-variance](agent-protocols/protocols/independent-filtering-variance) is the precedent: `filter: variance | mean`, both from the same paper, one `method_citation` covering both, and the protocol says exactly one must be chosen, that they must not be applied in sequence, and that the choice is recorded before anyone looks at results. @@ -114,10 +94,10 @@ concordant. ## Say what the protocol doesn't do -Provide an explicit out-of-scope statement. It lets an agent +Provide an explicit out-of-scope statement. This lets an agent compose protocols without wondering whether two of them overlap, and it stops a protocol from quietly growing into a pipeline when the protocol is often followed by other downstream analysis. -For example, `independent-filtering-variance` ends by saying it doesn't define or perform hypothesis testing, p-value +For example, [independent-filtering-variance](agent-protocols/protocols/independent-filtering-variance) ends by saying it doesn't define or perform hypothesis testing, p-value calculation, multiple-testing adjustment, differential expression, or interpretation of discoveries. ## Say what has to be recorded @@ -130,39 +110,24 @@ Where a choice has to be made — a filter, a threshold, a candidate list — ma ## When you can't find a source -Rare, but some methods are old or folkloric enough to have no identifiable first publication. Say so in the -pull request instead of reaching for a convenient recent paper. +Rare, but some methods are old or folkloric enough to have no identifiable first publication. Say so instead of reaching for a convenient recent paper. A missing `method_citation` validates today, so nothing stops you leaving it out. Descriptive protocols that claim no method — a study table, a corpus summary — legitimately have nothing to cite. For a protocol that does run a named method, an empty field is a question to raise, not an answer. -Raise it in [`waldronlab/agent-protocol-standard`](https://github.com/waldronlab/agent-protocol-standard/issues) -— the standard may need a way to say "classical method, no primary source", and format decisions don't get -made in this repository. - ## Before you open the pull request Check: - For an atomic protocol claiming a method, that it is one method, and you can name the paper that proposed it. Composite: its steps name the constituent protocols and the order, without restating what those - protocols already say. Descriptive or reporting protocol: claims no method, and says so. -- It's a unit someone would use in more than one analysis, and there's a plausible substitute for it. + protocols already say. +- It's a unit that can be reused in different analyses. - `method_citation` is the method's origin. `protocol_citation`, if present, describes this procedure. - If a sibling protocol shares your `method_citation`, `## Notes` says how yours differs. - An expert in the field could have a complete picture of the protocol without reading any code. -- Every *optional* parameter has a stated default and a reason. Required inputs — a dataset, a covariate, a - candidate list — are named as required, with no unstated default. +- Every configurable parameter has a stated default. Required inputs (data, software, parameters) are stated in the Materials section. - The out-of-scope section exists and is specific. -- The validator passes: - - ```sh - git clone https://github.com/waldronlab/agent-protocol-standard.git - Rscript agent-protocol-standard/scripts/validate-protocol.R protocols - ``` - - It checks conformance, not correctness. It can't tell you that you cited the wrong paper, bundled two - methods, or left a decision implicit. That's what review is for. -The format is pre-1.0 (`spec_version: 0.1.0`) and still moving. If a rule here gets in the way of describing -a method honestly, raise it as an issue rather than working around it. The standard is young enough to fix. +The protocol standard is still pre-release and experimental. If a rule here gets in the way of describing +a protocol unambiguously, raise it as an issue rather than working around it. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9096dbf..1c28091 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,61 +1,41 @@ # Contributing -## The format is defined elsewhere +Your primary concern as a contributor is **clarity, absence of ambiguity, scientific correctness, and accurate citation**. Our continuous integration (CI) tools will automatically validate your formatting syntax when you open a pull request, so focus on the science. -[`PROTOCOL_STANDARD.md`](https://github.com/waldronlab/agent-protocol-standard/blob/main/PROTOCOL_STANDARD.md) -in `waldronlab/agent-protocol-standard` is the authority on what a `protocol.md` must contain. This -repository holds content only; it does not define or extend the format. If you believe the format -itself needs to change, open an issue there rather than working around it here. - -## Deciding what to write - -This file covers mechanics. [`AUTHORING.md`](AUTHORING.md) covers the judgement calls the mechanics -cannot check: finding the paper that actually proposed a method, recognizing when what looks like one -protocol is really two, when alternatives may live inside a single protocol, and what to state as out -of scope. Read it before writing a new protocol. - -## Adding a protocol - -1. Create `protocols//protocol.md`. The directory name must exactly match the `name` field in - the YAML frontmatter. -2. Fill in the required frontmatter — `name`, `description`, `version`, `authors`, `date`, `status` — - and the provenance fields that apply. An atomic protocol carries exactly one `method_citation` naming the - primary literature where the method was published. -3. End the file with a `## History & Reviews` section. A first release has one version entry, a - `#### Changes` bullet list, and `*No reviews yet.*` under `#### Reviews`; omit the `reviews:` - frontmatter field entirely until someone has actually reviewed it. -4. Open a pull request. CI validates every protocol in the repository and will tell you precisely - what is wrong. +## Step 1: Claim an Issue +Browse the [Issues](https://github.com/waldronlab/agent-protocols/issues) tab and look for the `good-first-protocol` label. +To claim an issue, **comment on that issue stating you are beginning work**. Use the issue thread to post questions, comments, and other follow-up while you draft the protocol. -## Changing an existing protocol +## Step 2: Set up from Template +Do not write the complex YAML frontmatter from scratch. Instead, copy the official boilerplate from the standard repository: +[Protocol Template](https://github.com/waldronlab/agent-protocol-standard/tree/main/template) -Bump `version:` and `date:`, and add a matching entry at the **top** of `## History & Reviews` with -the same version and date. The validator enforces that the top entry agrees with the frontmatter, so -a version bump without a history entry fails. +1. Create a directory `protocols//`. The directory name must exactly match the `name` field in your YAML frontmatter. +2. Copy the template into your new directory as `protocol.md`. -## Recording a review +## Step 3: Write +Read [`AUTHORING.md`](AUTHORING.md) for philosophical guidelines on how to write your protocol. It covers the judgement calls that CI cannot check: finding the paper that actually proposed a method, recognizing when what looks like one protocol is really two, and what to state as out of scope. -A review is recorded twice, and the two must agree: a `**Review by **` block under the version -that was reviewed, and an entry in the frontmatter `reviews:` array whose `protocol_version` is that -version. The validator compares the date, status, and ORCID across both, so a human reader and a -machine reader can never draw different conclusions. +Fill in the required frontmatter (`name`, `description`, `version`, `authors`, `date`, `status`, `method_citation`) and write the human-readable instructions. -A review applies to the version it was given for. Do not move an existing review onto a new release. +End the file with a `## History & Reviews` section as demonstrated in the template. Omit the `reviews:` frontmatter field entirely until someone has actually reviewed it. -## Do not edit `PROTOCOLS.yaml` +## Step 4: Validate and Submit +Open a pull request. **You can rely on CI for validation.** The repository's automated testing infrastructure will check your syntax and tell you precisely what is wrong. Focus on getting the scientific correctness right. -It is generated by CI on every push to `main` that changes a protocol or the generator workflow -itself — the index is a function of both. Hand edits are overwritten by the next such push — and -until then, the index disagrees with the protocols it claims to describe, which is worse than -either. +*(If you prefer to run the validator locally before pushing, you can clone `waldronlab/agent-protocol-standard` and run `Rscript scripts/validate-protocol.R `)* -## Running the validator locally +--- -Optional — CI runs it on every pull request — but faster to iterate against: +## Changing an existing protocol +Bump `version:` and `date:`, and add a matching entry at the **top** of `## History & Reviews` with the same version and date. The validator enforces that the top entry agrees with the frontmatter. -```sh -git clone https://github.com/waldronlab/agent-protocol-standard.git -Rscript agent-protocol-standard/scripts/validate-protocol.R protocols -``` +## Recording a review +A review is recorded twice, and the two must agree: a `**Review by **` block under the version that was reviewed, and an entry in the frontmatter `reviews:` array whose `protocol_version` is that version. The validator compares the date, status, and ORCID across both. A review applies to the version it was given for. Do not move an existing review onto a new release. + +## Do not edit `PROTOCOLS.yaml` +It is generated by CI on every push to `main` that changes a protocol or the generator workflow itself. Hand edits will be overwritten. -It needs the R packages `rmarkdown` and `yaml`. +## Reference: The Protocol Format +[`PROTOCOL_STANDARD.md`](https://github.com/waldronlab/agent-protocol-standard/blob/main/PROTOCOL_STANDARD.md) +in `waldronlab/agent-protocol-standard` is the definitive dictionary on what a `protocol.md` must contain. If you are unsure what a specific frontmatter field means, consult the standard. This repository holds content only; it does not define or extend the format. diff --git a/README.md b/README.md index 019e14b..91a8be2 100644 --- a/README.md +++ b/README.md @@ -18,9 +18,11 @@ never edit it by hand, and do not expect a hand-written list anywhere to be auth ## Contributing a protocol -Read [`PROTOCOL_STANDARD.md`](https://github.com/waldronlab/agent-protocol-standard/blob/main/PROTOCOL_STANDARD.md) -first; it is the authority on the format, and CI enforces it on every pull request. See -[CONTRIBUTING.md](CONTRIBUTING.md). +We welcome new protocols! Your primary concern as a contributor should be **clarity and scientific correctness**. Don't worry too much about formatting syntax—our continuous integration (CI) tools will automatically check that for you when you open a pull request. + +To get started: +1. **Find an idea:** Check our [Issues](https://github.com/waldronlab/agent-protocols/issues) tab and look for the `good-first-protocol` label to find a protocol that needs writing. +2. **Read the guide:** Follow the step-by-step instructions in [CONTRIBUTING.md](CONTRIBUTING.md) to claim an issue, set up your file from a template, and submit your work. ## Discovery From 985856a2b54ed69be1d667afac195cc36e981b8a Mon Sep 17 00:00:00 2001 From: Levi Waldron Date: Sun, 13 Sep 2026 11:40:31 -0400 Subject: [PATCH 2/5] Migrate to the two-question citation scheme MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follows waldronlab/agent-protocol-standard ADR 0014: `method_citation` becomes `method_origin_citation` and optional, `protocol_citation` becomes required. The six HUMAnN protocols cite the eLife HUMAnN paper as `protocol_citation`, since it describes the database-build procedures. `independent-filtering-variance` is a first definition — the procedure is written here, not transcribed — so it cites this collection's concept DOI, which is how a protocol states that rather than leaving a gap a reader has to interpret. AUTHORING.md's citation section is rewritten as the two questions with their own answers, rather than one distinction explained against itself. It no longer needs the rewrite-the-steps counterfactual. No version bumps: the procedures are unchanged and a bump would cascade through `humann4-database-build`'s five pinned dependencies, as with the `stable` change. Co-Authored-By: Claude Opus 5 (1M context) --- AUTHORING.md | 67 ++++++++++++------- CONTRIBUTING.md | 2 +- .../humann4-augmented-clustering/protocol.md | 5 +- .../humann4-chocophlan-build/protocol.md | 5 +- protocols/humann4-database-build/protocol.md | 5 +- protocols/humann4-sgb-aggregation/protocol.md | 5 +- .../protocol.md | 5 +- protocols/humann4-utility-mapping/protocol.md | 5 +- .../protocol.md | 6 +- 9 files changed, 60 insertions(+), 45 deletions(-) diff --git a/AUTHORING.md b/AUTHORING.md index c00e364..2a6732a 100644 --- a/AUTHORING.md +++ b/AUTHORING.md @@ -24,29 +24,50 @@ example of appropriate tone and level of detail. Each name says what it identifies. `*_citation` points at other work; `*_doi` identifies a thing. -| Field | Identifies | Points at | +Two of them are citations, and each answers one question. Answer them separately. + +| Field | The question | Required? | |---|---|---| -| `method_citation` | the **method** this protocol performs | the primary literature that proposed it | -| `protocol_citation` | a publication **describing or validating this protocol** | the procedure as written here, parameters included | -| `artifact_doi` | **this document**, as a citable artifact | a DOI minted for it, e.g. from protocols.io | -| `collection_doi` | the **repository or collection** housing it | a Zenodo record | +| `protocol_citation` | **Who published these instructions?** | Yes | +| `method_origin_citation` | **Who invented the method?** | No | +| `artifact_doi` | a DOI for **this document**, e.g. from protocols.io | No | +| `collection_doi` | a DOI for the **collection** housing it, e.g. a Zenodo record | No | + +### Who published these instructions? + +If you wrote this protocol from a paper's methods section, that paper. If you wrote it yourself, nobody +did — so name this protocol's own DOI instead: `artifact_doi` if it has one, otherwise `collection_doi`. + +That is not a formality. Repeating `collection_doi` here is how a protocol says **"first definition, +published here"**, and it is the normal answer for a protocol written in this repository: + +```yaml +# transcribed from a published analysis +protocol_citation: "10.1038/s41591-019-0405-7" + +# written here; nobody published this procedure before +protocol_citation: "10.5281/zenodo.22731694" # this repository's concept DOI +``` + +Use the concept DOI, not a version DOI. A version DOI would name the exact bytes, but you cannot write +the DOI of a release that does not exist yet. + +### Who invented the method? -Protocols carry one `method_citation` and zero or one `protocol_citation`: +The primary literature where the method was **first proposed** — not a paper that used it. Leave-one- +dataset-out cross-validation was first applied to microbiome data by Pasolli 2016, but proposed by +Riester 2014. Riester is the answer; citing Pasolli here is the mistake this field exists to prevent. -- Atomic protocols can have method_citation and nothing else. Use protocol_citation only when a paper really does describe the procedure as you've written it, which is in some way distinguished from the general method. -- Composite protocols automatically inherit the `method_citation` from the protocols they comprise, provide one only if a primary publication describes the workflow as you've defined it. -- When to use `protocol_citation`: If a paper describes running a specific sequence of existing methods (e.g., a published pipeline or a specific analysis workflow). There would be no `protocol_citation` only if you are writing a first definition of the protocol. -`method_citation` is the method in general, `protocol_citation` is precise usage: the parameter values, -thresholds and choices this protocol specifies. Be sure to cite the primary literature that proposed the method -or defined the protocol, not a later paper that used the method or protocol. +**Omit the field if the protocol originates no method.** Documenting how to build a tool's reference +database is not a method somebody proposed. An omitted field is honest; one filled in to satisfy CI is +not, and nothing in the validator can tell the difference. -One method can be the basis of several protocols; for example, Random Forest classification is a method with one -origin, with many published parameterizations that give different answers from the -same data. Each is its own protocol; they share a `method_citation` but differ in `protocol_citation`. +A composite inherits its constituents' origins automatically. Give it one of its own only where the +composition was itself published as a method. -If you're unsure which field a DOI belongs in, ask whether it would still be right if you rewrote the -steps. A method's origin survives a rewrite, so that's `method_citation`. A paper describing this -procedure doesn't, so that's `protocol_citation`. +One method can be the basis of several protocols. Random forest classification has one origin and many +published parameterizations that give different answers from the same data. Each is its own protocol: +they share a `method_origin_citation` and differ in `protocol_citation`. ## If one paper didn't propose everything the protocol does, it's more than one protocol @@ -82,10 +103,10 @@ A protocol can offer a choice when the alternatives come from the same source, t produce the same kind of output, and mean the same thing. Then it's a tuning knob, not a different protocol. [independent-filtering-variance](agent-protocols/protocols/independent-filtering-variance) is the precedent: `filter: variance | mean`, both from the same paper, one -`method_citation` covering both, and the protocol says exactly one must be chosen, that they must not be +`method_origin_citation` covering both, and the protocol says exactly one must be chosen, that they must not be applied in sequence, and that the choice is recorded before anyone looks at results. -`method_citation` is the tell. If the alternatives need two citations, they're two protocols. +`method_origin_citation` is the tell. If the alternatives need two citations, they're two protocols. This is how protocols work elsewhere too. Nature Protocols and protocols.io allow branch points freely, but for procedural variants of one method, not for swapping in a different method by a different author. OECD @@ -112,7 +133,7 @@ Where a choice has to be made — a filter, a threshold, a candidate list — ma Rare, but some methods are old or folkloric enough to have no identifiable first publication. Say so instead of reaching for a convenient recent paper. -A missing `method_citation` validates today, so nothing stops you leaving it out. Descriptive protocols that +A missing `method_origin_citation` validates today, so nothing stops you leaving it out. Descriptive protocols that claim no method — a study table, a corpus summary — legitimately have nothing to cite. For a protocol that does run a named method, an empty field is a question to raise, not an answer. @@ -123,8 +144,8 @@ does run a named method, an empty field is a question to raise, not an answer. it. Composite: its steps name the constituent protocols and the order, without restating what those protocols already say. - It's a unit that can be reused in different analyses. -- `method_citation` is the method's origin. `protocol_citation`, if present, describes this procedure. -- If a sibling protocol shares your `method_citation`, `## Notes` says how yours differs. +- `method_origin_citation` is the method's origin. `protocol_citation`, if present, describes this procedure. +- If a sibling protocol shares your `method_origin_citation`, `## Notes` says how yours differs. - An expert in the field could have a complete picture of the protocol without reading any code. - Every configurable parameter has a stated default. Required inputs (data, software, parameters) are stated in the Materials section. - The out-of-scope section exists and is specific. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1c28091..76d978e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -16,7 +16,7 @@ Do not write the complex YAML frontmatter from scratch. Instead, copy the offici ## Step 3: Write Read [`AUTHORING.md`](AUTHORING.md) for philosophical guidelines on how to write your protocol. It covers the judgement calls that CI cannot check: finding the paper that actually proposed a method, recognizing when what looks like one protocol is really two, and what to state as out of scope. -Fill in the required frontmatter (`name`, `description`, `version`, `authors`, `date`, `status`, `method_citation`) and write the human-readable instructions. +Fill in the required frontmatter (`name`, `description`, `version`, `authors`, `date`, `status`, `protocol_citation`) and write the human-readable instructions. Add `method_origin_citation` where the protocol performs a method someone proposed; see [AUTHORING.md](AUTHORING.md). End the file with a `## History & Reviews` section as demonstrated in the template. Omit the `reviews:` frontmatter field entirely until someone has actually reviewed it. diff --git a/protocols/humann4-augmented-clustering/protocol.md b/protocols/humann4-augmented-clustering/protocol.md index 2f5c836..101b9e9 100644 --- a/protocols/humann4-augmented-clustering/protocol.md +++ b/protocols/humann4-augmented-clustering/protocol.md @@ -21,9 +21,8 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - -method_citation: "10.1016/j.cell.2019.01.001" +protocol_citation: "10.7554/eLife.65088" +method_origin_citation: "10.1016/j.cell.2019.01.001" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/humann4-chocophlan-build/protocol.md b/protocols/humann4-chocophlan-build/protocol.md index 387900a..9177e9c 100644 --- a/protocols/humann4-chocophlan-build/protocol.md +++ b/protocols/humann4-chocophlan-build/protocol.md @@ -21,9 +21,8 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - -method_citation: "10.7554/eLife.65088" +protocol_citation: "10.7554/eLife.65088" +method_origin_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/humann4-database-build/protocol.md b/protocols/humann4-database-build/protocol.md index 0a576e2..28b37a7 100644 --- a/protocols/humann4-database-build/protocol.md +++ b/protocols/humann4-database-build/protocol.md @@ -22,11 +22,10 @@ type: composite artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - +protocol_citation: "10.7554/eLife.65088" # The bioBakery paper proposes this database-construction pipeline as a method, so it belongs here even # though the protocol is composite. The constituent protocols' own method_citations are inherited. -method_citation: "10.7554/eLife.65088" +method_origin_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/humann4-sgb-aggregation/protocol.md b/protocols/humann4-sgb-aggregation/protocol.md index d7d8244..76e713d 100644 --- a/protocols/humann4-sgb-aggregation/protocol.md +++ b/protocols/humann4-sgb-aggregation/protocol.md @@ -21,9 +21,8 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - -method_citation: "10.1016/j.cell.2019.01.001" +protocol_citation: "10.7554/eLife.65088" +method_origin_citation: "10.1016/j.cell.2019.01.001" upstream_repositories: - "https://github.com/biobakery/metaphlan" diff --git a/protocols/humann4-translated-search-build/protocol.md b/protocols/humann4-translated-search-build/protocol.md index 4c8c8fc..7adf4a8 100644 --- a/protocols/humann4-translated-search-build/protocol.md +++ b/protocols/humann4-translated-search-build/protocol.md @@ -21,9 +21,8 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - -method_citation: "10.7554/eLife.65088" +protocol_citation: "10.7554/eLife.65088" +method_origin_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/humann4-utility-mapping/protocol.md b/protocols/humann4-utility-mapping/protocol.md index 990c6d2..62d9ba8 100644 --- a/protocols/humann4-utility-mapping/protocol.md +++ b/protocols/humann4-utility-mapping/protocol.md @@ -21,9 +21,8 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - -method_citation: "10.7554/eLife.65088" +protocol_citation: "10.7554/eLife.65088" +method_origin_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/independent-filtering-variance/protocol.md b/protocols/independent-filtering-variance/protocol.md index 9880a3b..ef122e7 100644 --- a/protocols/independent-filtering-variance/protocol.md +++ b/protocols/independent-filtering-variance/protocol.md @@ -18,9 +18,9 @@ type: "atomic" artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ +protocol_citation: "10.5281/zenodo.22731694" license: "CC-BY-4.0" -method_citation: "10.1073/pnas.0914005107" +method_origin_citation: "10.1073/pnas.0914005107" protocols_used: [] category: "Statistical Analysis" tags: [filtering, variance, mean, high-throughput] @@ -86,7 +86,7 @@ This protocol covers only the filtering operation. Any downstream testing or mul - Rewrote the steps to be language-agnostic, removing the R implementation and describing the filtering operation in prose. - Added overall mean filtering as an alternative to overall variance filtering, matching the two strategies described in the source paper. - Clarified the `theta` cutoff and the use of `floor()` when determining how many features to remove. -- Removed `publication_doi`, which duplicated `citation`. *(Both fields were later renamed to `protocol_citation` and `method_citation`; this entry keeps the names in use at the time.)* +- Removed `publication_doi`, which duplicated `citation`. *(Both fields were later renamed to `protocol_citation` and `method_origin_citation`; this entry keeps the names in use at the time.)* #### Reviews From 1a280ebe779081e69c8e41bac3524f979edbe632 Mon Sep 17 00:00:00 2001 From: Levi Waldron Date: Sun, 13 Sep 2026 11:55:56 -0400 Subject: [PATCH 3/5] Address review: omit method origins that nobody proposed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The four database-build protocols — chocophlan, translated-search, utility- mapping, and the composite that sequences them — no longer carry a method_origin_citation. Building a tool's reference database originates no method, which is ADR 0014's own motivating example; keeping the eLife DOI there labelled that paper as the method's inventor rather than as the description of the procedure, which is what it is and where it now sits. The maintainer and Copilot reached this independently. independent-filtering-variance cites Bourgon 2010 as its protocol_citation rather than this collection. It encodes what that paper describes without fixing additional parameters or steps, so the paper published these instructions. Requiring a prespecified tie-breaker is operational detail that makes a published procedure executable, not a new procedure — if that were enough to constitute a first definition, nothing would ever be a transcription. No protocol in this repository is a first definition today. Also from the review, all pre-existing rather than introduced here: three typos in AUTHORING.md, two links that resolved to `agent-protocols/agent-protocols/` from the repository root, a checklist line still saying protocol_citation is optional, and a local validation command that passed a single protocol path where the validator takes a directory. Co-Authored-By: Claude Opus 5 (1M context) --- AUTHORING.md | 12 ++++++------ CONTRIBUTING.md | 2 +- protocols/humann4-chocophlan-build/protocol.md | 1 - protocols/humann4-database-build/protocol.md | 3 --- .../humann4-translated-search-build/protocol.md | 1 - protocols/humann4-utility-mapping/protocol.md | 1 - protocols/independent-filtering-variance/protocol.md | 2 +- 7 files changed, 8 insertions(+), 14 deletions(-) diff --git a/AUTHORING.md b/AUTHORING.md index 2a6732a..928e1e5 100644 --- a/AUTHORING.md +++ b/AUTHORING.md @@ -15,7 +15,7 @@ implementation, more detail is needed. Note that code hides decisions such as default settings in library defaults, but such defaults can change with package versions or between different implementations of a method. A protocol should specify -the procedure is enough detail not to be affected by such implementation differences. +the procedure in enough detail not to be affected by such implementation differences. See [`independent-filtering-variance`](protocols/independent-filtering-variance/protocol.md) for an example of appropriate tone and level of detail. @@ -82,11 +82,11 @@ However it involves three different atomic enrichment protocols, and a composite Splitting the steps of a protocol into more protocols can unnecessarily increase complexity. Split a step of a protocol out into its own protocol only if **all three** are yes: -1. **Would different analyses use the step on their own?** If a step only ever shows up this protocol, +1. **Would different analyses use the step on their own?** If a step only ever shows up in this protocol, splitting buys no reuse. 2. **Is there a plausible substitute for the step?** Split where real analyses vary. CLR is a good atomic protocol because it has substitutes: arcsin-square-root, log-transformation, or no transformation. -"Divide by the geometric mean"has no substitute inside CLR, so this does not motivate splitting this step +"Divide by the geometric mean" has no substitute inside CLR, so this does not motivate splitting this step into its own protocol. 3. **Does it have a name people use?** A named method is citable; a step inside one isn't. Usually why the naming question and the citation question give the same answer. @@ -102,7 +102,7 @@ wouldn't split it out. Make the call and say why in `## Notes`. A protocol can offer a choice when the alternatives come from the same source, take the same inputs, produce the same kind of output, and mean the same thing. Then it's a tuning knob, not a different protocol. -[independent-filtering-variance](agent-protocols/protocols/independent-filtering-variance) is the precedent: `filter: variance | mean`, both from the same paper, one +[independent-filtering-variance](protocols/independent-filtering-variance/protocol.md) is the precedent: `filter: variance | mean`, both from the same paper, one `method_origin_citation` covering both, and the protocol says exactly one must be chosen, that they must not be applied in sequence, and that the choice is recorded before anyone looks at results. @@ -118,7 +118,7 @@ concordant. Provide an explicit out-of-scope statement. This lets an agent compose protocols without wondering whether two of them overlap, and it stops a protocol from quietly growing into a pipeline when the protocol is often followed by other downstream analysis. -For example, [independent-filtering-variance](agent-protocols/protocols/independent-filtering-variance) ends by saying it doesn't define or perform hypothesis testing, p-value +For example, [independent-filtering-variance](protocols/independent-filtering-variance/protocol.md) ends by saying it doesn't define or perform hypothesis testing, p-value calculation, multiple-testing adjustment, differential expression, or interpretation of discoveries. ## Say what has to be recorded @@ -144,7 +144,7 @@ does run a named method, an empty field is a question to raise, not an answer. it. Composite: its steps name the constituent protocols and the order, without restating what those protocols already say. - It's a unit that can be reused in different analyses. -- `method_origin_citation` is the method's origin. `protocol_citation`, if present, describes this procedure. +- `protocol_citation` is required: who published these instructions. `method_origin_citation` is optional: who invented the method. - If a sibling protocol shares your `method_origin_citation`, `## Notes` says how yours differs. - An expert in the field could have a complete picture of the protocol without reading any code. - Every configurable parameter has a stated default. Required inputs (data, software, parameters) are stated in the Materials section. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 76d978e..862792a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -23,7 +23,7 @@ End the file with a `## History & Reviews` section as demonstrated in the templa ## Step 4: Validate and Submit Open a pull request. **You can rely on CI for validation.** The repository's automated testing infrastructure will check your syntax and tell you precisely what is wrong. Focus on getting the scientific correctness right. -*(If you prefer to run the validator locally before pushing, you can clone `waldronlab/agent-protocol-standard` and run `Rscript scripts/validate-protocol.R `)* +*(To run the validator locally before pushing, clone `waldronlab/agent-protocol-standard` beside this repository and, from here, run `Rscript ../agent-protocol-standard/scripts/validate-protocol.R protocols`. It takes the protocols directory, not a single file, and validates every protocol in it.)* --- diff --git a/protocols/humann4-chocophlan-build/protocol.md b/protocols/humann4-chocophlan-build/protocol.md index 9177e9c..f3dd699 100644 --- a/protocols/humann4-chocophlan-build/protocol.md +++ b/protocols/humann4-chocophlan-build/protocol.md @@ -22,7 +22,6 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" protocol_citation: "10.7554/eLife.65088" -method_origin_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/humann4-database-build/protocol.md b/protocols/humann4-database-build/protocol.md index 28b37a7..2b888ff 100644 --- a/protocols/humann4-database-build/protocol.md +++ b/protocols/humann4-database-build/protocol.md @@ -23,9 +23,6 @@ artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" protocol_citation: "10.7554/eLife.65088" -# The bioBakery paper proposes this database-construction pipeline as a method, so it belongs here even -# though the protocol is composite. The constituent protocols' own method_citations are inherited. -method_origin_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/humann4-translated-search-build/protocol.md b/protocols/humann4-translated-search-build/protocol.md index 7adf4a8..87f862c 100644 --- a/protocols/humann4-translated-search-build/protocol.md +++ b/protocols/humann4-translated-search-build/protocol.md @@ -22,7 +22,6 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" protocol_citation: "10.7554/eLife.65088" -method_origin_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/humann4-utility-mapping/protocol.md b/protocols/humann4-utility-mapping/protocol.md index 62d9ba8..0b54e0b 100644 --- a/protocols/humann4-utility-mapping/protocol.md +++ b/protocols/humann4-utility-mapping/protocol.md @@ -22,7 +22,6 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" protocol_citation: "10.7554/eLife.65088" -method_origin_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/independent-filtering-variance/protocol.md b/protocols/independent-filtering-variance/protocol.md index ef122e7..a89460d 100644 --- a/protocols/independent-filtering-variance/protocol.md +++ b/protocols/independent-filtering-variance/protocol.md @@ -18,7 +18,7 @@ type: "atomic" artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: "10.5281/zenodo.22731694" +protocol_citation: "10.1073/pnas.0914005107" license: "CC-BY-4.0" method_origin_citation: "10.1073/pnas.0914005107" protocols_used: [] From d41e24c0947ed1f0daba4975d11982505554ff94 Mon Sep 17 00:00:00 2001 From: Levi Waldron Date: Sun, 13 Sep 2026 12:14:45 -0400 Subject: [PATCH 4/5] Address review comments on the citation guidance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Maintainer: drop the leave-one-dataset-out example, and drop the "omit the field" instruction, which explained at length something the sentence above it already said. What remains is that a paper applying an established method is the commonest wrong answer, because it is usually the paper you read. Also from the review: the two protocol_citation examples were one YAML block with the key repeated, so copying it would have been rejected by a strict parser or silently taken the second value. Two blocks now. And "ie" is "i.e.". The lead-in called self-citation "the normal answer for a protocol written in this repository". After citing Bourgon on independent-filtering-variance that is false — every protocol here encodes a published procedure, and none is a first definition. It now says so. Co-Authored-By: Claude Opus 5 (1M context) --- AUTHORING.md | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/AUTHORING.md b/AUTHORING.md index 928e1e5..a71399c 100644 --- a/AUTHORING.md +++ b/AUTHORING.md @@ -39,13 +39,18 @@ If you wrote this protocol from a paper's methods section, that paper. If you wr did — so name this protocol's own DOI instead: `artifact_doi` if it has one, otherwise `collection_doi`. That is not a formality. Repeating `collection_doi` here is how a protocol says **"first definition, -published here"**, and it is the normal answer for a protocol written in this repository: +published here"** rather than leaving a gap a reader has to interpret. No protocol in this repository +does so today — every one of them encodes a published procedure — so expect the first case more often: + +Transcribed from a published analysis: ```yaml -# transcribed from a published analysis protocol_citation: "10.1038/s41591-019-0405-7" +``` -# written here; nobody published this procedure before +Written here, with no publication describing the procedure: + +```yaml protocol_citation: "10.5281/zenodo.22731694" # this repository's concept DOI ``` @@ -54,13 +59,11 @@ the DOI of a release that does not exist yet. ### Who invented the method? -The primary literature where the method was **first proposed** — not a paper that used it. Leave-one- -dataset-out cross-validation was first applied to microbiome data by Pasolli 2016, but proposed by -Riester 2014. Riester is the answer; citing Pasolli here is the mistake this field exists to prevent. +The primary literature where the method was **first proposed** — not a paper that used it. A paper that +applies an established method is the commonest wrong answer here, because it is usually the paper you +read. -**Omit the field if the protocol originates no method.** Documenting how to build a tool's reference -database is not a method somebody proposed. An omitted field is honest; one filled in to satisfy CI is -not, and nothing in the validator can tell the difference. +Omit the field if the protocol performs no method somebody proposed. A composite inherits its constituents' origins automatically. Give it one of its own only where the composition was itself published as a method. @@ -91,7 +94,7 @@ into its own protocol. 3. **Does it have a name people use?** A named method is citable; a step inside one isn't. Usually why the naming question and the citation question give the same answer. -Another way to think about it is: in a methods section, would you say "we did X", ie name the procedure? If so, it's a protocol. Or would you say, "we did X as part of Y". If so, X is likely a step in protocol Y. +Another way to think about it is: in a methods section, would you say "we did X", i.e. name the procedure? If so, it's a protocol. Or would you say, "we did X as part of Y". If so, X is likely a step in protocol Y. Some cases are arguable. Geometric mean _does_ have other uses, but for microbiome researchers I (Levi) can only think of its use inside CLR, and it wouldn't be much burden to redefine it elsewhere if needed, so I From a95090fa47f405d8c6f27be33f5ec0861119f666 Mon Sep 17 00:00:00 2001 From: Levi Waldron Date: Sun, 13 Sep 2026 12:22:53 -0400 Subject: [PATCH 5/5] Frame self-citation as the rare exception it is MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The guidance presented the two answers as a pair of equally likely cases. They are not: a protocol almost always encodes a procedure someone published, and that paper is the answer. Self-citation applies only where no publication describes the procedure, and is expected to be rare. The transcription case now leads, self-citation is marked as the exception, and the instruction is to reach for it only after looking for the paper and concluding there is none. No protocol in this repository needs it today — including independent-filtering-variance, which encodes what Bourgon 2010 describes. Co-Authored-By: Claude Opus 5 (1M context) --- AUTHORING.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/AUTHORING.md b/AUTHORING.md index a71399c..220ec09 100644 --- a/AUTHORING.md +++ b/AUTHORING.md @@ -35,25 +35,25 @@ Two of them are citations, and each answers one question. Answer them separately ### Who published these instructions? -If you wrote this protocol from a paper's methods section, that paper. If you wrote it yourself, nobody -did — so name this protocol's own DOI instead: `artifact_doi` if it has one, otherwise `collection_doi`. - -That is not a formality. Repeating `collection_doi` here is how a protocol says **"first definition, -published here"** rather than leaving a gap a reader has to interpret. No protocol in this repository -does so today — every one of them encodes a published procedure — so expect the first case more often: - -Transcribed from a published analysis: +Almost always: the paper whose methods section you wrote this protocol from. ```yaml protocol_citation: "10.1038/s41591-019-0405-7" ``` -Written here, with no publication describing the procedure: +**The exception, which should be rare.** If no publication describes the procedure — you are writing a +genuine first definition — name this protocol's own DOI instead: `artifact_doi` if it has one, otherwise +`collection_doi`. ```yaml protocol_citation: "10.5281/zenodo.22731694" # this repository's concept DOI ``` +Repeating `collection_doi` here is how a protocol states "published here" rather than leaving a gap a +reader has to interpret. Reach for it only after looking for the paper and concluding there is none: no +protocol in this repository needs it today, `independent-filtering-variance` included — it encodes what +Bourgon 2010 describes, so Bourgon is the answer. + Use the concept DOI, not a version DOI. A version DOI would name the exact bytes, but you cannot write the DOI of a release that does not exist yet.