diff --git a/AUTHORING.md b/AUTHORING.md index 6d1ee2e..220ec09 100644 --- a/AUTHORING.md +++ b/AUTHORING.md @@ -1,5 +1,7 @@ # Writing a protocol +Your primary focus should be **clarity, absence of ambiguity, scientific correctness, and accurate citation**. CI will validate formatting, but cannot validate the science for you. + [`CONTRIBUTING.md`](CONTRIBUTING.md) covers mechanics: where files go, how to bump a version, how to record a review. [`PROTOCOL_STANDARD.md`](https://github.com/waldronlab/agent-protocol-standard/blob/main/PROTOCOL_STANDARD.md) defines the format. This file covers what neither can check for you — what goes in a protocol, and @@ -9,103 +11,105 @@ where one protocol ends and the next starts. Write what to do and why, precisely enough that two people, or two agents working in two languages, get the same answer. Not an R script with comments. If a step only makes sense once you've read an -implementation, it isn't done. - -That's harder than it sounds, and it's the point. Code hides decisions in library defaults: which -linkage, which tie-breaker, which denominator, what happens to a missing value. Prose drags each one -into the open where someone can argue with it. Most of the work is finding the decisions your -reference implementation was making silently. +implementation, more detail is needed. -Copy [`independent-filtering-variance`](protocols/independent-filtering-variance/protocol.md) for tone -and level of detail. +Note that code hides decisions such as default settings in library defaults, but such defaults can +change with package versions or between different implementations of a method. A protocol should specify +the procedure in enough detail not to be affected by such implementation differences. -## Cite the method's origin, not its users - -An atomic protocol carries exactly one `method_citation`: the primary literature where the method was -first published. Not the paper you took the analysis from. Not the paper that made it popular. The one -that proposed it. This is important to ensuring accuracy of citation of original literature. +See [`independent-filtering-variance`](protocols/independent-filtering-variance/protocol.md) for an +example of appropriate tone and level of detail. ## The four citation and DOI fields Each name says what it identifies. `*_citation` points at other work; `*_doi` identifies a thing. -| Field | Identifies | Points at | +Two of them are citations, and each answers one question. Answer them separately. + +| Field | The question | Required? | |---|---|---| -| `method_citation` | the **method** this protocol performs | the primary literature that proposed it | -| `protocol_citation` | a publication **describing or validating this protocol** | the procedure as written here, parameters included | -| `artifact_doi` | **this document**, as a citable artifact | a DOI minted for it, e.g. from protocols.io | -| `collection_doi` | the **repository or collection** housing it | a Zenodo record | +| `protocol_citation` | **Who published these instructions?** | Yes | +| `method_origin_citation` | **Who invented the method?** | No | +| `artifact_doi` | a DOI for **this document**, e.g. from protocols.io | No | +| `collection_doi` | a DOI for the **collection** housing it, e.g. a Zenodo record | No | + +### Who published these instructions? + +Almost always: the paper whose methods section you wrote this protocol from. + +```yaml +protocol_citation: "10.1038/s41591-019-0405-7" +``` -The first two get confused, because both can point at a paper containing the method. The difference: -`method_citation` is the method in general, `protocol_citation` is precise usage — the parameter values, -thresholds and choices this protocol fixes. +**The exception, which should be rare.** If no publication describes the procedure — you are writing a +genuine first definition — name this protocol's own DOI instead: `artifact_doi` if it has one, otherwise +`collection_doi`. -So one method can be the basis of several protocols. Random forest classification is one method with one -origin, but two published parameterizations are different procedures that give different answers from the -same data. Each is its own protocol. They share a `method_citation` and differ in `protocol_citation`. +```yaml +protocol_citation: "10.5281/zenodo.22731694" # this repository's concept DOI +``` -Worth remembering, because it's what stops the one-method-one-citation rule from forcing unlike procedures -into one document. If two candidate protocols run the same method with different parameters from different -sources, the rule isn't telling you to merge them. Siblings are fine. +Repeating `collection_doi` here is how a protocol states "published here" rather than leaving a gap a +reader has to interpret. Reach for it only after looking for the paper and concluding there is none: no +protocol in this repository needs it today, `independent-filtering-variance` included — it encodes what +Bourgon 2010 describes, so Bourgon is the answer. -If you're unsure which field a DOI belongs in, ask whether it would still be right if you rewrote the -steps. A method's origin survives a rewrite, so that's `method_citation`. A paper describing this -procedure doesn't, because it would no longer be the procedure described — `protocol_citation`. +Use the concept DOI, not a version DOI. A version DOI would name the exact bytes, but you cannot write +the DOI of a release that does not exist yet. -Example: "We employed random forest classification (`method_citation`) as implemented by Pasolli et al. 2016. (`protocol_citation`)." +### Who invented the method? -Most composites have no `method_citation` — they sequence existing protocols, propose nothing new, and -inherit their constituents' citations. A paper describing that kind of pipeline is a `protocol_citation`. -But a string of methods can be published as a method in its own right, and then `method_citation` is right: -a two-stage hierarchical meta-analysis is a method, and so is `humann4-database-build`. Ask whether the -paper proposes the combination as an approach people would cite, or just describes running the steps. +The primary literature where the method was **first proposed** — not a paper that used it. A paper that +applies an established method is the commonest wrong answer here, because it is usually the paper you +read. -Atomic protocols can have `method_citation` and nothing else. Use `protocol_citation` only when a -paper really does describe the procedure as you've written it, which is in some way distinguished from the general method. +Omit the field if the protocol performs no method somebody proposed. + +A composite inherits its constituents' origins automatically. Give it one of its own only where the +composition was itself published as a method. + +One method can be the basis of several protocols. Random forest classification has one origin and many +published parameterizations that give different answers from the same data. Each is its own protocol: +they share a `method_origin_citation` and differ in `protocol_citation`. ## If one paper didn't propose everything the protocol does, it's more than one protocol Examples from previous reviews of protocol proposals: -- **Microbe set enrichment.** Looked like one protocol with a method parameter. ORA, PADOG and CBEA have - three citations, three null hypotheses, and three different input types — a thresholded list, a labelled - abundance matrix, a compositional matrix. Three protocols, plus a composite that benchmarks them. -- **Prevalence filtering and CLR.** Drafted as one preprocessing protocol. Downstream analyses need to run - with the transformation and without it, so splitting made CLR an optional slot instead of a mandatory step. +- **Benchmarking of microbe set enrichment methods.** Looked like one protocol as described in the BugSigDB publication. +However it involves three different atomic enrichment protocols, and a composite benchmarking protocol. +- **Centered Log Ratio transformation (CLR) followed by filtering on microbial prevalence.** Seen multiple times in preprocessing workflows, but these are distinct protocols because they have different citations and can be used independently of each other. -The pattern: a bundled protocol may feel atomic when one paper happened to do all of it at once, but that's a fact -about the paper, not about the methods. +## When to stop splitting up steps into new protocols -## When to stop splitting +Splitting the steps of a protocol into more protocols can unnecessarily increase complexity. +Split a step of a protocol out into its own protocol only if **all three** are yes: -An atomic protocol is a composable unit that's useful as a whole in building analyses. Do not split further than necessary. Split only if all three are yes: - -1. **Would two different analyses use this on its own?** If it only ever shows up inside one bigger thing, +1. **Would different analyses use the step on their own?** If a step only ever shows up in this protocol, splitting buys no reuse. -2. **Is there a plausible substitute?** Split where real analyses vary. CLR has - substitutes — arcsin-square-root, or no transformation — so the transformation is a unit. "Divide by the - geometric mean" has no substitute inside CLR, so it isn't. +2. **Is there a plausible substitute for the step?** Split where real analyses vary. CLR is a good atomic +protocol because it has substitutes: arcsin-square-root, log-transformation, or no transformation. +"Divide by the geometric mean" has no substitute inside CLR, so this does not motivate splitting this step +into its own protocol. 3. **Does it have a name people use?** A named method is citable; a step inside one isn't. Usually why the naming question and the citation question give the same answer. -Another way to think about it is: in a methods section, would you say "we did X", ie name the procedure? If so, it's a protocol. Or would you say, "we did X as part of Y". If so, X is likely a step in protocol Y. - -For example, "Abundances were -CLR-transformed" is one clause and one protocol. Nobody writes "we divided by the geometric mean and took -logarithms", because naming CLR already covers it. CLR can be considered an atomic protocol with two steps. +Another way to think about it is: in a methods section, would you say "we did X", i.e. name the procedure? If so, it's a protocol. Or would you say, "we did X as part of Y". If so, X is likely a step in protocol Y. -Some cases are arguable. Make the call and say why in `## Notes`. +Some cases are arguable. Geometric mean _does_ have other uses, but for microbiome researchers I (Levi) +can only think of its use inside CLR, and it wouldn't be much burden to redefine it elsewhere if needed, so I +wouldn't split it out. Make the call and say why in `## Notes`. ## Alternatives inside one protocol, sometimes A protocol can offer a choice when the alternatives come from the same source, take the same inputs, -produce the same kind of output, and mean the same thing. Then it's a tuning knob, not a different method. +produce the same kind of output, and mean the same thing. Then it's a tuning knob, not a different protocol. -`independent-filtering-variance` is the precedent: `filter: variance | mean`, both from the same paper, one -`method_citation` covering both, and the protocol says exactly one must be chosen, that they must not be +[independent-filtering-variance](protocols/independent-filtering-variance/protocol.md) is the precedent: `filter: variance | mean`, both from the same paper, one +`method_origin_citation` covering both, and the protocol says exactly one must be chosen, that they must not be applied in sequence, and that the choice is recorded before anyone looks at results. -`method_citation` is the tell. If the alternatives need two citations, they're two protocols. +`method_origin_citation` is the tell. If the alternatives need two citations, they're two protocols. This is how protocols work elsewhere too. Nature Protocols and protocols.io allow branch points freely, but for procedural variants of one method, not for swapping in a different method by a different author. OECD @@ -114,10 +118,10 @@ concordant. ## Say what the protocol doesn't do -Provide an explicit out-of-scope statement. It lets an agent +Provide an explicit out-of-scope statement. This lets an agent compose protocols without wondering whether two of them overlap, and it stops a protocol from quietly growing into a pipeline when the protocol is often followed by other downstream analysis. -For example, `independent-filtering-variance` ends by saying it doesn't define or perform hypothesis testing, p-value +For example, [independent-filtering-variance](protocols/independent-filtering-variance/protocol.md) ends by saying it doesn't define or perform hypothesis testing, p-value calculation, multiple-testing adjustment, differential expression, or interpretation of discoveries. ## Say what has to be recorded @@ -130,39 +134,24 @@ Where a choice has to be made — a filter, a threshold, a candidate list — ma ## When you can't find a source -Rare, but some methods are old or folkloric enough to have no identifiable first publication. Say so in the -pull request instead of reaching for a convenient recent paper. +Rare, but some methods are old or folkloric enough to have no identifiable first publication. Say so instead of reaching for a convenient recent paper. -A missing `method_citation` validates today, so nothing stops you leaving it out. Descriptive protocols that +A missing `method_origin_citation` validates today, so nothing stops you leaving it out. Descriptive protocols that claim no method — a study table, a corpus summary — legitimately have nothing to cite. For a protocol that does run a named method, an empty field is a question to raise, not an answer. -Raise it in [`waldronlab/agent-protocol-standard`](https://github.com/waldronlab/agent-protocol-standard/issues) -— the standard may need a way to say "classical method, no primary source", and format decisions don't get -made in this repository. - ## Before you open the pull request Check: - For an atomic protocol claiming a method, that it is one method, and you can name the paper that proposed it. Composite: its steps name the constituent protocols and the order, without restating what those - protocols already say. Descriptive or reporting protocol: claims no method, and says so. -- It's a unit someone would use in more than one analysis, and there's a plausible substitute for it. -- `method_citation` is the method's origin. `protocol_citation`, if present, describes this procedure. -- If a sibling protocol shares your `method_citation`, `## Notes` says how yours differs. + protocols already say. +- It's a unit that can be reused in different analyses. +- `protocol_citation` is required: who published these instructions. `method_origin_citation` is optional: who invented the method. +- If a sibling protocol shares your `method_origin_citation`, `## Notes` says how yours differs. - An expert in the field could have a complete picture of the protocol without reading any code. -- Every *optional* parameter has a stated default and a reason. Required inputs — a dataset, a covariate, a - candidate list — are named as required, with no unstated default. +- Every configurable parameter has a stated default. Required inputs (data, software, parameters) are stated in the Materials section. - The out-of-scope section exists and is specific. -- The validator passes: - - ```sh - git clone https://github.com/waldronlab/agent-protocol-standard.git - Rscript agent-protocol-standard/scripts/validate-protocol.R protocols - ``` - - It checks conformance, not correctness. It can't tell you that you cited the wrong paper, bundled two - methods, or left a decision implicit. That's what review is for. -The format is pre-1.0 (`spec_version: 0.1.0`) and still moving. If a rule here gets in the way of describing -a method honestly, raise it as an issue rather than working around it. The standard is young enough to fix. +The protocol standard is still pre-release and experimental. If a rule here gets in the way of describing +a protocol unambiguously, raise it as an issue rather than working around it. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9096dbf..862792a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,61 +1,41 @@ # Contributing -## The format is defined elsewhere +Your primary concern as a contributor is **clarity, absence of ambiguity, scientific correctness, and accurate citation**. Our continuous integration (CI) tools will automatically validate your formatting syntax when you open a pull request, so focus on the science. -[`PROTOCOL_STANDARD.md`](https://github.com/waldronlab/agent-protocol-standard/blob/main/PROTOCOL_STANDARD.md) -in `waldronlab/agent-protocol-standard` is the authority on what a `protocol.md` must contain. This -repository holds content only; it does not define or extend the format. If you believe the format -itself needs to change, open an issue there rather than working around it here. - -## Deciding what to write - -This file covers mechanics. [`AUTHORING.md`](AUTHORING.md) covers the judgement calls the mechanics -cannot check: finding the paper that actually proposed a method, recognizing when what looks like one -protocol is really two, when alternatives may live inside a single protocol, and what to state as out -of scope. Read it before writing a new protocol. - -## Adding a protocol - -1. Create `protocols//protocol.md`. The directory name must exactly match the `name` field in - the YAML frontmatter. -2. Fill in the required frontmatter — `name`, `description`, `version`, `authors`, `date`, `status` — - and the provenance fields that apply. An atomic protocol carries exactly one `method_citation` naming the - primary literature where the method was published. -3. End the file with a `## History & Reviews` section. A first release has one version entry, a - `#### Changes` bullet list, and `*No reviews yet.*` under `#### Reviews`; omit the `reviews:` - frontmatter field entirely until someone has actually reviewed it. -4. Open a pull request. CI validates every protocol in the repository and will tell you precisely - what is wrong. +## Step 1: Claim an Issue +Browse the [Issues](https://github.com/waldronlab/agent-protocols/issues) tab and look for the `good-first-protocol` label. +To claim an issue, **comment on that issue stating you are beginning work**. Use the issue thread to post questions, comments, and other follow-up while you draft the protocol. -## Changing an existing protocol +## Step 2: Set up from Template +Do not write the complex YAML frontmatter from scratch. Instead, copy the official boilerplate from the standard repository: +[Protocol Template](https://github.com/waldronlab/agent-protocol-standard/tree/main/template) -Bump `version:` and `date:`, and add a matching entry at the **top** of `## History & Reviews` with -the same version and date. The validator enforces that the top entry agrees with the frontmatter, so -a version bump without a history entry fails. +1. Create a directory `protocols//`. The directory name must exactly match the `name` field in your YAML frontmatter. +2. Copy the template into your new directory as `protocol.md`. -## Recording a review +## Step 3: Write +Read [`AUTHORING.md`](AUTHORING.md) for philosophical guidelines on how to write your protocol. It covers the judgement calls that CI cannot check: finding the paper that actually proposed a method, recognizing when what looks like one protocol is really two, and what to state as out of scope. -A review is recorded twice, and the two must agree: a `**Review by **` block under the version -that was reviewed, and an entry in the frontmatter `reviews:` array whose `protocol_version` is that -version. The validator compares the date, status, and ORCID across both, so a human reader and a -machine reader can never draw different conclusions. +Fill in the required frontmatter (`name`, `description`, `version`, `authors`, `date`, `status`, `protocol_citation`) and write the human-readable instructions. Add `method_origin_citation` where the protocol performs a method someone proposed; see [AUTHORING.md](AUTHORING.md). -A review applies to the version it was given for. Do not move an existing review onto a new release. +End the file with a `## History & Reviews` section as demonstrated in the template. Omit the `reviews:` frontmatter field entirely until someone has actually reviewed it. -## Do not edit `PROTOCOLS.yaml` +## Step 4: Validate and Submit +Open a pull request. **You can rely on CI for validation.** The repository's automated testing infrastructure will check your syntax and tell you precisely what is wrong. Focus on getting the scientific correctness right. -It is generated by CI on every push to `main` that changes a protocol or the generator workflow -itself — the index is a function of both. Hand edits are overwritten by the next such push — and -until then, the index disagrees with the protocols it claims to describe, which is worse than -either. +*(To run the validator locally before pushing, clone `waldronlab/agent-protocol-standard` beside this repository and, from here, run `Rscript ../agent-protocol-standard/scripts/validate-protocol.R protocols`. It takes the protocols directory, not a single file, and validates every protocol in it.)* -## Running the validator locally +--- -Optional — CI runs it on every pull request — but faster to iterate against: +## Changing an existing protocol +Bump `version:` and `date:`, and add a matching entry at the **top** of `## History & Reviews` with the same version and date. The validator enforces that the top entry agrees with the frontmatter. -```sh -git clone https://github.com/waldronlab/agent-protocol-standard.git -Rscript agent-protocol-standard/scripts/validate-protocol.R protocols -``` +## Recording a review +A review is recorded twice, and the two must agree: a `**Review by **` block under the version that was reviewed, and an entry in the frontmatter `reviews:` array whose `protocol_version` is that version. The validator compares the date, status, and ORCID across both. A review applies to the version it was given for. Do not move an existing review onto a new release. + +## Do not edit `PROTOCOLS.yaml` +It is generated by CI on every push to `main` that changes a protocol or the generator workflow itself. Hand edits will be overwritten. -It needs the R packages `rmarkdown` and `yaml`. +## Reference: The Protocol Format +[`PROTOCOL_STANDARD.md`](https://github.com/waldronlab/agent-protocol-standard/blob/main/PROTOCOL_STANDARD.md) +in `waldronlab/agent-protocol-standard` is the definitive dictionary on what a `protocol.md` must contain. If you are unsure what a specific frontmatter field means, consult the standard. This repository holds content only; it does not define or extend the format. diff --git a/README.md b/README.md index 019e14b..91a8be2 100644 --- a/README.md +++ b/README.md @@ -18,9 +18,11 @@ never edit it by hand, and do not expect a hand-written list anywhere to be auth ## Contributing a protocol -Read [`PROTOCOL_STANDARD.md`](https://github.com/waldronlab/agent-protocol-standard/blob/main/PROTOCOL_STANDARD.md) -first; it is the authority on the format, and CI enforces it on every pull request. See -[CONTRIBUTING.md](CONTRIBUTING.md). +We welcome new protocols! Your primary concern as a contributor should be **clarity and scientific correctness**. Don't worry too much about formatting syntax—our continuous integration (CI) tools will automatically check that for you when you open a pull request. + +To get started: +1. **Find an idea:** Check our [Issues](https://github.com/waldronlab/agent-protocols/issues) tab and look for the `good-first-protocol` label to find a protocol that needs writing. +2. **Read the guide:** Follow the step-by-step instructions in [CONTRIBUTING.md](CONTRIBUTING.md) to claim an issue, set up your file from a template, and submit your work. ## Discovery diff --git a/protocols/humann4-augmented-clustering/protocol.md b/protocols/humann4-augmented-clustering/protocol.md index 2f5c836..101b9e9 100644 --- a/protocols/humann4-augmented-clustering/protocol.md +++ b/protocols/humann4-augmented-clustering/protocol.md @@ -21,9 +21,8 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - -method_citation: "10.1016/j.cell.2019.01.001" +protocol_citation: "10.7554/eLife.65088" +method_origin_citation: "10.1016/j.cell.2019.01.001" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/humann4-chocophlan-build/protocol.md b/protocols/humann4-chocophlan-build/protocol.md index 387900a..f3dd699 100644 --- a/protocols/humann4-chocophlan-build/protocol.md +++ b/protocols/humann4-chocophlan-build/protocol.md @@ -21,9 +21,7 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - -method_citation: "10.7554/eLife.65088" +protocol_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/humann4-database-build/protocol.md b/protocols/humann4-database-build/protocol.md index 0a576e2..2b888ff 100644 --- a/protocols/humann4-database-build/protocol.md +++ b/protocols/humann4-database-build/protocol.md @@ -22,11 +22,7 @@ type: composite artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - -# The bioBakery paper proposes this database-construction pipeline as a method, so it belongs here even -# though the protocol is composite. The constituent protocols' own method_citations are inherited. -method_citation: "10.7554/eLife.65088" +protocol_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/humann4-sgb-aggregation/protocol.md b/protocols/humann4-sgb-aggregation/protocol.md index d7d8244..76e713d 100644 --- a/protocols/humann4-sgb-aggregation/protocol.md +++ b/protocols/humann4-sgb-aggregation/protocol.md @@ -21,9 +21,8 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - -method_citation: "10.1016/j.cell.2019.01.001" +protocol_citation: "10.7554/eLife.65088" +method_origin_citation: "10.1016/j.cell.2019.01.001" upstream_repositories: - "https://github.com/biobakery/metaphlan" diff --git a/protocols/humann4-translated-search-build/protocol.md b/protocols/humann4-translated-search-build/protocol.md index 4c8c8fc..87f862c 100644 --- a/protocols/humann4-translated-search-build/protocol.md +++ b/protocols/humann4-translated-search-build/protocol.md @@ -21,9 +21,7 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - -method_citation: "10.7554/eLife.65088" +protocol_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/humann4-utility-mapping/protocol.md b/protocols/humann4-utility-mapping/protocol.md index 990c6d2..0b54e0b 100644 --- a/protocols/humann4-utility-mapping/protocol.md +++ b/protocols/humann4-utility-mapping/protocol.md @@ -21,9 +21,7 @@ type: atomic artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ - -method_citation: "10.7554/eLife.65088" +protocol_citation: "10.7554/eLife.65088" upstream_repositories: - "https://github.com/biobakery/humann" diff --git a/protocols/independent-filtering-variance/protocol.md b/protocols/independent-filtering-variance/protocol.md index 9880a3b..a89460d 100644 --- a/protocols/independent-filtering-variance/protocol.md +++ b/protocols/independent-filtering-variance/protocol.md @@ -18,9 +18,9 @@ type: "atomic" artifact_doi: ~ collection_doi: "10.5281/zenodo.22731694" -protocol_citation: ~ +protocol_citation: "10.1073/pnas.0914005107" license: "CC-BY-4.0" -method_citation: "10.1073/pnas.0914005107" +method_origin_citation: "10.1073/pnas.0914005107" protocols_used: [] category: "Statistical Analysis" tags: [filtering, variance, mean, high-throughput] @@ -86,7 +86,7 @@ This protocol covers only the filtering operation. Any downstream testing or mul - Rewrote the steps to be language-agnostic, removing the R implementation and describing the filtering operation in prose. - Added overall mean filtering as an alternative to overall variance filtering, matching the two strategies described in the source paper. - Clarified the `theta` cutoff and the use of `floor()` when determining how many features to remove. -- Removed `publication_doi`, which duplicated `citation`. *(Both fields were later renamed to `protocol_citation` and `method_citation`; this entry keeps the names in use at the time.)* +- Removed `publication_doi`, which duplicated `citation`. *(Both fields were later renamed to `protocol_citation` and `method_origin_citation`; this entry keeps the names in use at the time.)* #### Reviews