Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions docs/configuration/advanced-example.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,11 @@ template:
# Statistical metadata as edge annotations (method: value = constant,
# method: column = per-row)
annotations:
- {annotation: sample_size, method: value, encoding: 9}
- {annotation: supporting_study_size, method: value, encoding: 9}
- {annotation: p_value, method: column, encoding: C}
- {annotation: multiple_testing_correction_method, method: value, encoding: Benjamini Hochberg}
- {annotation: relationship_strength, method: column, encoding: B} # Spearman rho
- {annotation: effect_size, method: column, encoding: B} # Spearman rho value
- {annotation: effect_type, method: value, encoding: spearmans_rho}
- {annotation: assertion_method, method: value, encoding: Spearman correlation}

# Freetext catch-all for context that doesn't map to a structured annotation
Expand Down Expand Up @@ -97,7 +98,8 @@ whole table or `method: column` to pull per-row notes from the source (see
{"id":"CHEBI:41774","name":"13C-tamoxifen","category":["biolink:ChemicalEntity"]}
```

**Edges:** Allow-listed annotation columns (`sample_size`, `p_value`, `relationship_strength`) stay as
**Edges:** Allow-listed annotation columns (`supporting_study_size`, `p_value`, `effect_size`,
`effect_type`) stay as
top-level edge fields (numeric annotations as controlled-notation strings). Any non-Biolink-slot name —
here `assertion_method`, `multiple_testing_correction_method`, `miscellaneous_notes` — folds into the
edge's `supporting_text` list as `"name: value"` entries (sorted alphabetically), alongside the built-in
Expand All @@ -109,9 +111,10 @@ edge's `supporting_text` list as `"name: value"` entries (sorted alphabetically)
"subject": "NCBITaxon:47715",
"predicate": "biolink:correlated_with",
"object": "CHEBI:41774",
"sample_size": "9.000",
"supporting_study_size": "9",
"p_value": "1.0000e-03",
"relationship_strength": "0.8500",
"effect_size": "0.85",
"effect_type": "spearmans_rho",
"supporting_text": [
"assertion_method: Spearman correlation",
"extracted_from_row_number: 3",
Expand Down Expand Up @@ -198,7 +201,7 @@ template:

annotations:
- {annotation: p_value, method: column, encoding: E}
- {annotation: relationship_strength, method: column, encoding: C}
- {annotation: effect_size, method: column, encoding: C}
```

Each column-mapped node gets its own `prioritize` list to guide disambiguation. `remove` strips each
Expand Down Expand Up @@ -252,7 +255,7 @@ sections:
source:
row_slice: [2, auto]
annotations:
- {annotation: relationship_strength, method: column, encoding: B}
- {annotation: effect_size, method: column, encoding: B}

- statement:
object:
Expand All @@ -261,7 +264,7 @@ sections:
source:
row_slice: [2, auto]
annotations:
- {annotation: relationship_strength, method: column, encoding: C}
- {annotation: effect_size, method: column, encoding: C}

# ... (one section per metabolite column; pattern repeats)
```
Expand Down
8 changes: 4 additions & 4 deletions docs/configuration/table.md
Original file line number Diff line number Diff line change
Expand Up @@ -410,14 +410,14 @@ Optional edge attributes (statistical metadata, notes, etc.).

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `annotation` | String | Yes | Attribute name (e.g., `"p_value"`, `"sample_size"`). Lowercased and trimmed of leading/trailing whitespace at parse time; underscores are preserved (use snake_case). |
| `annotation` | String | Yes | Attribute name (e.g., `"p_value"`, `"effect_size"`). Lowercased and trimmed of leading/trailing whitespace at parse time; underscores are preserved (use snake_case). |
| (inherits Encoding) | | | All Encoding fields available (method, encoding, regex, etc.) |

**Example:**
```yaml
annotations:
- {annotation: p_value, method: column, encoding: C} # Read from column C
- {annotation: sample_size, method: value, encoding: 450} # Literal value for all edges
- {annotation: p_value, method: column, encoding: C} # Read from column C
- {annotation: supporting_study_size, method: value, encoding: 450} # Literal value for all edges
- {annotation: multiple_testing_correction_method, method: value, encoding: "Benjamini Hochberg"}

# Descriptive name of your choice — folded into `supporting_text` on output.
Expand All @@ -430,7 +430,7 @@ annotations:

Annotation names fall into two groups at build time:

- **Biolink-native slots** — names matching a [Biolink Association](https://biolink.github.io/biolink-model/) slot (e.g. `p_value`, `sample_size`, `knowledge_level`, `primary_knowledge_source`, `adjusted_p_value`, `supporting_text`, `publications`, the qualifier slots like `severity_qualifier` / `disease_context_qualifier`) are written to edges verbatim.
- **Allowed edge fields** — names on the edge allow-list: [Biolink Association](https://biolink.github.io/biolink-model/) slots, qualifier slots, and curated KGX/Tablassert edge fields (e.g. `p_value`, `adjusted_p_value`, `knowledge_level`, `primary_knowledge_source`, `supporting_text`, `publications`, `supporting_study_size`, `effect_size`, `effect_type`, qualifier slots like `severity_qualifier` / `disease_context_qualifier`) are written to edges verbatim.
- **Tablassert pipeline fields** — `upstream_resource_ids`, `source_record_urls`.

Any other annotation name is treated as **supporting context**. At the end of `compile_graph`, tablassert sweeps the edge columns: for each non-allow-listed name it emits `"name: value"` entries into the edge's `supporting_text` (a `list[str]`), then drops the original column. Behavior worth knowing:
Expand Down
7 changes: 5 additions & 2 deletions docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,9 +127,12 @@ template:
- annotation: p_value
method: column
encoding: C
- annotation: relationship_strength
- annotation: effect_size
method: column
encoding: B
- annotation: effect_type
method: value
encoding: spearmans_rho
- annotation: assertion_method
method: value
encoding: "Spearman correlation"
Expand Down Expand Up @@ -241,7 +244,7 @@ template:
- annotation: p_value
method: column
encoding: C
- annotation: sample_size
- annotation: supporting_study_size
method: column
encoding: D
```
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/tutorial-table.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,6 @@ template:
- annotation: p_value
method: column
encoding: C
- annotation: sample_size
- annotation: supporting_study_size
method: column
encoding: D
6 changes: 3 additions & 3 deletions docs/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ template:
- annotation: p_value
method: column
encoding: C
- annotation: sample_size
- annotation: supporting_study_size
method: column
encoding: D
```
Expand Down Expand Up @@ -154,8 +154,8 @@ head -n 2 TUTORIAL_KG_1.0.0.edges.ndjson

Example output (numeric annotation columns are emitted as controlled-notation strings — p-values in scientific notation):
```json
{"id":"2cfea591-0f8f-33af-a7df-03da531d3359","subject":"HGNC:11998","predicate":"biolink:associated_with","object":"MONDO:0008903","p_value":"1.0000e-03","sample_size":"450"}
{"id":"7b1c9d02-5e8a-4f3b-9c1d-2a6e8f0b4d7c","subject":"HGNC:1100","predicate":"biolink:associated_with","object":"MONDO:0005041","p_value":"1.0000e-04","sample_size":"1200"}
{"id":"2cfea591-0f8f-33af-a7df-03da531d3359","subject":"HGNC:11998","predicate":"biolink:associated_with","object":"MONDO:0008903","p_value":"1.0000e-03","supporting_study_size":"450"}
{"id":"7b1c9d02-5e8a-4f3b-9c1d-2a6e8f0b4d7c","subject":"HGNC:1100","predicate":"biolink:associated_with","object":"MONDO:0005041","p_value":"1.0000e-04","supporting_study_size":"1200"}
```

**RIG file:**
Expand Down
4 changes: 3 additions & 1 deletion examples/agent/optimized_instructions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,9 @@ instructions: "# ROLE + TASK\nYou are an expert knowledge-graph (KG) engineer. Y
\ name, or its caption explicitly establishes that entity for the rows. A hard-coded object applied to every\n row must\
\ be justified by the table's actual context.\n3. CAPTURE STATISTICAL ANNOTATIONS: when the table has p_value, q_value,\
\ fold_change, z_score, lfsr, beta,\n standard_error, sample_size, or similar columns, add them as annotations (annotation:\
\ p_value / q_value /\n relationship_strength / sample_size, method: column, encoding: <letter>). Do NOT silently drop\
\ p_value / q_value /\n effect_size / supporting_study_size, method: column, encoding: <letter>). For effect_type, use method:\
\ column when the table provides it, else method: value with a fixed valid\n Biolink effect type (e.g. spearmans_rho) — emit\
\ effect_type ONLY alongside an effect_size annotation. Do NOT silently drop\
\ statistical\n columns — they are part of the evidence.\n4. PICK THE RIGHT OBJECT COLUMN: the object column must actually\
\ contain the intended entity. Verify with\n read_table that the column holds the entity type you claim (e.g. a protein-abundance\
\ table with UniProt\n IDs in columns A/B should map those, not a gene-symbol column elsewhere).\n5. prioritize GUIDANCE\
Expand Down
13 changes: 9 additions & 4 deletions src/tablassert/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -1926,9 +1926,12 @@ def call(prompt: str) -> str:
mappable table/worksheet; each section supplies its OWN `source` (the table's local path + that
file's source.url, plus sheet/row_slice/delimiter as needed) and its OWN `statement`. Within each
section choose column-letter encodings for entity columns and literal CURIEs for fixed values;
pick a valid biolink predicate; add statistical annotations (p_value / sample_size /
relationship_strength) when that table has those columns. A single-table article is still ONE
config with ONE section.
pick a valid biolink predicate; add statistical annotations (p_value / supporting_study_size /
effect_size / effect_type) when that table has them — method: column for table-provided columns,
method: value for a fixed valid value (e.g. effect_type: spearmans_rho when every row is a
Spearman correlation). Emit effect_type ONLY alongside an effect_size annotation: the pipeline
nulls an effect_type without a numeric effect_size. A single-table article is still ONE config
with ONE section.

## ReAct workflow + planning
Reason in an explicit ReAct loop (Thought -> Action -> Observation) and re-plan every few steps:
Expand Down Expand Up @@ -1972,7 +1975,9 @@ def call(prompt: str) -> str:
provenance: {repo: PMID, publication: "12345678"}
annotations:
- {annotation: p_value, method: column, encoding: C}
- {annotation: sample_size, method: column, encoding: D}
- {annotation: supporting_study_size, method: column, encoding: D}
- {annotation: effect_size, method: column, encoding: E}
- {annotation: effect_type, method: value, encoding: odds_ratio}

# (b) ALAMV6 — an excel organism~chemical correlation table (fixed chemical object)
source: {kind: excel, local: ./ALAM.XLSX, sheet: "all correlations", row_slice: [2, auto]}
Expand Down
62 changes: 60 additions & 2 deletions src/tablassert/biolink.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@
- ``ALLOWED_EDGE_FIELDS``: the Biolink ``Association`` model fields (walked over
the MRO) unioned with the qualifier slot names and a curated set of KGX /
Tablassert edge columns that are not Biolink Association fields.
- ``EffectTypes``: the 25 permissible ``effect_type`` values from Biolink PR
#1774 (merged), defined locally because the pinned ``biolink-model`` release
predates the PR; switch to the model's enum once it ships.

The enums are built dynamically at runtime from the model. For static type
checking, ``TYPE_CHECKING`` stub classes (declaring only the members referenced
Expand All @@ -50,7 +53,18 @@
if TYPE_CHECKING:
from linkml_runtime.utils.schemaview import SchemaView

__all__ = ["ALLOWED_EDGE_FIELDS", "BIOLINK_VERSION", "AgentTypes", "Categories", "EdgeCategories", "KnowledgeLevels", "Predicates", "Qualifiers"]
__all__ = [
"ALLOWED_EDGE_FIELDS",
"BIOLINK_VERSION",
"EFFECT_TYPE_VALUES",
"AgentTypes",
"Categories",
"EdgeCategories",
"EffectTypes",
"KnowledgeLevels",
"Predicates",
"Qualifiers",
]


def _screaming_snake(camel: str) -> str:
Expand Down Expand Up @@ -217,6 +231,39 @@ def _biolink_enum_values(enum_cls: type[Enum]) -> list[str]:
return sorted(str(member.value) for member in enum_cls)


# Permissible ``effect_type`` values, verbatim from the ``EffectTypeEnum`` permitted
# values of Biolink PR #1774 (merged, but not in the pinned biolink-model 4.4.3
# release; close_mappings intentionally ignored). Kept as a plain tuple so
# ``coerce`` can consume it without touching the enum.
EFFECT_TYPE_VALUES: tuple[str, ...] = (
"regression_coefficient",
"log2_fold_change",
"wald_ratio",
"inverse_variance_weighted",
"mr_egger",
"weighted_median",
"standardized_mean_difference",
"cohens_d",
"hedges_g",
"glasss_delta",
"strictly_standardized_mean_difference",
"correlation_coefficient",
"pearsons_r",
"spearmans_rho",
"kendalls_tau",
"polychoric_correlation",
"matthews_correlation_coefficient",
"goodman_kruskal_gamma",
"r2_linkage_disequilibrium",
"odds_ratio",
"relative_risk",
"hazard_ratio",
"eta_squared",
"omega_squared",
"root_mean_square_standardized_effect",
)


# Edge columns Tablassert / KGX emit that are neither Biolink ``Association`` model
# fields nor qualifier slot names: synonym carryover from NamedThing, KGX provenance
# and denormalized fields, supporting-study evidence slots, and Tablassert pipeline
Expand All @@ -227,6 +274,11 @@ def _biolink_enum_values(enum_cls: type[Enum]) -> list[str]:
TABLASERT_EDGE_EXTRAS: frozenset[str] = frozenset(
[
"broad_synonym",
# PR #1774 edge attributes; absent from biolink-model 4.4.3 Association.model_fields,
# so the union keeps them out of fold_unknown_to_supporting_text and they reach the
# final edges. Harmless once a future biolink-model ships them as real fields.
"effect_size",
"effect_type",
"equivalent_identifiers",
"evidence_direction",
"evidence_type",
Expand All @@ -237,7 +289,6 @@ def _biolink_enum_values(enum_cls: type[Enum]) -> list[str]:
"provided_by",
"related_synonym",
"relation",
"relationship_strength",
"source_record_urls",
"statistical_significance_qualifier",
"supporting_documents",
Expand Down Expand Up @@ -287,13 +338,20 @@ class AgentTypes(str, Enum):
DATA_ANALYSIS_PIPELINE: AgentTypes
MANUAL_AGENT: AgentTypes

class EffectTypes(str, Enum):
ODDS_RATIO: EffectTypes
SPEARMANS_RHO: EffectTypes

else:
Categories = _build_str_enum("Categories", _entity_category_names())
EdgeCategories = _build_str_enum("EdgeCategories", _association_names())
Predicates = _build_str_enum("Predicates", _predicate_values())
Qualifiers = _build_str_enum("Qualifiers", _qualifier_values())
KnowledgeLevels = _build_str_enum("KnowledgeLevels", _biolink_enum_values(cast("type[Enum]", _bm.KnowledgeLevelEnum)))
AgentTypes = _build_str_enum("AgentTypes", _biolink_enum_values(cast("type[Enum]", _bm.AgentTypeEnum)))
# Defined locally until biolink-model ships PR #1774, then switch to
# _biolink_enum_values(_bm.EffectTypeEnum).
EffectTypes = _build_str_enum("EffectTypes", list(EFFECT_TYPE_VALUES))


_schema_definition: Any = _schema().schema
Expand Down
Loading
Loading