Extract asset anatomy from HERD ontology links in NWB files #1979

Summary

Follow-up to #1806. NWB files can carry a HERD table (/general/external_resources) that links values in the file to ontology terms. When a HERD row links a value to an anatomy term (UBERON, Allen MBA, ...), dandi should use that term for the asset's anatomy. The author asserted these links, so they should take precedence over the string matching added in #1807 (and extended in #1825).

In #1806 we agreed to put a BioSample with .anatomy into each asset's wasDerivedFrom, then aggregate across assets in dandischema. This issue feeds that same slot from HERD.

Example: DANDI:001888

https://dandiarchive.org/dandiset/001888/draft holds rat EEG/EMG, 23 files. Each file has a HERD table. For sub-GRIN2B-129/sub-GRIN2B-129_ses-2021-03-26_ecephys.nwb:

The same table also links Subject.species, Subject.strain and Subject.genotype (NCBITaxon:10116, RRID:RGD_2308852, RRID:RGD_14394515). This issue covers anatomy only, but the HERD reader could also cover strain RRIDs, which #1724 proposed getting from HERD and which aren't extracted yet (#1716 added the strain text only).

The current asset metadata for these files (uploaded with dandi-cli 0.76.8) has no wasDerivedFrom, so no anatomy. String matching would not help here: the subject is a rat, which #1807 skips, and lab abbreviations like S1_Tr and M2_Fra would not match UBERON labels or synonyms even with #1825. HERD already holds the answer.

Proposed behavior

1. In pynwb_utils, if the file has nwb.external_resources, collect every HERD entity whose ID uses an anatomy ontology prefix (UBERON:, MBA:, and other atlas prefixes we recognize), whatever object or field it is linked from.
2. Skip entities linked from Subject objects. UBERON also contains life-stage terms (e.g. UBERON:0000113 "post-juvenile adult stage") that a converter could link from Subject.age. Species, strain and genotype are handled separately anyway.
3. Turn each remaining entity into models.Anatomy(identifier=<entity_uri>, name=<label>) and dedupe by identifier.
4. Put the result in the same place feat: extract brain area anatomy from NWB location fields #1807 does: wasDerivedFrom[BioSample].anatomy, on the deepest real BioSample, or on the synthetic brain-region-sample when there is none.
5. Precedence: if HERD gives anatomy for a file, use it and skip string matching. String matching stays as the fallback for files without HERD.
6. Apply to all species. HERD needs no species check.

Why filter by ontology rather than by source field: anatomy is often linked from fields outside the standard location attributes and columns, such as a brain_area column on the units table, a custom region column, or an injection site in an extension type. A list of allowed fields would miss these and keep growing.

Open questions

• Labels. HERD stores entity_id and entity_uri but no label, and Anatomy.name is required and, since Make name required and non-empty for Contributor and BaseType dandi-schema#444, must be non-empty. Options: look up the label in the UBERON table bundled by feat: add UBERON brain anatomy matching for all species #1825 (nervous system only, so it would miss trapezius muscle); query OLS at upload time; or fall back to the HERD key (S1_Tr), which is a poor display name. My suggestion: bundled table first, then the key.
• Non-brain anatomy. EMG → trapezius muscle is a real recording site but not a brain area. I think it belongs in anatomy, since the field is anatomy rather than brain area, but feat: extract brain area anatomy from NWB location fields #1807 is framed around brain areas.
• Redundant ancestors. cortex (group level) is an ancestor of every electrode-level term. Keep both, or drop terms that have a more specific descendant in the same file?
• Body parts that aren't recording sites. A stimulus table could link a column like "whisker touched" to UBERON. Under the rule above it would count as anatomy. This seems rare and arguably fits "anatomical part studied", so I'd accept it for now and add a skip list only if it turns up in practice.

Dandiset-level summary (out of scope here)

AssetsSummary has no anatomy field, and the archive computes it with dandischema's aggregate_assets_summary, so dandi-cli cannot fill it. Showing anatomy at the Dandiset level needs a dandi-schema change: add anatomy: Optional[List[Anatomy]] to AssetsSummary and collect it from wasDerivedFrom BioSamples in _add_asset_to_stats. Filed as dandi/dandi-schema#446. The Dandiset about field is curated by users, so it shouldn't be filled automatically.