QA of CTD Ingest KGX

[mbrush]

QA Resources:

• KGX Summary report: https://docs.google.com/spreadsheets/d/1plJAeoZpfiUyUFo6aNTM95g0SLEFv1pASo5v9cgQHa8/edit?gid=618684675#gid=618684675
• original RIG: https://github.com/NCATSTranslator/translator-ingests/blob/main/src/translator_ingest/ingests/ctd/ctd_rig.yaml
• RIG updates PR: CTD RIG updates from QA process #233
• Phase 2 Ingest Survey (with some data examples): https://docs.google.com/spreadsheets/d/1R9z-vywupNrD_3ywuOt_sntcTrNlGmhiUWDXUdkPVpM/edit?

---

Issues to consider / address for January updated release:

• Is it right that there are so many associated_with edges with GeneOrGeneProduct as the subject. From the metaedges rollup sheet:

  • GeneOrGeneProduct associated_with AnatomicalEntity (15,000)
  • GeneOrGeneProduct associated_with BiologicalProcess (182,000)
  • GeneOrGeneProduct associated_with DiseaseOrPhenotypicFeature (86,000)
  • GeneOrGeneProduct associated_with MolecularActivity (18,000)
  • GeneOrGeneProduct associated_with Pathway (34,000)
  • . . . .
    Based on the RIG, the associated_with predicate is used only for the Chem-Disease file or Chem-GO/Pathway files - where chemical entities should be the subjects. I know some Chemicals may end up mapping/normalizing to genes (e.g. biological signaling molecules like insulin, interleukins, etc) - but the counts here seem too high to be explained by this.

• Is it right that there are so many correlated_with edges between Chemicals and Diseases/Phenotypes (65722) . . . these must come from the exposures file, right? And if I recall there were not that many records there where the study outcome reported an actual correlation? Same for Gene/Product correlations with Disease/Phenotype (1380)

• I feel like we should implement a filter on the Chem-Disease associations from the Chem-Diseases File. There are a ton of these (>3M) and most IMO are not significant or meaningful - being based only on a small handful of shared gene associations. I feel like this has the potential to introduce spurious / unreliable edges and inferences into our Results. The data indicates these are only statistical associations in KLAT, and gives p-values/scores - but the UI or and algorithms currently have no way of showing/using this info to convey the lower quality of these edges. Can we see what was done to filter these in Phase 2, and maybe try to replicate/modify this approach?

• Consider similar filters based on p-values for enrichment-based Chem - GO term / Pathway edges - which are also very abundant (>7M in total)

• Edge types that I don’t see specified in RIG - from the metaedges rollup sheet:

  • ChemicalEntity associated_with GeneOrGeneProduct (158)
  • Disease/Phenotype associated_with MolecularActivity (6)
  • Disease/phenotype associated_with Pathway (5)
    Low numbers on these, but thought I'd point them out. The issue behind this may have boader implications I am not appreciating.

• The RIG indicates that we create edges using the affects_sensistivity_to predicate (and its subpredicates) from the chem-gene-ixns file - but I don't see edges of this type in the summary report.

• RIG Issues:

  • @EvanDietzMorris please address my #comments in the RIG (starting with "MHB QUESTION:")
  • I updated EdgeType objects with a few new fields

---

Issues to consider longer term:

• In the metaedges sheet for CTD, there are 32 edges of the type 'ChemicalEntity', 'affects'/'causes', 'CellularComponent' (see row 4 here). When I look at examples of these in the ctd samples sheet here (row 660), the object is a GO term, and the association type is biolink:ChemicalEntityToBiologicalProcessAssociation' - suggesting the category should be Biological Process. I think the problematic edges come from the CTD_pheno_term_ixns.tsv file, which report manually curated phenotypes associated with chemicals - but the phenotypes are representing using a combination of a GO term and a direction. The problem arises in the small number of cases where the GO term is a cellular component, and the curator assigns a direction - leading to non-sensical statements like "Chem X affects/causes increase autophagosome".
  Possible solutions: We can either leave these as is and accept they are not quite right (there are only 32), or implement code that removes them, or code that removes the 'causes' qualified predicate and the object aspect direction from edges where the object is a cellular component - so the statement becomes "Chemical affects autophagosome" - which is fine (and maybe change the Association type on these as well).

• It feels strange to me that we use the same 'corerlated_with' predicate to represent tow very different types of associations in CTD: (1) Those based on a statistically significant over-representation of shared gene associations in the CTD data, and (2) those based on actual real-world studies that show chemical exposures correlated with incidence of Disease in a population of people. The latter is a direct, measured correlation in the real world. The former is a more indirect correlation based on shared annotations to genes in a data set. Importantly, there is a subtle different in their KL/AT (both are statistical_associations, but the latter has a manual_Agent and the former a data_analysis_pipeline). But IMO this is not sufficient to advertise the meaningful different in provenance and utility of these different types of correlations. We should consider additional ways to distinguish these, - perhaps distinct KL terms for these (real_world-statistical-association vs data-based-statistical-association). or perhaps different correlation predicates (e.g. directly_correlates_with, indirectly_correlates_with, or correlates_in_real_world_with and correlates_in_data_with).

---

Questions:

[DnlRKorn]

I double checked and based on the current ingest code, the sample in 660 is coming from CTD_chem_go_enriched.tsv.gz (GO:0005776 is annotated with Pathway before normalization and it's predicate has the category ChemicalEntityToBiologicalProcessAssociation, which only occurs in the following ingest https://github.com/NCATSTranslator/translator-ingests/blob/main/src/translator_ingest/ingests/ctd/ctd.py#L316-L337)

That file is described here: https://ctdbase.org/help/chemGODetailHelp.jsp -

CTD contains curated interactions between chemicals and genes/proteins. Many of these genes/proteins have Gene Ontology (GO) annotations that provide information about their associated biological processes, molecular functions, and cellular components. To provide insight into the biological properties that may be affected by a chemical, this report provides a list of GO terms that are statistically enriched among the genes/proteins that interact with the chemical or one of its descendants. GO terms are displayed in order of significance.

So CTD is definitely meaning to provide these relationships as fairly generic. We may want to change the chem_go_enriched ingest to be less specific than ChemicalEntityToBiologicalProcessAssociation

[DnlRKorn]

I just dug into the data further and confirmed Matt is right. Here are the lines from CTD_pheno_term_ixns.tsv this edge is pulled from

chemicalname	chemicalid	casrn	phenotypename	phenotypeid	comentionedterms	organism	organismid	interaction	interactionactions	anatomyterms	inferencegenesymbols	pubmedids
pectolinarigenin	C528671		autophagosome	GO:0005776	Homo sapiens	9606	pectolinarigenin results in increased autophagosome	increases^phenotype	1^Urinary Bladder^D001743|2^Cell Line, Tumor^D045744	MAP1LC3B^81631^GENE|SQSTM1^8878^GENE	37393350
pectolinarigenin	C528671		autophagosome	GO:0005776	pectolinarigenin^C528671^MESH	Homo sapiens	9606	[pectolinarigenin results in increased autophagy] which results in increased autophagosome	increases^phenotype	1^Urinary Bladder^D001743|2^Cell Line, Tumor^D045744	MAP1LC3B^81631^GENE|SQSTM1^8878^GENE	37393350

Which means the nodes file generated by the CTD ingest dropped the PhenotypicFeature tag for GO:0005776 when it wrote out the KGX. Is this also a bug or expected behavior?

[EvanDietzMorris]

Is it right that there are so many associated_with edges with GeneOrGeneProduct as the subject?

GeneOrGeneProduct is a mixin that gets applied to a number of different node categories. So the granularity of the summary is a bit confusing.

These are the leaf biolink types for subjects on associated_with edges.. it looks like this diversity is really due to the normalization category assignments for MESH identifiers, frequently normalized to UMLS. None of them get assigned Gene though.

Individual categories:

• biolink:ChemicalEntity
• biolink:ChemicalMixture
• biolink:ComplexMolecularMixture
• biolink:Disease
• biolink:Drug
• biolink:MolecularMixture
• biolink:Protein
• biolink:SmallMolecule

Conflated categories:

• biolink:SmallMolecule + biolink:MolecularMixture

Is it right that there are so many correlated_with edges between Chemicals and Diseases/Phenotypes?

All of the "correlated_with" edges come from the CTD_chemicals_diseases file rows designated "marker/mechanism" (whereas the exposure events file produces "positively_correlated_with" and "negatively_correlated_with"). That's a giant file with a bunch of MESH ids as subjects so it makes sense to me. I think again there is some confusion produced by the types in the summary, these are the leaf types for subjects on correlated_with edges (no Genes).

Individual:

• biolink:ChemicalEntity
• biolink:ComplexMolecularMixture
• biolink:Drug
• biolink:MolecularMixture
• biolink:Protein
• biolink:SmallMolecule

Conflated categories:

• biolink:SmallMolecule + biolink:Drug
• biolink:SmallMolecule + biolink:MolecularMixture

we should implement a filter on the Chem-Disease associations from the Chem-Diseases File & similar filters based on p-values for enrichment-based Chem - GO term / Pathway edges

I would probably agree with this. In phase 2 robokop never ingested rows from the chem-disease file that didn't have marker/mechanism or therapeutic, and we compared the publication counts for relationships with both kinds of rows and only created one of them.

There may a mix up regarding statistical association / p values - the Chem-Disease file only produces "knowledge assertion" KL edges. But for those files with p values we don't currently have a filter for those either.

Edge types that I don’t see specified in RIG

I think these are all explained by normalization of MESH or GO ids.

edges using the affects_sensitivity_to

I'm not sure how I missed these but looking at the mappings we apply when parsing that file I don't know where they would come from. Right now all edges from chem_gene_ixns use affects with a variety of qualifiers. "sensitivity" doesn't seem to be referred to in the file. So we'll have to go back and see how these edges were derived in phase 2. I'll look back at our ingest survey and see if there's anything there about it.

[EvanDietzMorris]

Regarding Dan's last comment, GO:0005776 is certainly a weird example, I see in the ingest output nodes file that GO:0005776 is first designated a Pathway node - because it occurs (many times) in the chem_go_enriched file. Because koza is currently configured to just throw out duplicate node writes, and we process the chem_go file first, it doesn't get assigned PhenotypicFeature or anything else at that stage.

Really though, that's not very important, after normalization it looks like:
{"id": "GO:0005776", "category": ["biolink:CellularComponent", "biolink:AnatomicalEntity", "biolink:PhysicalEssence", "biolink:OrganismalEntity", "biolink:SubjectOfInvestigation", "biolink:BiologicalEntity", "biolink:ThingWithTaxon", "biolink:NamedThing", "biolink:PhysicalEssenceOrOccurrent"], "name": "autophagosome", "equivalent_identifiers": ["GO:0005776"], "information_content": 76.9}

So in the end I think Matt is right, it's a weird assertion if we call all of those relationships causes. Unfortunately it's hard to filter at the ingest level because we don't know which terms are CellularComponents until after normalization.

[EvanDietzMorris]

I believe I have addressed most of the questions here and edited the RIG accordingly, except for auditing the node_type_info section. I'll try to look at that early next week.

In short, we are not currently filtering the Chem-Disease or Chem - GO term / Pathway files but we might want to, and all of the seemingly unexpected node/edge type outputs can be attributed to normalization of identifier namespaces that contain a variety of things - mostly MESH and GO.