HGNC robot template #113

joeflack4 · 2024-06-06T20:47:23Z

Addresses sub-tasks in:

HGNC robot template mondo-ingest#558

Update mondo_genes.csv to be a proper ROBOT template: mondo-omim-genes.robot.tsv

Changes

HGNC ROBOT template

Rename: mondo_genes.csv --> mondo-omim-genes.robot.tsv
Update: Change from CSV to TSV
Update: Set a ROBOT sub-header
Update: remove < > around URIs
Update: remove ?'s at start of col names
Update: insert source_code col, w/ values: MONDO:OMIM

General:

Add: run.sh: For running ODK. And updated README.md w/ docs about that.
Update: README.md: Put some less important stuff in

CC: @souzadevinicius Thought this would be a good one for you to review

joeflack4 · 2024-06-06T22:55:37Z

.github/workflows/buid_and_release.yml

Successful release

https://github.com/monarch-initiative/omim/releases/tag/2024-06-06

joeflack4 · 2024-06-06T23:02:50Z

.gitignore

@@ -34,4 +34,4 @@ omim.json
 mondo_exactmatch_omim.sssom.tsv
 mondo_exactmatch_omimps.sssom.tsv
 omim.owl
-mondo_genes.csv
+mondo_genes.robot.tsv


Rename: mondo_genes.csv --> mondo-omim-genes.robot.tsv

Throughout the code base.

README.md

makefile

joeflack4 · 2024-06-06T23:04:34Z

makefile

@@ -35,8 +35,18 @@ omim.owl: omim.ttl mondo_exactmatch_omim.sssom.owl mondo_exactmatch_omimps.sssom
 		query --update sparql/hgnc_links.ru \
 		convert -f ofn -o $@

-mondo_genes.csv: omim.owl
+mondo_genes.robot.tsv: omim.owl


Update: Output in TSV now instead of CSV

ROBOT automatically does this based on the file extension

Now renamed to mondo-omim-genes.robot.tsv

joeflack4 · 2024-06-06T23:04:49Z

makefile

+	# Insert the source_code column as the second to last column
+	awk 'BEGIN {FS=OFS="\t"} {if (NR==1) {$$(NF+1)=$$(NF); $$(NF-1)="?source_code";} else {$$(NF+1)=$$(NF); $$(NF-1)="MONDO:OMIM";}} 1' $@ > temp_file && mv temp_file $@
+	# Remove the first character of each field in the header
+	awk 'BEGIN {FS=OFS="\t"} NR==1 {for (i=1; i<=NF; i++) $$i=substr($$i, 2)} {print}' $@ > temp_file && mv temp_file $@


Update: Remove the first character, a question mark (?), from each field in the header. This is an artefact of the SPARQL query.

makefile

joeflack4 · 2024-06-06T23:06:39Z

makefile

+	awk 'BEGIN {FS=OFS="\t"} NR>1 {gsub(/^<|>$$/, "", $$1); gsub(/^<|>$$/, "", $$2); gsub(/^<|>$$/, "", $$5)} {print}' $@ > temp_file && mv temp_file $@
+	# Insert ROBOT subheader
+	robot_subheader="ID\tSC 'has material basis in germline mutation in' some %\t>A oboInOwl:source\t>A oboInOwl:source\t" && \
+	sed 1a"$$robot_subheader" $@ > temp_file && mv temp_file $@


Add: ROBOT subheader

run.sh

joeflack4 · 2024-06-06T23:23:45Z

makefile

+	awk 'BEGIN {FS=OFS="\t"} NR>1 {gsub(/^<|>$$/, "", $$1); gsub(/^<|>$$/, "", $$2); gsub(/^<|>$$/, "", $$5)} {print}' $@ > temp_file && mv temp_file $@
+	# Insert ROBOT subheader
+	robot_subheader="ID\tSC 'has material basis in germline mutation in' some %\t>A oboInOwl:source\t>A oboInOwl:source\t" && \
+	sed 1a"$$robot_subheader" $@ > temp_file && mv temp_file $@


hgnc_id : SC 'has material basis in germline mutation in' some %

What Nico wrote in the issue was a placeholder for the actual thing. I went and looked through some examples we had of this pattern SC '<PROPERTY>' some %, and also found the correct string representation 'has material basis in germline mutation in'. I'm basing it off of several different locations in mondo where I saw this: '%s and ''has material basis in germline mutation in'' some %s'

- Rename: mondo_genes.csv --> mondo_genes.robot.tsv - Update: Change from CSV to TSV - Update: Set a ROBOT sub-header - Update: remove < > around URIs - Update: remove ?'s at start of col names - Update: insert source_code col, w/ values: MONDO:OMIM General: - Add: run.sh: For running ODK. And updated README.md w/ docs about that. - Update: README.md: Put some less important stuff in <details>

matentzn

I am weary of the extreme use of awk, but as long as it is dockerized.. I would advice caution on this and focus on building mondolib

makefile

run.sh

twhetzel · 2024-06-11T21:22:32Z

I am also weary of the extreme use of awk and would prefer to find another option.

joeflack4 · 2024-06-11T21:44:25Z

makefile

@@ -35,8 +35,18 @@ omim.owl: omim.ttl mondo_exactmatch_omim.sssom.owl mondo_exactmatch_omimps.sssom
 		query --update sparql/hgnc_links.ru \
 		convert -f ofn -o $@

-mondo_genes.csv: omim.owl
+mondo_genes.robot.tsv: omim.owl


Rewrite implementation: awk --> pandas

@joeflack4 refactor

Nico:

I am weary of the extreme use of awk, but as long as it is dockerized.. I would advice caution on this and focus on building mondolib

Trish:

I am also weary of the extreme use of awk and would prefer to find another option.

Haha, this is funny, because I feel the same way. I thought for some reason you guys would probably prefer a ShellScript solution to pandas, but that was also when I thought I only needed to do 2 manipulations, but it turned out to be 4.

After I wrote that, I sent this to my friend who heavily uses awk and sed, who I've been trying to get to use pandas. Not sure if you guys are familiar with this meme, lol:

Meme

It should be an easy rewrite into pandas, so I'll do that.

Thanks, I appreciate your efforts here, but also think something more readable and more easily portable to a common solution in mondolib eventually will be helpful longer term :)

@twhetzel and/or @matentzn review

Done! Please take a look at the new Python file and refactored make goal.

I also added column sorting. Forgot to do that before, and it's not entirely unimportant.

I re-ran the goal and the output is the same as what I've attached to the release, the only difference being the sorting. I'll update that file shortly.

RE: mondolib refactor: I'm sure there's some kind of ROBOT-template-fu that we could move over there, but I'm not sure yet what that would be. I write a lot of code that looks similar, but the ROBOT templates and the modifications I do to create them vary quite a bit.

- Update: Refactor method to do this from ShellScript / awk to Python / pandas. - Update: Now sorts columns General - Update: .gitignore: Simplified ignores for files at root. - Add: Utility function to handle < > around URIs

omim2obo/mondo_omim_genes_robot_tsv.py

twhetzel

Fine other than the open question about the source column, which will be sorted Monday.

omim2obo/mondo_omim_genes_robot_tsv.py

- Delete: source_code column (w/ values: MONDO:OMIM) - Bug fix: No longer adding exact match gene annotations if >1 gene associated with MIM.

omim2obo/mondo_omim_genes_robot_tsv.py

joeflack4 marked this pull request as draft June 6, 2024 20:47

joeflack4 force-pushed the hgnc-template branch from 9e20043 to 23c3d59 Compare June 6, 2024 21:01

joeflack4 self-assigned this Jun 6, 2024

joeflack4 added enhancement New feature or request omim labels Jun 6, 2024

joeflack4 force-pushed the hgnc-template branch from 23c3d59 to 1cd0c31 Compare June 6, 2024 22:22

joeflack4 marked this pull request as ready for review June 6, 2024 22:22

joeflack4 force-pushed the hgnc-template branch from 1cd0c31 to 223f0bb Compare June 6, 2024 22:25

joeflack4 commented Jun 6, 2024

View reviewed changes

joeflack4 requested review from matentzn and twhetzel and removed request for matentzn June 6, 2024 23:08

joeflack4 mentioned this pull request Jun 6, 2024

HGNC robot template monarch-initiative/mondo-ingest#559

Merged

9 tasks

joeflack4 commented Jun 6, 2024

View reviewed changes

joeflack4 force-pushed the hgnc-template branch from 223f0bb to 47d744c Compare June 6, 2024 23:34

matentzn approved these changes Jun 11, 2024

View reviewed changes

makefile Outdated Show resolved Hide resolved

makefile Outdated Show resolved Hide resolved

run.sh Show resolved Hide resolved

joeflack4 mentioned this pull request Jun 11, 2024

Utility function: Remove brackets from <URI> strings monarch-initiative/mondolib#13

Open

joeflack4 commented Jun 11, 2024

View reviewed changes

joeflack4 force-pushed the hgnc-template branch 2 times, most recently from 5c18c12 to 264181a Compare June 11, 2024 22:47

joeflack4 added the hgnc label Jun 13, 2024

HGNC ROBOT template

cb615e8

- Update: Refactor method to do this from ShellScript / awk to Python / pandas. - Update: Now sorts columns General - Update: .gitignore: Simplified ignores for files at root. - Add: Utility function to handle < > around URIs

joeflack4 force-pushed the hgnc-template branch from 264181a to cb615e8 Compare June 14, 2024 21:02

twhetzel reviewed Jun 14, 2024

View reviewed changes

omim2obo/mondo_omim_genes_robot_tsv.py Outdated Show resolved Hide resolved

twhetzel reviewed Jun 14, 2024

View reviewed changes

omim2obo/mondo_omim_genes_robot_tsv.py Show resolved Hide resolved

twhetzel requested review from twhetzel and matentzn June 14, 2024 21:28

twhetzel reviewed Jun 14, 2024

View reviewed changes

joeflack4 mentioned this pull request Jun 14, 2024

Update Genes from OMIM (do not merge) monarch-initiative/mondo#7806

Draft

joeflack4 commented Jun 18, 2024

View reviewed changes

omim2obo/mondo_omim_genes_robot_tsv.py Show resolved Hide resolved

joeflack4 force-pushed the hgnc-template branch from 72bc9d8 to 6eaecdf Compare June 18, 2024 22:27

HGNC ROBOT template

6b3a48d

- Delete: source_code column (w/ values: MONDO:OMIM) - Bug fix: No longer adding exact match gene annotations if >1 gene associated with MIM.

joeflack4 force-pushed the hgnc-template branch from 6eaecdf to 6b3a48d Compare June 18, 2024 22:28

joeflack4 commented Jun 18, 2024

View reviewed changes

omim2obo/mondo_omim_genes_robot_tsv.py Show resolved Hide resolved

twhetzel approved these changes Jun 25, 2024

View reviewed changes

joeflack4 merged commit 89d2517 into main Jun 26, 2024

joeflack4 deleted the hgnc-template branch June 26, 2024 03:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HGNC robot template #113

HGNC robot template #113

joeflack4 commented Jun 6, 2024 •

edited

Loading

joeflack4 Jun 6, 2024

joeflack4 Jun 6, 2024 •

edited

Loading

joeflack4 Jun 6, 2024

joeflack4 Jun 14, 2024

joeflack4 Jun 6, 2024

joeflack4 Jun 6, 2024

joeflack4 Jun 6, 2024 •

edited

Loading

matentzn left a comment

twhetzel commented Jun 11, 2024

joeflack4 Jun 11, 2024 •

edited

Loading

twhetzel Jun 11, 2024

joeflack4 Jun 11, 2024 •

edited

Loading

twhetzel left a comment •

edited

Loading

HGNC robot template #113

HGNC robot template #113

Conversation

joeflack4 commented Jun 6, 2024 • edited Loading

Overview

Changes

joeflack4 Jun 6, 2024

Choose a reason for hiding this comment

Successful release

joeflack4 Jun 6, 2024 • edited Loading

Choose a reason for hiding this comment

Rename: mondo_genes.csv --> mondo-omim-genes.robot.tsv

joeflack4 Jun 6, 2024

Choose a reason for hiding this comment

joeflack4 Jun 14, 2024

Choose a reason for hiding this comment

joeflack4 Jun 6, 2024

Choose a reason for hiding this comment

joeflack4 Jun 6, 2024

Choose a reason for hiding this comment

joeflack4 Jun 6, 2024 • edited Loading

Choose a reason for hiding this comment

hgnc_id : SC 'has material basis in germline mutation in' some %

matentzn left a comment

Choose a reason for hiding this comment

twhetzel commented Jun 11, 2024

joeflack4 Jun 11, 2024 • edited Loading

Choose a reason for hiding this comment

Rewrite implementation: awk --> pandas

twhetzel Jun 11, 2024

Choose a reason for hiding this comment

joeflack4 Jun 11, 2024 • edited Loading

Choose a reason for hiding this comment

twhetzel left a comment • edited Loading

Choose a reason for hiding this comment

joeflack4 commented Jun 6, 2024 •

edited

Loading

joeflack4 Jun 6, 2024 •

edited

Loading

Rename: `mondo_genes.csv` --> `mondo-omim-genes.robot.tsv`

joeflack4 Jun 6, 2024 •

edited

Loading

`hgnc_id` : `SC 'has material basis in germline mutation in' some %`

joeflack4 Jun 11, 2024 •

edited

Loading

Rewrite implementation: `awk` --> `pandas`

joeflack4 Jun 11, 2024 •

edited

Loading

twhetzel left a comment •

edited

Loading