Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DI-809 refactor #9

Merged
merged 65 commits into from
Sep 23, 2024
Merged
Show file tree
Hide file tree
Changes from 55 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
f9687f0
update dxapp.json
jethror1 Sep 13, 2024
5ce2a82
remove bedtools tar
jethror1 Sep 13, 2024
bc1b432
remove shell script
jethror1 Sep 13, 2024
bc38d05
move python app code
jethror1 Sep 13, 2024
191fd68
add .gitignore
jethror1 Sep 13, 2024
9f2e35a
add functions to get variant counts and calculate ratios
jethror1 Sep 13, 2024
fb1998d
formatting
jethror1 Sep 13, 2024
db7ec5c
more comments
jethror1 Sep 13, 2024
3d9f4ec
remove comment
jethror1 Sep 13, 2024
c437920
update readme
jethror1 Sep 16, 2024
d211dc3
fix formatting
jethror1 Sep 16, 2024
23aa426
update readme
jethror1 Sep 16, 2024
b72ee3f
add uploading of output file
jethror1 Sep 16, 2024
fb91092
add bedtools dependency asset
jethror1 Sep 16, 2024
18fa0ce
add comments
jethror1 Sep 16, 2024
3d8e20e
remove default file for bed file
jethror1 Sep 16, 2024
f9ba5a5
add developers and authorizedUsers to dxapp.json
jethror1 Sep 16, 2024
ef462c6
add version to dxapp.json
jethror1 Sep 16, 2024
7147850
update entrypoint in dxapp.json
jethror1 Sep 16, 2024
7e921f0
add ubuntu version to dxapp.json
jethror1 Sep 16, 2024
97d8b84
add rounding
jethror1 Sep 19, 2024
24800a1
update interpreter in dxapp.json
jethror1 Sep 19, 2024
d6423ca
actually download the input files
jethror1 Sep 19, 2024
7eca8fb
update .gitignore
jethror1 Sep 19, 2024
d9cec7b
add pysam wheel
jethror1 Sep 19, 2024
10b991c
add requirements.txt
jethror1 Sep 19, 2024
354956b
add test setup
jethror1 Sep 19, 2024
5950946
skip ref hom sites
jethror1 Sep 19, 2024
5221ac4
bonus upload print
jethror1 Sep 19, 2024
8bf1c93
update pytest workflow
jethror1 Sep 19, 2024
e1ed568
start tests
jethror1 Sep 19, 2024
769bcf7
update .gitignore
jethror1 Sep 19, 2024
b476eb9
update bedtools install
jethror1 Sep 19, 2024
c145cf9
update bedtools install
jethror1 Sep 19, 2024
698b202
update requirement.txt
jethror1 Sep 19, 2024
574d355
lots of unit tests
jethror1 Sep 19, 2024
0199ffe
remove early rounding of aaf
jethror1 Sep 19, 2024
f3e6df8
add test data
jethror1 Sep 19, 2024
e3ef358
update .gitignore
jethror1 Sep 19, 2024
e16ce5b
update readme
jethror1 Sep 19, 2024
31781ab
remove output file name parameter
jethror1 Sep 19, 2024
b9e2775
remove developer readme
jethror1 Sep 19, 2024
2359b08
fix return in get_het_hom_counts
jethror1 Sep 20, 2024
324acb4
ensure only autosomes used for calculations
jethror1 Sep 20, 2024
54522b6
add tests for is_autosome
jethror1 Sep 20, 2024
11993a1
update comment
jethror1 Sep 20, 2024
b20429a
improve log of record
jethror1 Sep 20, 2024
fb926e1
fix to allow running locally and in DNAnexus
jethror1 Sep 20, 2024
88e58a2
fix pep8 issues
jethror1 Sep 20, 2024
48eed12
improve prints
jethror1 Sep 20, 2024
ef94e44
add comment
jethror1 Sep 20, 2024
ae857d9
update output field help
jethror1 Sep 20, 2024
c320619
Black reformat
jethror1 Sep 20, 2024
dbf1865
remove x var confusion
jethror1 Sep 20, 2024
f9d363f
improve logging of variants with missing fields
jethror1 Sep 20, 2024
b1a68d5
add unit tests for main
jethror1 Sep 20, 2024
1d424bd
add empty vcf
jethror1 Sep 20, 2024
963950f
add additional unit tests for main
jethror1 Sep 20, 2024
5db01f1
improve counter
jethror1 Sep 20, 2024
d766f84
separate alt het variants out
jethror1 Sep 20, 2024
fea204b
update readme
jethror1 Sep 20, 2024
429be63
add alt het variant to test.vcf
jethror1 Sep 20, 2024
d872a7f
update tests for alt het variant
jethror1 Sep 20, 2024
6224a32
add tests for skipping variants with missing values
jethror1 Sep 20, 2024
20aa472
fix readme typo
jethror1 Sep 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[run]
omit = tests/*
28 changes: 28 additions & 0 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: pytest
on: [push, pull_request]

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.8
uses: actions/setup-python@v1
with:
python-version: 3.8
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pipenv codecov pytest pytest-cov pytest-mock pytest-subtests
pip install -r requirements.txt
pipenv install --dev
- name: Install bedtools
run: |
wget https://github.com/arq5x/bedtools2/releases/download/v2.31.0/bedtools.static -O bedtools
chmod a+x bedtools
sudo mv bedtools /usr/bin/bedtools

bedtools
- name: Test with pytest
run: |
pytest -vv --cov .
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
*.vcf*
*.bed
*.pyc
.coverage
28 changes: 11 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
<!-- dx-header -->
# dnanexus_vcf_QC (DNAnexus Platform App)
# eggd_vcf_qc (DNAnexus Platform App)

## What does this app do?
This app runs a python script to perform simple vcf QC.
This app runs a python script to calculate the mean het, mean hom and het:hom ratio of non reference variants in a given vcf.

## What are typical use cases for this app?
This app should be executed stand-alone or as part of a DNAnexus workflow for a single sample.
Expand All @@ -11,20 +11,14 @@ This app should be executed stand-alone or as part of a DNAnexus workflow for a
The app requires a VCF file (.vcf) containing variants to be evaluated and a bed file defining the regions within which variants in the vcf should be evaluated.

## What does this app output?
The app outputs one file, where [outPrefix] is the vcf filename without extension:
1. [outPrefix].vcf.QC is a tab delimited file containing:
- sample id
- mean het ratio (mean AAF of het variants)
- mean homo ratio (mean AAF of hom variants)
- het:homo ratio (ratio of het to hom variants on autosomes)
- X homo:het ratio (ratio of het to hom variants on chrX)
- gender (inferred from X homo:het ratio)

## How does this app work?
The app runs a bash script which runs the python script which generated the output file. The output file is then uploaded to dnanexus.

## What are the limitations of this app
- Inferred gender is tuned to work with TSOE data and the TSOnePlus.bed. Use of other assays/bed files may result in incorrect inferred gender.
- Inferred gender may generate incorrect results for biological reasons e.g. for patient's whose parents are closely related, or patients with sex chromosome anomolies (Turner, Klinefelter etc.)
The app outputs one tab delimited file `[vcf_prefix].vcf.qc`, where `vcf_prefix` is the vcf filename without extension:

* sample id
* mean het ratio (mean AAF of het variants)
* mean hom ratio (mean AAF of hom variants)
* het:hom ratio (ratio of het to hom variants on autosomes)
* X het:hom ratio (ratio of het to hom variants on chrX)


## This app was made by EMEE GLH

31 changes: 0 additions & 31 deletions Readme.developer.md

This file was deleted.

42 changes: 23 additions & 19 deletions dxapp.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
{
"name": "eggd_vcf_qc_v1.0.1",
"title": "eggd_vcf_qc_v1.0.1",
"name": "eggd_vcf_qc",
"title": "eggd_vcf_qc",
"summary": "Simple QC of a vcf file. Includes deviation from expected allelic balance",
"tags":["NGS QC"],
"properties": {
"githubRelease": "v1.0.1"
},
"dxapi": "1.0.0",
"version": "2.0.0",
"inputSpec": [
{
"name": "vcf_file",
Expand All @@ -22,7 +20,6 @@
"help": "A bed file containing the regions to be evaluated in the vcf file",
"class": "file",
"optional": false,
"default":{"$dnanexus_link":{"project":"project-Fkb6Gkj433GVVvj73J7x8KbV", "id":"file-FkyGjY8433GXG33JF06B9pKP"}},
"patterns": ["*.bed"]
}
],
Expand All @@ -33,7 +30,7 @@
"patterns": [
"*"
],
"help": "A tab delimited vcf.QC file"
"help": "A tab delimited vcf.qc file"
}
],
"runSpec": {
Expand All @@ -42,23 +39,30 @@
"hours": 1
}
},
"execDepends": [
{"name": "pysam",
"package_manager": "pip",
"version": "0.12.0"}
],
"interpreter": "bash",
"release": "16.04",
"interpreter": "python3",
"release": "20.04",
"version": "0",
"distribution": "Ubuntu",
"file": "src/dnanexus_vcf_QC.sh"
"file": "src/vcf_qc.py",
"assetDepends": [
{
"name": "bedtools",
"project": "project-Fkb6Gkj433GVVvj73J7x8KbV",
"folder": "/app_assets/bedtools/bedtools_v2.30.0/",
"version": "2.30.0"
}
]
},
"access": {
"project": "CONTRIBUTE",
"allProjects": "VIEW",
"network": [
"*"
]
"allProjects": "VIEW"
},
"developers":[
"org-emee_1"
],
"authorizedUsers": [
"org-emee_1"
],
"regionalOptions": {
"aws:eu-central-1": {
"systemRequirements": {
Expand Down
2 changes: 2 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
dxpy==0.382.0
pysam==0.18.0
Binary file not shown.
141 changes: 0 additions & 141 deletions resources/usr/bin/vcf_QC.py

This file was deleted.

Empty file added src/__init__.py
Empty file.
Loading
Loading