Skip to content
This repository has been archived by the owner on Dec 19, 2023. It is now read-only.

New way of initializing bedstat #34

Open
wants to merge 38 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
f4bddb9
Dependencies R
khoroshevskyi Sep 15, 2022
0a8b508
update GDdata version
xuebingjie1990 Sep 19, 2022
35a487a
Dependencies R 2
khoroshevskyi Sep 20, 2022
59b6dff
add ability to plot gccontent and chrombins for genomes that not in G…
xuebingjie1990 Sep 20, 2022
d3be284
add ability to plot tss dist for genomes that not in GDdata
xuebingjie1990 Sep 20, 2022
01a5e29
update pipeline_interface
xuebingjie1990 Sep 20, 2022
7178e0d
add ability to plot partivtion plots for genomes that not in GDdata
xuebingjie1990 Sep 21, 2022
a66333a
use GD functions to build tss and geneModel
xuebingjie1990 Sep 26, 2022
233c27a
rm unnecessary print
xuebingjie1990 Sep 26, 2022
aa0d5a9
Dependencies R
khoroshevskyi Sep 15, 2022
1062812
Dependencies R 2
khoroshevskyi Sep 20, 2022
0d4e36b
new function for bedstat
khoroshevskyi Sep 27, 2022
85e083c
Merge remote-tracking branch 'origin/dev_alex' into dev_alex
khoroshevskyi Sep 27, 2022
727ca34
python_v
khoroshevskyi Sep 27, 2022
c9253f6
test _deps
khoroshevskyi Sep 27, 2022
9873c89
test _deps2
khoroshevskyi Sep 27, 2022
83d1d09
updated config
khoroshevskyi Sep 28, 2022
e293a45
updated password
khoroshevskyi Sep 30, 2022
d455799
added default values to run_bedstat
khoroshevskyi Sep 30, 2022
af4282d
test fixe 0.1
khoroshevskyi Sep 30, 2022
827e4a7
test fix 0.2
khoroshevskyi Sep 30, 2022
128219f
added package installation
khoroshevskyi Oct 5, 2022
f53d7dc
first new test
khoroshevskyi Oct 6, 2022
e4a5583
fixed config db
khoroshevskyi Oct 6, 2022
6672b03
added db check
khoroshevskyi Oct 6, 2022
bb846f2
check db entry test
khoroshevskyi Oct 6, 2022
bdad0ac
db error fix
khoroshevskyi Oct 6, 2022
e44d56d
record fix
khoroshevskyi Oct 6, 2022
88b5ed0
added python publishing, tests, and additional files
khoroshevskyi Oct 10, 2022
1577c4c
fixing bug: median_tss_dist not written in the file
xuebingjie1990 Oct 17, 2022
43bdee1
Update regionstat.R
xuebingjie1990 Oct 19, 2022
75a45f8
filter out local path for report to database
xuebingjie1990 Oct 20, 2022
60d82cc
Merge branch 'dev_alex' of github.com:databio/bedstat into dev_alex
xuebingjie1990 Oct 20, 2022
7a5b97b
fix input arg
xuebingjie1990 Oct 20, 2022
2da8744
if schema then filter out local path
xuebingjie1990 Oct 20, 2022
4968ba2
fix typo
xuebingjie1990 Oct 20, 2022
2c44d34
added default schema
khoroshevskyi Oct 20, 2022
82147a3
lint
khoroshevskyi Oct 20, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# This workflows will upload a Python Package using Twine when a release is created
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries

name: Upload Python Package

on:
release:
types: [created]

jobs:
deploy:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.x'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools wheel twine
- name: Build and publish
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
python setup.py sdist bdist_wheel
twine upload dist/*
44 changes: 23 additions & 21 deletions .github/workflows/test-pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@ name: Test bedstat pipeline

on:
push:
branches: [master, dev]
branches: [master, dev, dev_alex]
pull_request:
branches: [master, dev]

jobs:
pytest:
strategy:
matrix:
python-version: [3.6, 3.7, 3.8]
python-version: [3.6, 3.8]
os: [ubuntu-latest] # can't use macOS when using service containers or container jobs
r: [release]
runs-on: ${{ matrix.os }}
Expand Down Expand Up @@ -46,25 +46,27 @@ jobs:
sudo apt-get update
sudo apt-get install libcurl4-openssl-dev

- uses: r-lib/actions/setup-r@master
with:
r-version: ${{ matrix.r }}
- name: Install package
run: python -m pip install .

- name: Install R dependancies
run: Rscript scripts/installRdeps.R
- name: Run pytest tests
run: pytest tests -x -vv

- name: Run pipeline
run: looper run -p local tests/data/bedstat_config.yaml
# - uses: r-lib/actions/setup-r@master
# with:
# r-version: ${{ matrix.r }}
#
# - name: Install R dependancies
# run: Rscript scripts/installRdeps.R

- name: Test plots exist
run: |
if ls $GITHUB_WORKSPACE/outputs/bedstat_output/a6a08126cb6f4b1953ba0ec8675df85a/test_hg38*.png 1> /dev/null 2>&1; then
echo "SUCCESS";
else
echo "ERROR: files do not exist: $GITHUB_WORKSPACE/outputs/bedstat_output/a6a08126cb6f4b1953ba0ec8675df85a/test_hg38*.png";
exit 1
fi

- name: Test record in PostgreSQL
run: |
echo "from bbconf import BedBaseConf; from bbconf.const import *; bbc = BedBaseConf('$GITHUB_WORKSPACE/tests/data/config_min.yaml'); assert bbc.bed.record_count == 1, 'Number of records in the bedfiles table not equal 1'" | python3 -
# - name: Run pipeline
# run: looper run -p local tests/data/bedstat_config.yaml
#
# - name: Test plots exist
# run: |
# if ls $GITHUB_WORKSPACE/outputs/bedstat_output/a6a08126cb6f4b1953ba0ec8675df85a/test_hg38*.png 1> /dev/null 2>&1; then
# echo "SUCCESS";
# else
# echo "ERROR: files do not exist: $GITHUB_WORKSPACE/outputs/bedstat_output/a6a08126cb6f4b1953ba0ec8675df85a/test_hg38*.png";
# exit 1
# fi
9 changes: 9 additions & 0 deletions LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Copyright 2017 Nathan Sheffield

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
6 changes: 6 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
include README.md
include requirements/*
include schemas/*
include bedstat/scripts/*
include bedstat/tools/regionstat.R
include bedstat/pep_schema.yaml
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ The above command will build the csv file looper needs to run the pipeline on al

### 1. Validate your PEP with [`eido`](https://github.com/pepkit/eido)

The input PEP can be validated against the [JSON schema in this repository](pep_schema.yaml). This ensures the PEP consists of all required attributes to run `bedstat` pipeline.
The input PEP can be validated against the [JSON schema in this repository](schemas/pep_schema.yaml). This ensures the PEP consists of all required attributes to run `bedstat` pipeline.

```
eido validate <path/to/pep> -s https://schema.databio.org/pipelines/bedstat.yaml
Expand Down Expand Up @@ -58,7 +58,7 @@ The data loaded into PostgreSQL should persist between PostgreSQL invocations, o

## Additional dependencies

[regionstat.R](tools/regionstat.R) script is used to calculate the bed file statistics, so the pipeline also depends on several R packages:
[regionstat.R](bedstat/tools/regionstat.R) script is used to calculate the bed file statistics, so the pipeline also depends on several R packages:

* `R.utils`
* `BiocManager`
Expand All @@ -71,7 +71,7 @@ The data loaded into PostgreSQL should persist between PostgreSQL invocations, o
* `BSgenome.<organim>.UCSC.<genome>` *depending on the genome used*
* `LOLA`

you can use [installRdeps.R](scripts/installRdeps.R) helper script to easily install the required packages:
you can use [installRdeps.R](bedstat/scripts/installRdeps.R) helper script to easily install the required packages:

```
Rscript scripts/installRdeps.R
Expand Down
3 changes: 3 additions & 0 deletions bedstat/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
""" Package-level data """
from .bedstat import *
from ._version import __version__
1 change: 1 addition & 0 deletions bedstat/_version.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__version__ = "0.1.0"
Loading