Skip to content

Commit

Permalink
Merge pull request #7 from abichat/dev
Browse files Browse the repository at this point in the history
v0.1.0
  • Loading branch information
abichat authored Aug 16, 2022
2 parents 2322cbb + 0c77e0c commit a545ebb
Show file tree
Hide file tree
Showing 13 changed files with 102 additions and 133 deletions.
31 changes: 17 additions & 14 deletions .github/workflows/R-CMD-check-dev.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
on:
push:
branches:
- dev
pull_request:
branches:
- dev
Expand All @@ -10,16 +9,20 @@ name: R-CMD-check-dev

jobs:
R-CMD-check:
if: "!contains(github.event.head_commit.message, 'skip ci')"
runs-on: macOS-latest
runs-on: ubuntu-latest
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
R_KEEP_PKG_SOURCE: yes
steps:
- uses: actions/checkout@v2
- uses: r-lib/actions/setup-r@master
- name: Install dependencies
run: |
install.packages(c("remotes", "rcmdcheck"))
remotes::install_deps(dependencies = TRUE)
shell: Rscript {0}
- name: Check
run: rcmdcheck::rcmdcheck(args = "--no-manual", error_on = "error")
shell: Rscript {0}

- uses: r-lib/actions/setup-r@v2
with:
use-public-rspm: true

- uses: r-lib/actions/setup-r-dependencies@v2
with:
extra-packages: any::rcmdcheck
needs: check

- uses: r-lib/actions/check-r-package@v2
70 changes: 19 additions & 51 deletions .github/workflows/R-CMD-check.yaml
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
on:
push:
branches:
- master
pull_request:
branches:
- master
- dev

name: R-CMD-check

jobs:
R-CMD-check:
if: "!contains(github.event.head_commit.message, 'skip ci')"
runs-on: ${{ matrix.config.os }}

name: ${{ matrix.config.os }} (${{ matrix.config.r }})
Expand All @@ -19,63 +18,32 @@ jobs:
fail-fast: false
matrix:
config:
- {os: macOS-latest, r: 'release'}
- {os: windows-latest, r: 'release'}
- {os: macOS-latest, r: 'release'}
- {os: macOS-latest, r: 'devel'}
- {os: ubuntu-16.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/xenial/latest"}
- {os: ubuntu-latest, r: 'devel', http-user-agent: 'release'}
- {os: ubuntu-latest, r: 'release'}
- {os: ubuntu-latest, r: 'oldrel-1'}

env:
R_REMOTES_NO_ERRORS_FROM_WARNINGS: true
RSPM: ${{ matrix.config.rspm }}
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
R_KEEP_PKG_SOURCE: yes

steps:
- uses: actions/checkout@v2

- uses: r-lib/actions/setup-r@master
- uses: r-lib/actions/setup-pandoc@v2

- uses: r-lib/actions/setup-r@v2
with:
r-version: ${{ matrix.config.r }}
http-user-agent: ${{ matrix.config.http-user-agent }}
use-public-rspm: true

- uses: r-lib/actions/setup-pandoc@master

- name: Query dependencies
run: |
install.packages('remotes')
saveRDS(remotes::dev_package_deps(dependencies = TRUE), ".github/depends.Rds", version = 2)
writeLines(sprintf("R-%i.%i", getRversion()$major, getRversion()$minor), ".github/R-version")
shell: Rscript {0}

- name: Cache R packages
if: runner.os != 'Windows'
uses: actions/cache@v1
- uses: r-lib/actions/setup-r-dependencies@v2
with:
path: ${{ env.R_LIBS_USER }}
key: ${{ runner.os }}-${{ hashFiles('.github/R-version') }}-1-${{ hashFiles('.github/depends.Rds') }}
restore-keys: ${{ runner.os }}-${{ hashFiles('.github/R-version') }}-1-

- name: Install system dependencies
if: runner.os == 'Linux'
env:
RHUB_PLATFORM: linux-x86_64-ubuntu-gcc
run: |
Rscript -e "remotes::install_github('r-hub/sysreqs')"
sysreqs=$(Rscript -e "cat(sysreqs::sysreq_commands('DESCRIPTION'))")
sudo -s eval "$sysreqs"
- name: Install dependencies
run: |
remotes::install_deps(dependencies = TRUE)
remotes::install_cran("rcmdcheck")
shell: Rscript {0}

- name: Check
env:
_R_CHECK_CRAN_INCOMING_REMOTE_: false
run: rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning", check_dir = "check")
shell: Rscript {0}
extra-packages: any::rcmdcheck
needs: check

- name: Upload check results
if: failure()
uses: actions/upload-artifact@master
- uses: r-lib/actions/check-r-package@v2
with:
name: ${{ runner.os }}-r${{ matrix.config.r }}-results
path: check
upload-snapshots: true
54 changes: 29 additions & 25 deletions .github/workflows/pkgdown.yaml
Original file line number Diff line number Diff line change
@@ -1,42 +1,46 @@
# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
on:
push:
branches: master
pull_request:
branches: [main, master]
release:
types: [published]
workflow_dispatch:

name: pkgdown

jobs:
pkgdown:
runs-on: macOS-latest
runs-on: ubuntu-latest
# Only restrict concurrency for non-PR jobs
concurrency:
group: pkgdown-${{ github.event_name != 'pull_request' || github.run_id }}
env:
GITHUB_PAT: ${{ secrets.GH_TOKEN }}
steps:
- uses: actions/checkout@v2

- uses: r-lib/actions/setup-r@master
- uses: r-lib/actions/setup-pandoc@v2

- uses: r-lib/actions/setup-pandoc@master

- name: Query dependencies
run: |
install.packages('remotes')
saveRDS(remotes::dev_package_deps(dependencies = TRUE), "depends.Rds", version = 2)
shell: Rscript {0}
- uses: r-lib/actions/setup-r@v2
with:
use-public-rspm: true

- name: Cache R packages
uses: actions/cache@v1
- uses: r-lib/actions/setup-r-dependencies@v2
with:
path: ${{ env.R_LIBS_USER }}
key: macOS-r-3.6-${{ hashFiles('depends.Rds') }}
restore-keys: macOS-r-3.6-
extra-packages: any::pkgdown, local::.
needs: website

- name: Install dependencies
run: |
install.packages("remotes")
remotes::install_deps(dependencies = TRUE)
remotes::install_dev("pkgdown")
- name: Build site
run: pkgdown::build_site_github_pages(new_process = FALSE, install = FALSE)
shell: Rscript {0}

- name: Install package
run: R CMD INSTALL .

- name: Deploy package
run: pkgdown::deploy_to_branch(new_process = FALSE)
shell: Rscript {0}
- name: Deploy to GitHub pages 🚀
if: github.event_name != 'pull_request'
uses: JamesIves/[email protected]
with:
clean: false
branch: gh-pages
folder: docs
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.Rhistory
.RData
.Rproj.user
.DS_Store
21 changes: 8 additions & 13 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,24 +1,19 @@
Package: evabic
Title: Evaluation of Binary Classifiers
Version: 0.0.4
Version: 0.1.0
Authors@R:
person(given = "Antoine",
family = "Bichat",
role = c("aut", "cre"),
email = "[email protected]",
person("Antoine", "Bichat", , "[email protected]", role = c("aut", "cre"),
comment = c(ORCID = "0000-0001-6599-7081"))
Description: Evaluates the performance of binary classifiers.
Computes confusion measures (TP, TN, FP, FN), derived measures (TPR,
FDR, accuracy, F1, DOR, ..), and area under the curve. Outputs are
well suited for nested dataframes.
Description: Evaluates the performance of binary classifiers. Computes
confusion measures (TP, TN, FP, FN), derived measures (TPR, FDR,
accuracy, F1, DOR, ..), and area under the curve. Outputs are well
suited for nested dataframes.
License: GPL-3
URL: https://abichat.github.io/evabic,
https://github.com/abichat/evabic
URL: https://abichat.github.io/evabic, https://github.com/abichat/evabic
BugReports: https://github.com/abichat/evabic/issues
Suggests:
testthat (>= 2.1.0)
Encoding: UTF-8
Language: en-US
LazyData: true
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.0
RoxygenNote: 7.2.1
6 changes: 6 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
# evabic 0.1.0

* Doc update to fix HTML5 issues for CRAN check.

* Considered as stable.

# evabic 0.0.4

* Clear error message when using `nl2c()` with a unnamed logical vector.
Expand Down
3 changes: 2 additions & 1 deletion R/measures.R
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
#' Available measures in \code{evabic}
#'
#' @details
#' \if{html}{\figure{confusionmatrix.png}{options: width="50\%" align="middle"}}
#' \if{html}{\figure{confusionmatrix.png}{options: width="50\%"
#' style="float:middle"}}
#' \if{latex}{\figure{confusionmatrix.png}{options: width=15cm}}
#' \describe{
#' \item{TP}{True Positive}
Expand Down
8 changes: 3 additions & 5 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ knitr::opts_chunk$set(
)
```

# evabic <a href='https://abichat.github.io/evabic'><img src='man/figures/logo.png' align="right" height="139" /></a>
# evabic <a href='https://abichat.github.io/evabic'><img src='man/figures/logo.png' style="float:right" height="139" /></a>

```{r, echo = FALSE}
version <- as.vector(read.dcf('DESCRIPTION')[, 'Version'])
Expand All @@ -22,12 +22,10 @@ version <- gsub('-', '.', version)

<!-- badges: start -->
[![license](https://img.shields.io/badge/license-GPL--3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0.en.html)
[![lifecycle](https://img.shields.io/badge/lifecycle-maturing-blue.svg)](https://www.tidyverse.org/lifecycle/#maturing)
[![lifecycle](https://img.shields.io/badge/lifecycle-stable-brightgreen.svg)](https://www.tidyverse.org/lifecycle/#stable)
[![packageversion](https://img.shields.io/badge/package%20version-`r version`-orange.svg)](https://github.com/abichat/evabic/blob/master/DESCRIPTION)
[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/evabic)](https://cran.r-project.org/package=evabic)
[![Codecov test coverage](https://codecov.io/gh/abichat/evabic/branch/master/graph/badge.svg)](https://codecov.io/gh/abichat/evabic?branch=master)
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/c32dcc4c1c3f40a5950e1c10ea6dfb18)](https://www.codacy.com/app/abichat/evabic?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=abichat/evabic&amp;utm_campaign=Badge_Grade)
[![R build status](https://github.com/abichat/evabic/workflows/R-CMD-check/badge.svg)](https://github.com/abichat/evabic/actions)
[![R-CMD-check](https://github.com/abichat/evabic/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/abichat/evabic/actions/workflows/R-CMD-check.yaml)
[![Documentation](https://img.shields.io/badge/documentation-pkgdown-E91E63.svg)](https://abichat.github.io/evabic/)
[![last-commit](https://img.shields.io/github/last-commit/abichat/evabic.svg)](https://github.com/abichat/evabic/commits/master)
<!-- badges: end -->
Expand Down
33 changes: 13 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,15 @@

<!-- README.md is generated from README.Rmd. Please edit that file -->

# evabic <a href='https://abichat.github.io/evabic'><img src='man/figures/logo.png' align="right" height="139" /></a>
# evabic <a href='https://abichat.github.io/evabic'><img src='man/figures/logo.png' style="float:right" height="139" /></a>

<!-- badges: start -->

[![license](https://img.shields.io/badge/license-GPL--3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0.en.html)
[![lifecycle](https://img.shields.io/badge/lifecycle-maturing-blue.svg)](https://www.tidyverse.org/lifecycle/#maturing)
[![packageversion](https://img.shields.io/badge/package%20version-0.0.4-orange.svg)](https://github.com/abichat/evabic/blob/master/DESCRIPTION)
[![CRAN\_Status\_Badge](https://www.r-pkg.org/badges/version/evabic)](https://cran.r-project.org/package=evabic)
[![Codecov test
coverage](https://codecov.io/gh/abichat/evabic/branch/master/graph/badge.svg)](https://codecov.io/gh/abichat/evabic?branch=master)
[![Codacy
Badge](https://api.codacy.com/project/badge/Grade/c32dcc4c1c3f40a5950e1c10ea6dfb18)](https://www.codacy.com/app/abichat/evabic?utm_source=github.com&utm_medium=referral&utm_content=abichat/evabic&utm_campaign=Badge_Grade)
[![R build
status](https://github.com/abichat/evabic/workflows/R-CMD-check/badge.svg)](https://github.com/abichat/evabic/actions)
[![lifecycle](https://img.shields.io/badge/lifecycle-stable-brightgreen.svg)](https://www.tidyverse.org/lifecycle/#stable)
[![packageversion](https://img.shields.io/badge/package%20version-0.1.0-orange.svg)](https://github.com/abichat/evabic/blob/master/DESCRIPTION)
[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/evabic)](https://cran.r-project.org/package=evabic)
[![R-CMD-check](https://github.com/abichat/evabic/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/abichat/evabic/actions/workflows/R-CMD-check.yaml)
[![Documentation](https://img.shields.io/badge/documentation-pkgdown-E91E63.svg)](https://abichat.github.io/evabic/)
[![last-commit](https://img.shields.io/github/last-commit/abichat/evabic.svg)](https://github.com/abichat/evabic/commits/master)
<!-- badges: end -->
Expand Down Expand Up @@ -44,16 +39,14 @@ False Discovery Rate, Accuracy, F1…

``` r
evabic::ebc_allmeasures
#> [1] "TP" "FP" "FN" "TN" "TPR" "TNR" "PPV" "NPV" "FNR" "FPR" "FDR" "FOR"
#> [13] "ACC" "BACC" "F1" "PLR" "NLR" "DOR"
#> [1] "TP" "FP" "FN" "TN" "TPR" "TNR" "PPV" "NPV" "FNR" "FPR" "FDR"
#> [12] "FOR" "ACC" "BACC" "F1" "PLR" "NLR" "DOR"
```

All measures are computed from the confusion matrix:

<center>

<img src="man/figures/confusionmatrix.png" width="50%"/>

</center>

## Example
Expand Down Expand Up @@ -116,8 +109,10 @@ summary(model)
#> F-statistic: 69.99 on 7 and 42 DF, p-value: < 2.2e-16
pvalues <- summary(model)$coefficients[-1, 4]
pvalues
#> X1 X2 X3 X4 X5 X6 X7
#> 0.004366456 0.829771754 0.003469737 0.491828466 0.581608670 0.887948400 0.449664443
#> X1 X2 X3 X4 X5 X6
#> 0.004366456 0.829771754 0.003469737 0.491828466 0.581608670 0.887948400
#> X7
#> 0.449664443
detected_var <- names(pvalues[pvalues < 0.05])
detected_var
#> [1] "X1" "X3"
Expand Down Expand Up @@ -161,8 +156,7 @@ ebc_tidy(detected = pvalues < 0.05, true = predictors, m = 7,

With `ebc_tidy_by_threshold()`, you can ask for the evolution of
measures according to a moving threshold if you provide the vector of
p-values (or any
score).
p-values (or any score).

``` r
df_measures <- ebc_tidy_by_threshold(detection_values = pvalues, true = predictors, m = 7,
Expand All @@ -179,8 +173,7 @@ df_measures
#> 8 Inf 1.0000000 1.00 0.5714286 0.4285714 0.5000000 0.6000000
```

This makes it easy to plot various-threshold curves like ROC
curve.
This makes it easy to plot various-threshold curves like ROC curve.

``` r
plot(df_measures$FPR, df_measures$TPR, type = "b", xlab = "FPR", ylab = "TPR")
Expand Down
2 changes: 1 addition & 1 deletion dev_history.R
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ library(testthat)
load_all()

document()
attachment::att_to_description()
attachment::att_amend_desc()
use_tidy_description()

# covr::package_coverage()
Expand Down
3 changes: 1 addition & 2 deletions inst/WORDLIST
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
Codacy
Codecov
CMD
DOR
FN
FP
Expand Down
3 changes: 2 additions & 1 deletion man/ebc_allmeasures.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Binary file modified man/figures/README-roc-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit a545ebb

Please sign in to comment.