Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calculation of Feature Importance incorrect #263

Open
2 tasks done
operdeck opened this issue Sep 20, 2024 · 2 comments
Open
2 tasks done

Calculation of Feature Importance incorrect #263

operdeck opened this issue Sep 20, 2024 · 2 comments
Assignees
Labels
bug Something isn't working Python Issues related to the Python tools

Comments

@operdeck
Copy link
Collaborator

operdeck commented Sep 20, 2024

pdstools version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pdstools.

Issue description

The Feature Importance for NB models calculated by PDS tools isn't the same as in platform
The R version has a subtle issue not using the right laplace smoothing (1 rather than 1/#bins)
The Python version seems totally off, not calculating the diff from the mean and not scaling
Platform suffers from same issues as python implementation, tracking this under BUG-880410

Reproducible example

See Excel sheet for analysis

Expected behavior

All versions should give the exact same results

Installed versions

n/a, issues have been around for a while

@operdeck operdeck added bug Something isn't working Python Issues related to the Python tools labels Sep 20, 2024
@StijnKas
Copy link
Collaborator

@operdeck can we squeeze in a fix for #260 for this? Or do we not have a fix yet

@operdeck
Copy link
Collaborator Author

Lets park this one for a little, I am not certain on the solution. Explored a lot of things, then found that many of the variations are (very) strongly correlated, even with univariate AUC. So made this much lower prio for myself. Will pick up post v4 release, still valid, but not urgent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Python Issues related to the Python tools
Projects
None yet
Development

No branches or pull requests

2 participants