Skip to content

Commit

Permalink
add COMPAS examples documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
TFRicardoMoya committed Dec 9, 2023
1 parent 6bd59bf commit 75322f9
Show file tree
Hide file tree
Showing 17 changed files with 324 additions and 92 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,9 @@ venv.bak/

# mkdocs documentation
/site
/docs/build/html
/docs/build/log
/docs/build/*.tar.gz

# mypy
.mypy_cache/
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,15 +120,15 @@ This entry point takes the following parameters:

XAIoGraphs has been developed by the AI Products team (Telefónica I+D - Chief Data Officer)

* [Ricardo Moya](https://github.com/TFRicardoMoya)
* [Ricardo Moya](https://www.linkedin.com/in/phdricardomoya/)
* [Matteo Salvatori](https://github.com/matteo-salvatori)
* [Enrique Fernandez](https://github.com/QuiqueFdez)
* [Alejandro Manuel Arranz](https://github.com/cx02747)
* [Manuel Martín](https://github.com/mmarmar)
* [Mario Villaizan](https://github.com/mvvmvv)
* [Cesar García](https://github.com/cesarggtid)
* [David Cadenas](https://github.com/davidcadi)
* Alejandra Maria Alonso
* [Alejandra Maria Alonso](https://www.linkedin.com/in/alejandraalonsodiaz/)
* [Miguel Angel Martín](https://github.com/mamj-telefonica)
* [Oriol Arnau](https://github.com/oarnau)

Expand All @@ -137,4 +137,4 @@ XAIoGraphs has been developed by the AI Products team (Telefónica I+D - Chief D

# 📥 Contact

Contact with [@Ricardo Moya](https://github.com/TFRicardoMoya)
Contact with [@Ricardo Moya](https://www.linkedin.com/in/phdricardomoya/)
Binary file added docs/imgs/compas_example/XaioWeb_Fairness.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 5 additions & 0 deletions docs/source/api_reference/api_reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,11 @@ raw specifications might not be sufficient to provide comprehensive instructions
xaiographs.datasets.load_titanic
xaiographs.datasets.load_titanic_discretized
xaiographs.datasets.load_titanic_why
xaiographs.datasets.load_compas
xaiographs.datasets.load_compas_discretized
xaiographs.datasets.load_compas_why
xaiographs.datasets.load_compas_reality_discretized
xaiographs.datasets.load_compas_reality_why
xaiographs.datasets.load_body_performance
xaiographs.datasets.load_body_performance_discretized
xaiographs.datasets.load_body_performance_why
Expand Down
10 changes: 10 additions & 0 deletions docs/source/api_reference/datasets.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@
xaiographs.datasets.load_titanic
xaiographs.datasets.load_titanic_discretized
xaiographs.datasets.load_titanic_why
xaiographs.datasets.load_compas
xaiographs.datasets.load_compas_discretized
xaiographs.datasets.load_compas_why
xaiographs.datasets.load_compas_reality_discretized
xaiographs.datasets.load_compas_reality_why
xaiographs.datasets.load_body_performance
xaiographs.datasets.load_body_performance_discretized
xaiographs.datasets.load_body_performance_why
Expand All @@ -23,6 +28,11 @@
.. autofunction:: xaiographs.datasets.load_titanic
.. autofunction:: xaiographs.datasets.load_titanic_discretized
.. autofunction:: xaiographs.datasets.load_titanic_why
.. autofunction:: xaiographs.datasets.load_compas
.. autofunction:: xaiographs.datasets.load_compas_discretized
.. autofunction:: xaiographs.datasets.load_compas_why
.. autofunction:: xaiographs.datasets.load_compas_reality_discretized
.. autofunction:: xaiographs.datasets.load_compas_reality_why
.. autofunction:: xaiographs.datasets.load_body_performance
.. autofunction:: xaiographs.datasets.load_body_performance_discretized
.. autofunction:: xaiographs.datasets.load_body_performance_why
Expand Down
6 changes: 3 additions & 3 deletions docs/source/contributors/contributors.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# 🤝 Contributors

XAIoGraphs has been developed by AI Products team (Telefónica I+D - Chief Data Officer)
XAIoGraphs has been developed by ***Applied AI & Privacy*** team (Telefónica Innovación Digital - Chief Data Officer)

* [Ricardo Moya](https://github.com/TFRicardoMoya)
* [Ricardo Moya](https://www.linkedin.com/in/phdricardomoya/)
* [Matteo Salvatori](https://github.com/matteo-salvatori)
* [Enrique Fernandez](https://github.com/QuiqueFdez)
* [Alejandro Manuel Arranz](https://github.com/cx02747)
* [Manuel Martín](https://github.com/mmarmar)
* [Mario Villaizan](https://github.com/mvvmvv)
* [Cesar García](https://github.com/cesarggtid)
* [David Cadenas](https://github.com/davidcadi)
* Alejandra Maria Alonso
* [Alejandra Maria Alonso](https://www.linkedin.com/in/alejandraalonsodiaz/)
* [Miguel Angel Martín](https://github.com/mamj-telefonica)
* [Oriol Arnau](https://github.com/oarnau)
123 changes: 119 additions & 4 deletions docs/source/examples/compas.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,129 @@
[< ✏️ Examples](examples/examples)

# Compas Example
# COMPAS Example

This example highlights the most important features that the AI model considered while predicting the likelihood of
criminal recidivism. The [`COMPAS Dataset`](../user_guide/datasets.md#compas) is used to capture the characteristics
of criminals in Broward (Florida) between years 2013 and 2014.


This dataset can be obtained using the [`load_compas()`](../api_reference/datasets.md#xaiographs.datasets.load_compas)
function:

```python
>>> from xaiographs.datasets import load_compas
>>> df_dataset = load_compas()
>>> df_dataset.head(3)
id FirstName LastName Gender Age_range Ethnicity days_b_screening_arrest c_jail_in c_jail_out Days_in_jail c_charge_degree c_charge_desc is_recid is_violent_recid score_risk_recidivism score_text_risk_recidivism score_risk_violence score_text_risk_violence Low_Recid Medium_Recid High_Recid No_Recid score_text_risk_violence Low_Recid Medium_Recid High_Recid No_Recid Recid predict_two_year_recid real_two_year_recid
0 1 miguel hernandez Male Greater than 45 Other -1.0 13/8/13 6:03 14/8/13 5:41 1 F Aggravated Assault w/Firearm 0 0 1 Low 1 Low 1 0 0 1 Low 1 0 0 1 0 0 0
1 3 kevon dixon Male 25 - 45 African-American -1.0 26/1/13 3:45 5/2/13 5:36 10 F Felony Battery w/Prior Convict 1 1 3 Low 1 Low 1 0 0 0 Low 1 0 0 0 1 0 1
2 5 marcu brown Male Less than 25 African-American NaN NaN NaN 0 F Possession of Cannabis 0 0 8 High 6 Medium 0 0 1 1 Medium 0 0 1 1 0 1 0
```

To determine the explainability of this dataset, XAIoGraphs provides a dataset that has already been discretized and
columns with targets probabilities using
[`load_compas_discretized()`](../api_reference/datasets.md#xaiographs.datasets.load_compas_discretized) function:


```python
>>> from xaiographs.datasets import load_compas_discretized
>>> df_dataset, features_cols, target_cols, y_true, y_predict = load_compas_discretized()
>>> df_dataset.head(3)
id Gender Age_range Ethnicity MaritalStatus c_charge_degree is_recid is_violent_recid High_Recid Medium_Recid Low_Recid y_true y_predict
0 1 Male Greater than 45 Other Single F NO NO 0 0 1 No_Recid No_Recid
1 3 Male 25 - 45 African-American Single F YES YES 0 0 1 Recid No_Recid
2 5 Male 25 - 45 Other Separated M NO NO 0 0 1 No_Recid No_Recid
```


&nbsp;
## Compas Explainer
## Code Example

The following entry point (with Python virtual environment enabled) is used to demonstrate this example.

```python
>> compas_example
```

Alternatively, you may run the code below to view a full implementation of all XAIoGraphs functionalities with this Dataset:

```python
from xaiographs import Explainer
from xaiographs import Why
from xaiographs import Fairness
from xaiographs.datasets import load_compas_discretized, load_compas_why

LANG = 'en'

# LOAD DATASETS & SEMANTICS
df_compas, feature_cols, target_cols, y_true, y_predict = load_compas_discretized()
df_values_semantics, df_target_values_semantics = load_compas_why(language=LANG)

# EXPLAINER
explainer = Explainer(importance_engine='LIDE', verbose=1)
explainer.fit(df=df_compas, feature_cols=feature_cols, target_cols=target_cols)

# WHY
why = Why(language=LANG,
explainer=explainer,
why_values_semantics=df_values_semantics,
why_target_values_semantics=df_target_values_semantics,
verbose=1)
why.fit()

# FAIRNESS
f = Fairness(verbose=1)
f.fit(df=df_compas[feature_cols + [y_true] + [y_predict]],
sensitive_cols=['Ethnicity', 'Gender', 'Age_range'],
target_col=y_true,
predict_col=y_predict)
```

&nbsp;
## Compas Reason Why
## XAIoWeb COMPAS


After running the `.fit()` methods of each of the classes (one, two, or all three), a sequence of JSON files are
generated in the `xaioweb_files` folder to visualized in XAIoWeb interface.


To launch the web (with the virtual environment enabled), run the following entry point:

```python
>> xaioweb -d xaioweb_files -o -f
```

And the results seen in XAIoWeb are the following:

&nbsp;
#### Global Explainability
&nbsp;
```{image} ../../imgs/compas_example/XaioWeb_Global_Explainability.png
:alt: Global Explainability
:class: bg-primary
:width: 600px
:align: center
```

&nbsp;
#### Local Explainability
&nbsp;
```{image} ../../imgs/compas_example/XaioWeb_Local_Explainability.png
:alt: Local Explainability
:class: bg-primary
:width: 600px
:align: center
```

&nbsp;
#### Fairness
&nbsp;
```{image} ../../imgs/compas_example/XaioWeb_Fairness.png
:alt: Fairness
:class: bg-primary
:width: 600px
:align: center
```
&nbsp;
## Compas Fairness

[< ✏️ Examples](examples/examples)
110 changes: 106 additions & 4 deletions docs/source/examples/compas_reality.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,116 @@
[< ✏️ Examples](examples/examples)

# Compas Example
# COMPAS Reality Example

This example demonstrates which features are most important that indicate the recidivism of an criminal.
The [`COMPAS Dataset`](../user_guide/datasets.md#compas) was utilized to determine the characteristics of the
criminals and whether or not they reoffended two years following the crime.

```{warning}
This example explains reality, not model predictions. As a result, we cannot assess the model's fairness because we
lack predictions (y_predict) and cannot compare them to reality (y_true).
```

This dataset can be obtained using the [`load_compas()`](../api_reference/datasets.md#xaiographs.datasets.load_compas)
function:

```python
>>> from xaiographs.datasets import load_compas
>>> df_dataset = load_compas()
>>> df_dataset.head(3)
id FirstName LastName Gender Age_range Ethnicity days_b_screening_arrest c_jail_in c_jail_out Days_in_jail c_charge_degree c_charge_desc is_recid is_violent_recid score_risk_recidivism score_text_risk_recidivism score_risk_violence score_text_risk_violence Low_Recid Medium_Recid High_Recid No_Recid score_text_risk_violence Low_Recid Medium_Recid High_Recid No_Recid Recid predict_two_year_recid real_two_year_recid
0 1 miguel hernandez Male Greater than 45 Other -1.0 13/8/13 6:03 14/8/13 5:41 1 F Aggravated Assault w/Firearm 0 0 1 Low 1 Low 1 0 0 1 Low 1 0 0 1 0 0 0
1 3 kevon dixon Male 25 - 45 African-American -1.0 26/1/13 3:45 5/2/13 5:36 10 F Felony Battery w/Prior Convict 1 1 3 Low 1 Low 1 0 0 0 Low 1 0 0 0 1 0 1
2 5 marcu brown Male Less than 25 African-American NaN NaN NaN 0 F Possession of Cannabis 0 0 8 High 6 Medium 0 0 1 1 Medium 0 0 1 1 0 1 0
```

To determine the explainability of this dataset, XAIoGraphs provides a dataset that has already been discretized and
columns with targets probabilities using
[`load_compas_reality_discretized()`](../api_reference/datasets.md#xaiographs.datasets.load_compas_reality_discretized)
function:


```python
>>> from xaiographs.datasets import load_compas_reality_discretized
>>> df_dataset, features_cols, target_cols = load_compas_reality_discretized()
>>> df_dataset.head(3)
id Gender Age_range Ethnicity MaritalStatus c_charge_degree is_recid is_violent_recid Recid No_Recid
0 1 Male Greater than 45 Other Single F NO NO 0 1
1 3 Male 25 - 45 African-American Single F YES YES 1 0
2 5 Male 25 - 45 Other Separated M NO NO 0 1
```


&nbsp;
## Compas Explainer
## Code Example

The following entry point (with Python virtual environment enabled) is used to demonstrate this example.

```python
>> compas_reality_example
```

Alternatively, you may run the code below to view a full implementation of all XAIoGraphs functionalities with this Dataset:

```python
from xaiographs import Explainer
from xaiographs import Why
from xaiographs.datasets import load_compas_reality_discretized, load_compas_reality_why

LANG = 'en'

# LOAD DATASETS & SEMANTICS
df_compas, feature_cols, target_cols = load_compas_reality_discretized()
df_values_semantics, df_target_values_semantics = load_compas_reality_why(language=LANG)

# EXPLAINER
explainer = Explainer(importance_engine='LIDE', verbose=1)
explainer.fit(df=df_compas, feature_cols=feature_cols, target_cols=target_cols)

# WHY
why = Why(language=LANG,
explainer=explainer,
why_values_semantics=df_values_semantics,
why_target_values_semantics=df_target_values_semantics,
verbose=1)
why.fit()
```

&nbsp;
## Compas Reason Why
## XAIoWeb COMPAS


After running the `.fit()` methods of each of the classes (one, two, or all three), a sequence of JSON files are
generated in the `xaioweb_files` folder to visualized in XAIoWeb interface.


To launch the web (with the virtual environment enabled), run the following entry point:

```python
>> xaioweb -d xaioweb_files -o -f
```

And the results seen in XAIoWeb are the following:

&nbsp;
#### Global Explainability
&nbsp;
```{image} ../../imgs/compas_reality_example/XaioWeb_Global_Explainability.png
:alt: Global Explainability
:class: bg-primary
:width: 600px
:align: center
```

&nbsp;
#### Local Explainability
&nbsp;
```{image} ../../imgs/compas_reality_example/XaioWeb_Local_Explainability.png
:alt: Local Explainability
:class: bg-primary
:width: 600px
:align: center
```
&nbsp;
## Compas Fairness

[< ✏️ Examples](examples/examples)
18 changes: 9 additions & 9 deletions docs/source/examples/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ XAIoGraphs contains a set of examples that can be executed as `entry points`:



| Example | Entry Point | Rows | Num. Feats | Task |
|:--------------------------------------------------|:-------------------------------|:-----:|:----------:|:---------------:|
| [Titanic](titanic.md) | titanic_example | 1309 | 8 | Binary |
| [Body Performace](body_performance.md) | body_performance_example | 13393 | 11 | Multi-Class (3) |
| [Education Performance](education_performance.md) | education_performance_example | 145 | 29 | Multi-Class (5) |

[//]: # (| [Compas]&#40;compas.md&#41; | compas_example | ???? | ???? | Binary |)
| Example | Entry Point | Rows | Num. Feats | Task |
|:--------------------------------------------------|:------------------------------|:-----:|:----------:|:----------------:|
| [Titanic](titanic.md) | titanic_example | 1309 | 8 | Binary |
| [COMPAS](compas.md) | compas_example | 4230 | 7 | Multi-Class (3) |
| [COMPAS Reality](compas_reality.md) | compas_reality_example | 4230 | 7 | Binary |
| [Body Performace](body_performance.md) | body_performance_example | 13393 | 11 | Multi-Class (3) |
| [Education Performance](education_performance.md) | education_performance_example | 145 | 29 | Multi-Class (5) |


Use the entry points to see an example run with the XAIoGraphs library installed in a Python virtual environment
Expand All @@ -32,7 +32,7 @@ Run the following command with the virtual environment enabled to see the result
You can see more information about each of these examples at the links below:

* [Titanic Example](titanic.md)
* [Compas Example](compas.md)
* [Compas Reality Example](compas_reality.md)
* [Body Performace Example](body_performance.md)
* [Education Performance Example](education_performance.md)

[//]: # (* [Compas Example]&#40;compas.md&#41;)
4 changes: 4 additions & 0 deletions docs/source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ It displays the explanations outcomes in three sections: Global, Local and Fairn
```{include} quickstart/quickstart.md
```

<hr>

```{include} contributors/contributors.md
```

```{toctree}
:hidden:
Expand Down
Loading

0 comments on commit 75322f9

Please sign in to comment.