Releases: intel/intel-xai-tools
Releases · intel/intel-xai-tools
Intel® Explainable AI Tools v1.1.0
What's Changed
- Generate Model Cards independent of AI Framework (e.g. PyTorch, TensorFlow, etc) by removing TFMA hard dependence.
- Add NeuralChat SUT to ModelGauge for running MLCommons v0.5 Standard Safety Benchmark
- Provide User Interface for creating Model Cards Model Card Generator UI
- Docker Compose infra for Explainer and Model Card Generator deployments
- Adding non-root user to all containers
- Fuzzing support with tests for Explainer and ModelCardGen
- Added Python code styler checker to CICD
Jupyter Notebooks
- Jupyter notebook for running model card generator on hugging face model hugging-face-model-card.ipynb
Validated configuration
- Ubuntu 22.04 LTS
- Python 3.9, 3.10
- PyTorch 2.2.0
- Intel® Optimization for TensorFlow 2.14.0
- Torchvision 0.17.0
- TensorFlow Hub 0.15.0
Known limitations
Intel® Explainable AI Tools is only supported on Linux
GitHub pages:
https://intel.github.io/intel-xai-tools/v1.1.0/
New Contributors
Contributors
mhbuehler and etcylfleet
Assets 2
Intel® Explainable AI Tools v1.0.0
New Features
- Rearchitected project to provide plugin-based approach via Poetry dependency manager
- LLM Explainer: Hugging Face attributions plugin that uses SHAP to explain generative text LLM model output
- Dockerfiles for explainer and model card generator with Jupyter interface and example notebooks
Jupyter Notebooks
Bug fixes
- Removed Pipeline explainer
- Added unit test for new LLM Explainer
- All notebooks and unit tests updated for latest API
Validated configuration
- Ubuntu 22.04 LTS
- Python 3.9, 3.10
- Intel® Optimization for TensorFlow 2.14.0
- PyTorch 2.2.0
- Torchvision 0.17.0
- TensorFlow Hub 0.15.0
Known limitations
- Intel® Explainable AI Tools is only supported on Linux
GitHub pages:
https://intelai.github.io/intel-xai-tools/v1.0.0/
Latest binaries published here https://storage.googleapis.com/public-artifacts/xai/intel_ai_safety-1.0.0-py3-none-any.whl
Intel® Explainable AI Tools v0.6.0
New Features
- Added link to repo on all model card generations
Jupyter Notebooks
- Added EigenCAM notebook
Bug fixes
- Added unit tests for EigenCAM and refactored the EigenCAM class to be consistent with explainer class structures
- Updated versions of dependencies in order to work on python 3.8 - 3.10
- Updated package requirements for notebooks
Validated configuration
- Ubuntu 22.04 LTS
- Python 3.8, 3.9, 3.10
- Intel® Optimization for TensorFlow 2.13.0
- PyTorch 2.0.1
- Torchvision 0.15.2
- TensorFlow Hub 0.14.0
Known limitations
- Intel® Explainable AI Tools is only supported on Linux
GitHub pages:
Intel® Explainable AI Tools v0.5.0
New Features
- ShapUI: a user interface to explore and compare impact scores of model predictions for each record of a tabular data set and discover insights of a model's behavior.
- Added info panel feature with text descriptions designed to help user with interpreting graphs
Jupyter Notebooks
- Added notebook to benchmark
PartitionExplainer()in AI Kit environment against basic Python environment.
Bug fixes
- Improved consistency of code between explainer
visualizemethods - Split TensorFlow* implementations from PyTorch implementation for both Explainer and Model Card Generator
- Fixes to links in documentation
- Improve test coverage for Explainer's attributions module
- Documented how to use Model Card Generator with multiple TF records
- Improved compatibility for dependencies on Python 3.9
- Moved Model Card Generator Notebooks to common directory as Explainer
- Simplified directory structure
- Fixed tests dependence on UCI Machine Learning dataset URL
Validated configuration
- Ubuntu 22.04 LTS
- Python 3.9
- Intel® Optimization for TensorFlow 2.12.0
- PyTorch 1.13.1
- Torchvision 0.14.1
- TensorFlow Hub 0.13.0
Known limitations
- Intel® Explainable AI Tools is only supported on Python 3.9
GitHub pages:
Intel® Explainable AI Tools v0.4.0
New Features:
- ShapUI: a user interface to explore and compare impact scores of model predictions for each record of a tabular data set and discover insights of a model's behavior.
- Added info panel feature with text descriptions designed to help user with interpreting graphs
- Experimental support for
Python 3.10
Validated configuration
- Ubuntu 22.04 LTS
- Python 3.9, 3.10
- Intel® Optimization for TensorFlow 2.11.0
- PyTorch 1.13.1
- Torchvision 0.14.1
- TensorFlow Hub 0.12.0
Known limitations
- Model Card Generator is only supported on Python 3.9 and did not get packaged as part of installer wheel
Intel® Explainable AI Tools v0.3.0
New Features:
- Single installer for both Model card generator and Explainers
Explainers:
- Unified Explainers' APIs
- CAM explainer which utilizes XGradCAM, the SOTA CAM method
- EigenCAM explainer for object detection model (FasterRCNN, YOLO)
- Compatibility support for Frozen models introduced by SciPy 1.10
Jupyter Notebooks
- ResNet50 ImageNet Classification using the CAM Explainer
- Custom CNN MNIST Classification using the Attributions Explainer
- Custom NN NewsGroups Classification using the Attributions Explainer
- Custom CNN CIFAR-10 Classification using the Attributions Explainer
- Multimodal Breast Cancer Detection Explainability
- Fine Tuned Text Classifier with PyTorch using the Intel® Explainable AI API
- Custom Neural Network Heart Disease Classification using the Attributions Explainer
Bug fixes:
- Many documentation improvements
- Improve test coverage for both Explainer and Model card generator-
Validated configuration
- Ubuntu 20.04 LTS
- Python 3.9
- Intel® Optimization for TensorFlow 2.11.0
- PyTorch 1.13.1
- Torchvision 0.14.1
- TensorFlow Hub 0.12.0
Known limitations
- Intel® Explainable AI Tools in only supported on Python 3.9
Intel® Explainable AI Tools v0.2
New Features:
Model Card Generator:
- Support for general model overview plots visualize performance as a function of threshold score.
- Support for interactive plots to visualize fairness metrics across data groupings.
- Added support for Model Card generation for PyTorch models.
- Added support for Model Cards for multiple datasets.
Explainer:
- Allows injection of XAI methods into Python workflows/notebooks without requiring version compatibility of resident packages in the active python environment.
- Supports 3 explainable plugin methods:
- feature attributions: Explains a model’s predictions based on how the model has weighted features it’s been trained on
- metrics: calculates and plots the standard base metrics used to evaluate model performance
- language model explanations: explains transformer based language models by visualizing input token importance, hidden state contributions, sequence embeddings and attention heads
- An interactive CLI allows the user to install each plugin. Provides a simple solution to create new plugins and expand on existing plugins.
- Complete documentation with notebooks examples in the natural language, computer vision, and data frame domain.
Bug fixes:
Model Card Generator:
- N/A
Explainer:
- N/A, Initial public release
Supported Configurations
Intel® Explainable AI Tools v0.2.0 is validated on the following environment:
- Ubuntu 20.04 LTS
- Python 3.9
Intel® Explainable AI Tools v0.0.1
Supported Frameworks
- TensorFlow
New features
- Model Card Generator:
Allows users to create interactive HTML reports of containing model performance and fairness metrics.
Supports general model overview plots visualize performance as a function of threshold score.
Supports interactive plots to visualize fairness metrics across data groupings.
Bug fixes:
- N/A
Supported Configurations
Intel® Explainable AI Tools v0.0.1 is validated on the following environment:
- Ubuntu 20.04 LTS
- Python 3.8, 3.9