Other interesting tools to explain your model:
- [Lucid](https://github.com/tensorflow/lucid) the wonderful library specialized in feature visualization from OpenAI. -- [Captum](https://captum.ai/) the Pytorch library for Interpretability research +- [Captum](https://captum.ai/) the PyTorch library for Interpretability research - [Tf-explain](https://github.com/sicara/tf-explain) that implement multiples attribution methods and propose callbacks API for tensorflow. - [Alibi Explain](https://github.com/SeldonIO/alibi) for model inspection and interpretation - [SHAP](https://github.com/slundberg/shap) a very popular library to compute local explanations using the classic Shapley values from game theory and their related extensions diff --git a/TUTORIALS.md b/TUTORIALS.md index 6dd83321..24aa0791 100644 --- a/TUTORIALS.md +++ b/TUTORIALS.md @@ -1,22 +1,24 @@ # Tutorials: Notebooks π -We propose here several tutorials to discover the different functionnalities that the library has to offer. +We propose here several tutorials to discover the different functionalities that the library has to offer. We decided to host those tutorials on [Google Colab](https://colab.research.google.com/notebooks/intro.ipynb?utm_source=scs-index) mainly because you will be able to play the notebooks with a GPU which should greatly improve your User eXperience. -Here is the lists of the availables tutorial for now: +Here is the lists of the available tutorial for now: ## Getting Started -| **Tutorial Name** | Notebook | -| :-------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------: | -| Getting Started | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | +| **Tutorial Name** | Notebook | +| :------------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------: | +| Getting Started | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | | Sanity checks for Saliency Maps | [](https://colab.research.google.com/drive/1uJOmAg6RjlOIJj6SWN9sYRamBdHAuyaS) | -| Tabular data and Regression | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | -| Metrics | [](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) | -| Concept Activation Vectors | [](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE) | -| Feature Visualization | [](https://colab.research.google.com/drive/1st43K9AH-UL4eZM1S4QdyrOi7Epa5K8v) | +| Tabular data and Regression | [](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) | +| Object detection | [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) | +| Semantic Segmentation | [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) | +| Metrics | [](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) | +| Concept Activation Vectors | [](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE) | +| Feature Visualization | [](https://colab.research.google.com/drive/1st43K9AH-UL4eZM1S4QdyrOi7Epa5K8v) | ## Attributions @@ -52,10 +54,12 @@ Here is the lists of the availables tutorial for now: ## PyTorch Wrapper -| **Tutorial Name** | Notebook | -| :-------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------: | -| PyTorch's model: Getting started | [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) | -| Metrics: With Pytorch's model| [](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) | +| **Tutorial Name** | Notebook | +| :------------------------------------- | :-----------------------------------------------------------------------------------------------------------------------------------------------------: | +| PyTorch models: Getting started | [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) | +| Metrics: With PyTorch models | [](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) | +| Object detection on PyTorch model | [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) | +| Semantic Segmentation on PyTorch model | [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) | ## Concepts extraction diff --git a/docs/api/attributions/api_attributions.md b/docs/api/attributions/api_attributions.md index 60fa6089..74ff3e9c 100644 --- a/docs/api/attributions/api_attributions.md +++ b/docs/api/attributions/api_attributions.md @@ -4,111 +4,364 @@ -## Context + + + + +## Context ## + In 2013, [Simonyan et al.](http://arxiv.org/abs/1312.6034) proposed a first attribution method, opening the way to a wide range of approaches which could be defined as follow: !!!definition The main objective in attributions techniques is to highlight the discriminating variables for decision-making. For instance, with Computer Vision (CV) tasks, the main goal is to underline the pixels contributing the most in the input image(s) leading to the modelβs output(s). -## Common API -All attribution methods inherit from the Base class `BlackBoxExplainer`. This base class can be instanciated with three parameters: -- `model`: the model from which we want to obtain attributions (e.g: InceptionV3, ResNet, ...), see the [model expectations](../model/) for more details -- `batch_size`: an integer which allows to either process inputs per batch (gradient-based methods) or process perturbed samples of an input per batch (inputs are therefore process one by one) -- `operator`: function g to explain, see the [Operator documentation](../operator/) for more details -In addition, all class inheriting from `BlackBoxExplainer` should implement an `explain` method: + + + +## Common API ## ```python -@abstractmethod -def explain(self, - inputs: Union[tf.data.Dataset, tf.Tensor, np.array], - targets: Optional[Union[tf.Tensor, np.array]] = None) -> tf.Tensor: - raise NotImplementedError() - -def __call__(self, - inputs: tf.Tensor, - labels: tf.Tensor) -> tf.Tensor: - """Explain alias""" - return self.explain(inputs, labels) +explainer = Method(model, batch_size, operator) +explanation = explainer(inputs, targets) ``` -`inputs`: Must be one of the following: a `tf.data.Dataset` (in which case you should not provide targets), a `tf.Tensor` or a `np.ndarray`. +The API have two steps: -- If inputs are images, the expected shape of `inputs` is $(N, H, W, C)$ following the TF's conventions where: - - $N$ is the number of inputs - - $H$ is the height of the images - - $W$ is the width of the images - - $C$ is the number of channels (works for $C=3$ or $C=1$, other values might not work or need further customization) +- **`explainer` instantiation**: `Method` is an attribution method among those displayed [methods tables](#methods). It inherits from the Base class `BlackBoxExplainer`. Their initialization takes 3 parameters apart from the specific ones and generates an `explainer`: + - `model`: the model from which we want to obtain attributions (e.g: InceptionV3, ResNet, ...), see the [model section](#model) for more details and specifications. + - `batch_size`: an integer which allows to either process inputs per batch (gradient-based methods) or process perturbed samples of an input per batch (inputs are therefore processed one by one). + - `operator`: enum identifying the task of the model (which is [Classification](../classification/) by default), string identifying the task, or function to explain, see the [task and operator section](#tasks-and-operator) for more detail. + +- **`explainer` call**: The call to `explainer` generates the explanations, it takes two parameters: + - `inputs`: the samples on which the explanations are requested, see [inputs section](#inputs) for more detail. + - `targets`: another parameter to specify what to explain in the `inputs`, it changes depending on the `operator`, see [targets section](#targets) for more detail. + +!!!info + The `__call__` method of explainers is an alias for the `explain` method. + +!!!info + This documentation page covers the different parameters of the common API of attributions methods. It is common between the different [tasks covered](#the-tasks-covered) by Xplique for attribution methods. -- If inputs are tabular data, the expected shape of `inputs` is $(N, W)$ where: - - $N$ is the number of inputs - - $W$ is the feature dimension of a single input - !!!tip - Please refer to the [table](../../../#whats-included) to see which methods might work with Tabular Data -- (Experimental) If inputs are Time Series, the expected shape of `inputs` is $(N, T, W)$ - - $N$ is the number of inputs - - $T$ is the temporal dimension of a single input - - $W$ is the feature dimension of a single input - !!!warning - By default `Lime` & `KernelShap` will treat such inputs as grey images. You will need to define a custom `map_to_interpret_space` function when instantiating those methods in order to create a meaningful mapping of Time-Series data into an interpretable space when building such explainers. Such an example is provided at the end of the [Lime's documentation](../lime/). - !!!note - If your model is not following the same conventions, please refer to the [Model documentation](../model/). -`targets`: Must be one of the following: a `tf.Tensor` or a `np.ndarray`. Its format depends on the task at end and what you want an explanation of. Thus, if you want to know more about `targets` you should read the documentation of [Model](../model/#tasks) or [Operator](../operator/#tasks). -Even though we made an harmonized API for all attributions methods it might be relevant for the user to distinguish Gradient based and Perturbation based methods, also often referenced respectively as white-box and black-box methods, as their hyperparameters settings might be quite different. +## Methods ## -## Perturbation-based approaches -Perturbation based methods focus on perturbing an input with a variety of techniques and, with the analysis of the resulting outputs, define an attribution representation. Thus, **there is no need to explicitly know the model architecture** as long as forward pass is available, which explain why they are also referenced as black-box methods. +Even though we made an harmonized API for all attributions methods, it might be relevant for the user to distinguish [Perturbation-based methods](#perturbation-based-approaches) and [Gradient-based methods](#gradient-based-approaches), also often referenced respectively as black-box and white-box methods, as their hyperparameters settings might be quite different. -Therefore, to use perturbation-based approaches you do not need a TF model. To know more, please see the [Callable](../../../callable) documentation. + + +### Perturbation-based approaches ### + +Perturbation based methods focus on perturbing an input with a variety of techniques and, with the analysis of the resulting outputs, define an attribution representation. Thus, **there is no need to explicitly know the model architecture** as long as forward pass is available, which explains why they are also referenced as black-box methods. + +Therefore, to use perturbation-based approaches you do not need a TF model. To know more, please see the [Callable](../callable/) documentation. Xplique includes the following black-box attributions: -| Method Name | **Tutorial** | Available with TF | Available with PyTorch* | -|:---------------- | :----------------------: | :---------------: | :---------------------: | -| KernelShap | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | β | β | -| Lime | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | β | β | -| Occlusion | [](https://colab.research.google.com/drive/15xmmlxQkNqNuXgHO51eKogXvLgs-sG4q) | β | β | -| Rise | [](https://colab.research.google.com/drive/1icu2b1JGfpTRa-ic8tBSXnqqfuCGW2mO) | β | β | -| Sobol Attribution | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | β | β | -| Hsic Attribution | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | β | β | +| Method Name and Documentation link | **Tutorial** | Available with TF | Available with PyTorch* | +|:-------------------------------------- | :----------------------: | :---------------: | :---------------------: | +| [KernelShap](../methods/kernel_shap/) | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | β | β | +| [Lime](../methods/lime/) | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | β | β | +| [Occlusion](../methods/occlusion/) | [](https://colab.research.google.com/drive/15xmmlxQkNqNuXgHO51eKogXvLgs-sG4q) | β | β | +| [Rise](../methods/rise/) | [](https://colab.research.google.com/drive/1icu2b1JGfpTRa-ic8tBSXnqqfuCGW2mO) | β | β | +| [Sobol Attribution](../methods/sobol/) | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | β | β | +| [Hsic Attribution](../methods/hsic/) | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | β | β | + +*: Before using a PyTorch model it is highly recommended to read the [dedicated documentation](../pytorch/) -*: Before using a PyTorch's model it is highly recommended to read the [dedicated documentation](../../../pytorch) -## Gradient-based approaches -Those approaches are also called white-box methods as **they require a full access to the model architecture**, notably it should **allow computing gradients**. Indeed, the core idea with the gradient-based approach is to use back-propagation, along other techniques, not to update the modelβs weights (which is already trained) but to reveal the most contributing inputs, potentially in a specific layer. All methods are available when the model work with TensorFlow but most methods also works with PyTorch (see [Xplique for PyTorch documentation](../../../pytorch)) +### Gradient-based approaches ### -| Method Name | **Tutorial** | Available with TF | Available with PyTorch* | -|:---------------- | :----------------------: | :---------------: | :---------------------: | -| DeconvNet | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | β |β | -| GradCAM | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | β |β | -| GradCAM++ | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | β |β | -| GradientInput | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | β | β | -| GuidedBackpropagation | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | β | β | -| IntegratedGradients | [](https://colab.research.google.com/drive/1UXJYVebDVIrkTOaOl-Zk6pHG3LWkPcLo) | β | β | -| Saliency | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | β | β | -| SmoothGrad | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | β | β | -| SquareGrad | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | β | β | -| VarGrad | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | β | β | +Those approaches are also called white-box methods as **they require a full access to the model's architecture**, notably it must **allow computing gradients**. Indeed, the core idea with the gradient-based approaches is to use back-propagation, not to update the modelβs weights (which is already trained) but to reveal the most contributing inputs, potentially in a specific layer. All methods are available when the model works with TensorFlow but most methods also work with PyTorch (see [Xplique for PyTorch documentation](../pytorch/)) -*: Before using a PyTorch's model it is highly recommended to read the [dedicated documentation](../../../pytorch) +| Method Name and Documentation link | **Tutorial** | Available with TF | Available with PyTorch* | +|:----------------------------------------------------------- | :----------------------: | :---------------: | :---------------------: | +| [DeconvNet](../methods/deconvnet/) | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | β |β | +| [GradCAM](../methods/grad_cam/) | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | β |β | +| [GradCAM++](../methods/grad_cam_pp/) | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | β |β | +| [GradientInput](../methods/gradient_input/) | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | β | β | +| [GuidedBackpropagation](../methods/guided_backpropagation/) | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | β |β | +| [IntegratedGradients](../methods/integrated_gradients/) | [](https://colab.research.google.com/drive/1UXJYVebDVIrkTOaOl-Zk6pHG3LWkPcLo) | β | β | +| [Saliency](../methods/saliency/) | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | β | β | +| [SmoothGrad](../methods/smoothgrad/) | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | β | β | +| [SquareGrad](../methods/square_grad/) | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | β | β | +| [VarGrad](../methods/vargrad/) | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | β | β | -In addition, those methods inherits from `WhiteBoxExplainer` (itself inheriting from `BlackBoxExplainer`). Thus, an additional `__init__` argument is added: `output_layer`. It is the layer to target for the output (e.g logits or after softmax). If an `int` is provided, it will be interpreted as a layer index, if a `string` is provided it will look for the layer name. Default to the last layer. +*: Before using a PyTorch model it is highly recommended to read the [dedicated documentation](../pytorch/) + +In addition, these methods inherit from `WhiteBoxExplainer` (itself inheriting from `BlackBoxExplainer`). Thus, an additional `__init__` argument is added: `output_layer`. It is the layer to target for the output (e.g logits or after softmax). If an `int` is provided, it will be interpreted as a layer index, if a `string` is provided it will look for the layer name. Default to the last layer. !!!tip It is recommended to use the layer before Softmax. !!!warning - The `output_layer` parameter will work well with TensorFlow's model. However, it will not work with PyTorch's model. For PyTorch, one should directly manipulate the model to focus on the layers of interest. + The `output_layer` parameter will work well with TensorFlow models. However, it will not work with PyTorch models. For PyTorch, one should directly manipulate the model to focus on the layers of interest. + +!!!info + The "white-box" explainers that work with PyTorch are those that only require the gradient of the model without having to "modify" some part of the model (e.g. Deconvnet will commute all original ReLU by a custom ReLU policy) + + + + + + + +## `model` ## + +`model` is the primary parameter of attribution methods: it represents model from which explanations are required. Even though we tried to support a wide-range of models, our attributions framework relies on some assumptions which we propose to see in this section. + +!!!warning + In case the `model` does not respect the specifications, a wrapper will be needed as described in the [Models not respecting the specifications section](#models-not-respecting-the-specifications). + +In practice, we expect the `model` to be callable for the `inputs` parameters -- *i.e.* we can do `model(inputs)`. We expect this call to produce the `outputs` variables that are the predictions of the model on those inputs. As for most attribution methods, we need to manipulate and/or link the `outputs` to the `inputs`. We assume that the latter follow conventional shapes described in the [inputs section](#inputs). !!!info - The "white-box" explainers that work with Pytorch are those that only require the gradient of the model without having to "modify" some part of the model (e.g. Deconvnet will commute all original ReLU by a custom ReLU policy) \ No newline at end of file + Depending on the [task and operator](#tasks-and-operator) there may be supplementary specifications for the model, mainly on the output of the model. + + + + + + + +## Tasks and `operator` ## + +`operator` is one of the main parameters for both attribution methods and [metrics](../metrics/api_metrics/). It defines the function that we want to explain. *E.g.*: In the case we have a classifier model, the function that we might want to explain is the one that given a target provides us the score of the model for that specific target -- *i.e* $model(input)[target]$. + +!!!note + The `operator` parameter is a feature available for version > $1.$. The `operator` default values are the ones used before the introduction of this new feature! + + + +### Leitmotiv ### + +The `operator` parameter was introduced to offer users a flexible way to adapt current attribution methods or metrics. It should help them to empirically tackle new use-cases/new tasks. Broadly speaking, it should amplify the user's ability to experiment. However, this also implies that it is the user's responsibility to make sure that its derivations are in-scope of the original method and make sense. + + + +### `operator` in practice ### + +In practice, the user does not manipulate the function in itself. The use of the operator can be divided in three steps: + +- Specify the operator to use in the method initialization (as shown in the [API description](#api-attributions-methods)). Possible values are either an enum encoding the [task](#the-tasks-covered), a string, or a [custom operator](#providing-custom-operator). +- Make sure the model follows the model's specification relative to the selected [task](#the-tasks-covered). +- Specify what to explain in `inputs` through `targets`, the `targets` parameter specifications depend on the [task](#the-tasks-covered). + + + +### The tasks covered ### + +The `operator` parameter depends on the task to explain, as the function to explain depends on the task. In the case of Xplique, the tasks in the following table are supported natively, but new operators are welcome, please feel free to contribute. + +| Task and Documentation link | `operator` parameter valuefrom `xplique.Tasks` Enum | Tutorial link | +| :------------------------------------------------- | :---------------------------------------------------------- | :------------ | +| [Classification](../classification/) | `CLASSIFICATION` | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | +| [Object Detection](../object_detection/) | `OBJECT_DETECTION` | [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) | +| [Regression](../regression/) | `REGRESSION` | [](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) | +| [Semantic Segmentation](../semantic_segmentation/) | `SEMANTIC_SEGMENTATION` | [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) | + +!!!info + Classification is the default behavior, *i.e.*, if no `operator` value is specified or `None` is given. + +!!!warning + To apply Xplique on different tasks, specifying the value of the `operator` is not enough. Be sure to respect the ["operator in practice" steps](#operator-in-practice). + + + +### Operators' Signature ### + +An `operator` is a function that we want to explain. This function takes as input $3$ parameters: + +- the `model` to explain as in the method instantiation (specifications in the [model section](#model)). +- the `inputs` parameter representing the samples to explain as in method call (specifications in [inputs section](#inputs)). +- the `targets` parameter encoding what to explain in the `inputs` (specifications in [targets section](#targets)). + +This function should return a **vector of scalar value** of size $(N,)$ where $N$ is the number of inputs in `inputs` -- *i.e* a scalar score per input. + +!!!note + For [gradient-based methods](#gradient-based-approaches) to work with the `operator`, it needs to be differentiable with respect to `inputs`. + + + +### The operators mechanism ### + +??? info "Operators behavior for Black-box attribution methods" + + For attribution approaches that do not require gradient computation, we mostly need to query the model. Thus, those methods need an inference function. If you provide an `operator`, it will be the inference function. + + More concretely, for this kind of approach, you want to compare some valued function for an original input and perturbed version of it: + + ```python + original_scores = operator(model, original_inputs, original_targets) + + # depending on the attribution method, this `perturbation_function` is different + perturbed_inputs, perturbed_targets = perturbation_function(original_inputs, original_targets) + perturbed_scores = operator(model, perturbed_inputs, perturbed_targets) + + # example of comparison of interest + diff_scores = math.sqrt((original_scores - perturbed_scores)**2) + ``` + +??? info "Operators behavior for White-box attribution methods" + + These methods usually require some gradients computation. The gradients that will be used are the ones of the operator function (see the `get_gradient_of_operator` method in the [Providing custom operator](#providing-custom-operator) section). + + + +### Providing custom operator ### + +The `operator` parameter also supports functions (*i.e.* `Callable`), this is considered a custom operator and in this case, you should be aware of the following points: + +- An assertion will be made to ensure it respects [operators' signature](#operators-signature). +- If you use any white-box explainer, your operator will go through the `get_gradient_of_operator` function below. + +??? example "Code of the `get_gradient_of_operator` function." + + ```python + def get_gradient_of_operator(operator): + """ + Get the gradient of an operator. + + Parameters + ---------- + operator + Operator of which to compute the gradient. + + Returns + ------- + gradient + Gradient of the operator. + """ + @tf.function + def gradient(model, inputs, targets): + with tf.GradientTape() as tape: + tape.watch(inputs) + scores = operator(model, inputs, targets) + + return tape.gradient(scores, inputs) + + return gradient + ``` + +!!!tip + Writing your operator with only tensorflow functions should increase your chance that this method does not yield any errors. In addition, providing a `@tf.function` decorator is also welcome! + +!!!warning + The `targets` parameter is the key to specifying what to explain and differs greatly depending on the operator. + + + + + + + +## Models not respecting the specifications ## + +!!!warning + In any case, when you are out of the scope of the original API, you should take a deep look at the source code to be sure that your Use Case will make sense. + + + +### My inputs follow a different shape convention + +In the case where you want to handle images or time series data that does not follow the previous conventions, it is recommended to reshape the data to the expected shape for the explainers (attribution methods) to handle them correctly. Then, you can simply define a wrapper of your model so that data is reshape to your model convenience when it is called. + +For example, if you have a `model` that classifies images but want the images to be channel-first (*i.e.* with $(N, C, H, W)$ shape) then you should: + +- Move the axis so inputs are $(N, H, W, C)$ for the explainers +- Write the following wrapper for your model: + +??? example "Example of a wrapper." + + ```python + class ModelWrapper(tf.keras.models.Model): + def __init__(self, nchw_model): + super(ModelWrapper, self).__init__() + self.model = nchw_model + + def __call__(self, nhwc_inputs): + # transform the NHWC inputs (wanted for the explainers) back to NCHW inputs + nchw_inputs = self._transform_inputs(nhwc_inputs) + # make predictions + outputs = self.nchw_model(nchw_inputs) + + return outputs + + def _transform_inputs(self, nhwc_inputs): + # include in this function all transformation + # needed for your model to work with NHWC inputs + # , here for example we move axis from channels last + # to channels first + nchw_inputs = np.moveaxis(nhwc_inputs, [3, 1, 2], [1, 2, 3]) + + return nchw_inputs + + wrapped_model = ModelWrapper(model) + explainer = Saliency(wrapped_model) + # images should be (N, H, W, C) for the explain call + explanations = explainer.explain(images, labels) + ``` + +### I have a PyTorch model + +Then you should definitely take a look at the [PyTorch documentation](../pytorch/)! + +### I have a model that is neither a tf.keras.Model nor a torch.nn.Module + +Then you should take a look at the [Callable documentation](../callable/) or you could take inspiration on the [PyTorch Wrapper](../pytorch/) to write a wrapper that will integrate your model into our API! + + + + + + + +## `inputs` ## + +!!!Warning + `inputs` in this section correspond to the argument in the `explain` method of `BlackBoxExplainer`. The `model` specified at the initialization of the `BlackBoxExplainer` should be able to be called through `model(inputs)`. Otherwise, a wrapper needs to be implemented as described in the [Models not respecting the specifications section](#models-not-respecting-the-specifications). + +`inputs`: Must be one of the following: a `tf.data.Dataset` (in which case you should not provide targets), a `tf.Tensor` or a `np.ndarray`. + +- If inputs are images, the expected shape of `inputs` is $(N, H, W, C)$ following the TF's conventions where: + - $N$: the number of inputs + - $H$: the height of the images + - $W$: the width of the images + - $C$: the number of channels (works for $C=3$ or $C=1$, other values might not work or need further customization) + +- If inputs are tabular data, the expected shape of `inputs` is $(N, W)$ where: + - $N$: the number of inputs + - $W$: the feature dimension of a single input + + !!!tip + Please refer to the [table of attributions available](../../../#whats-included) to see which methods might work with Tabular Data. + +- (Experimental) If inputs are Time Series, the expected shape of `inputs` is $(N, T, W)$ + - $N$: the number of inputs + - $T$: the temporal dimension of a single input + - $W$: the feature dimension of a single input + + !!!warning + By default `Lime` & `KernelShap` will treat such inputs as grey images. You will need to define a custom `map_to_interpret_space` function when instantiating these methods in order to create a meaningful mapping of Time-Series data into an interpretable space when building such explainers. An example of this is provided at the end of the [Lime's documentation](../methods/lime/). + + !!!note + If your model is not following the same conventions, please refer to the [model not respecting the specification documentation](#models-not-respecting-the-specifications). + + + + + + + +## `targets` ## + +`targets`: Must be one of the following: a `tf.Tensor` or a `np.ndarray`. It has a shape of $(N, ...)$ where N should match the first dimension of `inputs`, while $...$ depend on the task and operators. Indeed, the `targets` parameter is highly dependent on the `operator` selected for the attribution methods, hence, for more information please refer to the [tasks and operators table](#tasks-and-operator) which will lead you to the pertinent task documentation page. diff --git a/docs/callable.md b/docs/api/attributions/callable.md similarity index 84% rename from docs/callable.md rename to docs/api/attributions/callable.md index b5820ecf..78e83bb2 100644 --- a/docs/callable.md +++ b/docs/api/attributions/callable.md @@ -8,7 +8,7 @@ The model can be something else than a `tf.keras.Model` if it respects one of th - The model is a [TF Lite model](https://www.tensorflow.org/api_docs/python/tf/lite). Note this feature is experimental. - The model is a PyTorch model (see the [dedicated documentation](../pytorch/)) -In fact, what happens when a custom `operator` is not provided (see [operator's documentation](../api/attributions/operator)) and `model` (see [model's documentation](../api/attributions/model)) is not a `tf.keras.Model`, a `tf.Module` or a `tf.keras.layers.Layer` is that the `predictions_one_hot_callable` operator is used: +In fact, what happens when a custom `operator` is not provided (see [operator's documentation](../api_attributions/#tasks-and-operator)) and `model` (see [model's documentation](../api_attributions/#model)) is not a `tf.keras.Model`, a `tf.Module` or a `tf.keras.layers.Layer` is that the `predictions_one_hot_callable` operator is used: ```python def predictions_one_hot_callable( @@ -61,4 +61,4 @@ def predictions_one_hot_callable( return scores ``` -Knowing that, you are free to wrap your model to make it work with our API and/or write a more customizable `operator`(see [operator's documentation](../api/attributions/operator))! \ No newline at end of file +Knowing that, you are free to wrap your model to make it work with our API and/or write a more customizable `operator` (see [operator's documentation](../api_attributions/#providing-custom-operator))! \ No newline at end of file diff --git a/docs/api/attributions/classification.md b/docs/api/attributions/classification.md new file mode 100644 index 00000000..d626ba20 --- /dev/null +++ b/docs/api/attributions/classification.md @@ -0,0 +1,160 @@ +# Classification explanations with Xplique + +[Attributions: Getting started tutorial](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) + + + + + +## Which kind of tasks are supported by Xplique? + +With the [operator's api](../api/attributions/operator) you can treat many different problems with Xplique. There is one operator for each task. + +| Task and Documentation link | `operator` parameter value
from `xplique.Tasks` Enum | Tutorial link | +| :------------------------------------------------- | :---------------------------------------------------------- | :------------ | +| **Classification** | `CLASSIFICATION` | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | +| [Object Detection](../object_detection/) | `OBJECT_DETECTION` | [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) | +| [Regression](../regression/) | `REGRESSION` | [](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) | +| [Semantic Segmentation](../semantic_segmentation/) | `SEMANTIC_SEGMENTATION` | [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) | + +!!!info + They all share the [API for Xplique attribution methods](../api_attributions/). + + + + + +## Simple example + +```python +import xplique +from xplique.attributions import Saliency +from xplique.metrics import Deletion + +# load inputs and model +# ... + +# for classification it is recommended to remove softmax layer if there is one +# model.layers[-1].activation = tf.keras.activations.linear + +# for classification, `targets` are the one hot encoding of the predicted class +targets = tf.one_hot(tf.argmax(model(inputs), axis=-1), depth=nb_classes, axis=-1) + +# compute explanations by specifying the classification operator +explainer = Saliency(model, operator=xplique.Tasks.CLASSIFICATION) +explanations = explainer(inputs, targets) + +# compute metrics on those explanations +# if the softmax was removed, +# it is possible to specify it to obtain more interpretable metrics +metric = Deletion(model, inputs, targets, + operator=xplique.Tasks.CLASSIFICATION, activation="softmax") +score_saliency = metric(explanations) +``` + +!!!tip + In general, if you are doing classification tasks, it is better to not include the final softmax layer in your model but to work with logits instead! + + + + + +## How to use it? + +To apply attribution methods, the [**common API documentation**](../api_attributions/) describes the parameters and how to fix them. However, depending on the task and thus on the `operator`, there are three points that vary: + +- **[The `operator` parameter](#the-operator)** value, it is an Enum or a string identifying the task, + +- **[The model's output](#models-output)** specification, as `model(inputs)` is used in the computation of the operators, and + +- **[The `targets` parameter](#the-targets-parameter)** format, indeed, the `targets` parameter specifies what to explain and the format of such specification depends on the task. + + + + + +## The `operator` ## + +### How to specify it + +In Xplique, to adapt attribution methods, you should specify the task to the `operator` parameter. In the case of classification, with either: +```python +Method(model) +# or +Method(model, operator="classification") +# or +Method(model, operator=xplique.Tasks.CLASSIFICATION) +``` + +!!!info + Classification if the default behavior of Xplique attribution methods, hence there is no need to specify it. Nonetheless, it is recommended to still do so to ensure a good comprehension of what is explained. + + + +### The computation + +The classification operator multiplies model's predictions on `inputs` with `targets` and sum it for each input to explain. However, only one value should be non-zero in `targets`, thus, the classification operator returns the model output for the specified (via `targets`) class. +```python +scores = tf.reduce_sum(model(inputs) * targets, axis=-1) +``` + + + +### The behavior + +- In the case of [perturbation-based methods](../api_attributions/#gradient-based-approaches), the perturbation score corresponds to the difference between the initial logits value for the predicted classes and the same logits for predictions over perturbed inputs. +- For [gradient-based methods](../api_attributions/#perturbation-based-approaches), the gradient of logits of interest with respect to the inputs. + +The logits of interest are specified via the `targets` parameter described in [the related section](#the-targets-parameter). + + + + + +## Model's output ## + +We expect `model(inputs)` to yield a $(n, c)$ tensor or array where $n$ is the number of input samples and $c$ is the number of classes. + + + + + +## The `targets` parameter ## + +### Role + +The `targets` parameter specifies what is to explain in the `inputs`, it is passed to the `explain` or to the `__call__` method of an explainer or metric and used by the operators. In the case of classification, it indicates the class to explain, or specifies [contrastive explanations](#contrastive-explanations). + + + +### Format + +The `targets` parameter in the case of classification should have the same shape as the [model's output](#models-output) as they are multiplied point-wise. Hence, the shape is $(n, c)$ with $n$ the number of samples to be explained (it should match the first dimension of `inputs`) and $c$ the number of classes. The `targets` parameter expects values among ${-1, 0, 1}$ but most values should be $0$ and most of the time only one should be $1$ for each sample. $-1$ are only used for [contrastive explanations](#contrastive-explanations). + + + +### In practice + +In the [simple example](#simple-example), the `targets` value provided is computed with `tf.one_hot(tf.argmax(model(inputs), axis=-1), axis=-1)`. Literally, the one hot encoding of the predicted class, this specifies which class to explain. + +!!!tip + It is better to explain the predicted class than the expected class as the goal is to explain the model's prediction. + + + + + +## What can be explained with it? ## + +### Explain the predicted class ### + +By specifying `targets` with a one hot encoding of the predicted class, the explanation will highlight which features were important for this prediction. + + + +### Contrastive explanations ### + +By specifying `targets` with zeros everywhere, `1` for the first class, and `-1` for the second class. The explanation will show which features were important to predict the first and and not the second one. + +!!!tip + If the model made a mistake, an interesting explanation is predicted class versus expected class. \ No newline at end of file diff --git a/docs/api/attributions/deconvnet.md b/docs/api/attributions/methods/deconvnet.md similarity index 100% rename from docs/api/attributions/deconvnet.md rename to docs/api/attributions/methods/deconvnet.md diff --git a/docs/api/attributions/forgrad.md b/docs/api/attributions/methods/forgrad.md similarity index 100% rename from docs/api/attributions/forgrad.md rename to docs/api/attributions/methods/forgrad.md diff --git a/docs/api/attributions/grad_cam.md b/docs/api/attributions/methods/grad_cam.md similarity index 100% rename from docs/api/attributions/grad_cam.md rename to docs/api/attributions/methods/grad_cam.md diff --git a/docs/api/attributions/grad_cam_pp.md b/docs/api/attributions/methods/grad_cam_pp.md similarity index 100% rename from docs/api/attributions/grad_cam_pp.md rename to docs/api/attributions/methods/grad_cam_pp.md diff --git a/docs/api/attributions/gradient_input.md b/docs/api/attributions/methods/gradient_input.md similarity index 100% rename from docs/api/attributions/gradient_input.md rename to docs/api/attributions/methods/gradient_input.md diff --git a/docs/api/attributions/guided_backpropagation.md b/docs/api/attributions/methods/guided_backpropagation.md similarity index 100% rename from docs/api/attributions/guided_backpropagation.md rename to docs/api/attributions/methods/guided_backpropagation.md diff --git a/docs/api/attributions/hsic.md b/docs/api/attributions/methods/hsic.md similarity index 100% rename from docs/api/attributions/hsic.md rename to docs/api/attributions/methods/hsic.md diff --git a/docs/api/attributions/integrated_gradients.md b/docs/api/attributions/methods/integrated_gradients.md similarity index 100% rename from docs/api/attributions/integrated_gradients.md rename to docs/api/attributions/methods/integrated_gradients.md diff --git a/docs/api/attributions/kernel_shap.md b/docs/api/attributions/methods/kernel_shap.md similarity index 100% rename from docs/api/attributions/kernel_shap.md rename to docs/api/attributions/methods/kernel_shap.md diff --git a/docs/api/attributions/lime.md b/docs/api/attributions/methods/lime.md similarity index 100% rename from docs/api/attributions/lime.md rename to docs/api/attributions/methods/lime.md diff --git a/docs/api/attributions/occlusion.md b/docs/api/attributions/methods/occlusion.md similarity index 100% rename from docs/api/attributions/occlusion.md rename to docs/api/attributions/methods/occlusion.md diff --git a/docs/api/attributions/rise.md b/docs/api/attributions/methods/rise.md similarity index 100% rename from docs/api/attributions/rise.md rename to docs/api/attributions/methods/rise.md diff --git a/docs/api/attributions/saliency.md b/docs/api/attributions/methods/saliency.md similarity index 100% rename from docs/api/attributions/saliency.md rename to docs/api/attributions/methods/saliency.md diff --git a/docs/api/attributions/smoothgrad.md b/docs/api/attributions/methods/smoothgrad.md similarity index 100% rename from docs/api/attributions/smoothgrad.md rename to docs/api/attributions/methods/smoothgrad.md diff --git a/docs/api/attributions/sobol.md b/docs/api/attributions/methods/sobol.md similarity index 93% rename from docs/api/attributions/sobol.md rename to docs/api/attributions/methods/sobol.md index 175f623a..e5e447c2 100644 --- a/docs/api/attributions/sobol.md +++ b/docs/api/attributions/methods/sobol.md @@ -1,92 +1,93 @@ -# Sobol Attribution Method - - -
-[View colab tutorial](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) |
-
- 
-[View source](https://github.com/deel-ai/xplique/blob/master/xplique/attributions/global_sensitivity_analysis/sobol_attribution_method.py) |
-π° [Paper](https://arxiv.org/pdf/2111.04138.pdf)
-
-The Sobol attribution method from Fel, Cadène & al.[^1] is an attribution method grounded in Sensitivity Analysis.
-Beyond modeling the individual contributions of image regions, Sobol indices provide
-efficient way to capture higher-order interactions between image regions and their
-contributions to a neural networkβs prediction through the lens of variance.
-
-!!! quote
- The total Sobol index $ST_i$ which measures the contribution
- of the variable $X_i$ as well as its interactions of any order with any other input variables to the model
- output variance.
-
- -- [Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis (2021)](https://arxiv.org/pdf/2111.04138.pdf)[^1]
-
-More precisely, the attribution score $\phi_i$ for an input variable $x_i$, is defined as
-
-$$ \phi_i = \frac{\mathbb{E}_{X \sim i}(Var_{X_i}(f(x) | X_{\sim i}))} {Var
-(f(X
-))} $$
-
-Where $\mathbb{E}_{X \sim i}(Var_{X_i}(f(x) | X_{\sim i}))$ is the expected variance
-that would be left if all variables but $X_{\sim i}$ were to be fixed.
-
-
-In order to generate stochasticity($X_i$), a perturbation function is used and uses perturbation masks
-to modulate the generated perturbation. The perturbation functions available are inpainting
-that modulates pixel regions to a baseline state, amplitude and blurring.
-
-The calculation of the indices also requires an estimator -- in practice this parameter does not
-change the results much -- `JansenEstimator` being recommended.
-
-Finally the exploration of the manifold exploration is made using a sampling method, several samplers are proposed: Quasi-Monte
-Carlo (`ScipySobolSequence`, recommended) using Scipy's sobol sequence, Latin hypercubes
- -- `LHSAmpler` -- or Halton's sequences `HaltonSequence`.
-
-
-!!!tip
- For a quick a faithful explanations, we recommend to use `grid_size` in $[7, 12)$,
- `nb_design` in $\{16, 32, 64\}$ (more is useless), and a QMC sampler.
-
-## Example
-
-```python
-from xplique.attributions import SobolAttributionMethod
-from xplique.attributions.global_sensitivity_analysis import (
- JansenEstimator, GlenEstimator,
- LHSampler, ScipySobolSequence,
- HaltonSequence)
-
-# load images, labels and model
-# ...
-
-# default explainer (recommended)
-explainer = SobolAttributionMethod(model, grid_size=8, nb_design=32)
-explanations = method(images, labels) # one-hot encoded labels
-```
-
-If you want to change the estimator or the sampling:
-
-```python
-from xplique.attributions import SobolAttributionMethod
-from xplique.attributions.global_sensitivity_analysis import (
- JansenEstimator, GlenEstimator,
- LHSampler, ScipySobolSequence,
- HaltonSequence)
-
-# load images, labels and model
-# ...
-
-explainer_lhs = SobolAttributionMethod(model, grid_size=8, nb_design=32,
- sampler=LHSampler(),
- estimator=GlenEstimator())
-explanations_lhs = explainer_lhs(images, labels)
-```
-
-## Notebooks
-
-- [**Attribution Methods**: Getting started](https://colab.research.google.com/drive
-/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2)
-
-
-{{xplique.attributions.global_sensitivity_analysis.sobol_attribution_method.SobolAttributionMethod}}
-
-[^1]:[Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis (2021)](https://arxiv.org/pdf/2111.04138.pdf)
+# Sobol Attribution Method
+
+
+
+[View colab tutorial](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) |
+
+
+[View source](https://github.com/deel-ai/xplique/blob/master/xplique/attributions/global_sensitivity_analysis/sobol_attribution_method.py) |
+π° [Paper](https://arxiv.org/pdf/2111.04138.pdf)
+
+The Sobol attribution method from Fel, Cadène & al.[^1] is an attribution method grounded in Sensitivity Analysis.
+Beyond modeling the individual contributions of image regions, Sobol indices provide
+an efficient way to capture higher-order interactions between image regions and their
+contributions to a neural networkβs prediction through the lens of variance.
+
+!!! quote
+ The total Sobol index $ST_i$ which measures the contribution
+ of the variable $X_i$ as well as its interactions of any order with any other input variables to the model
+ output variance.
+
+ -- [Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis (2021)](https://arxiv.org/pdf/2111.04138.pdf)[^1]
+
+More precisely, the attribution score $\phi_i$ for an input variable $x_i$, is defined as
+
+$$ \phi_i = \frac{\mathbb{E}_{X \sim i}(Var_{X_i}(f(x) | X_{\sim i}))} {Var
+(f(X
+))} $$
+
+Where $\mathbb{E}_{X \sim i}(Var_{X_i}(f(x) | X_{\sim i}))$ is the expected variance
+that would be left if all variables but $X_{\sim i}$ were to be fixed.
+
+
+In order to generate stochasticity($X_i$), a perturbation function is used and uses perturbation masks
+to modulate the generated perturbation. The perturbation functions available are inpainting
+that modulates pixel regions to a baseline state, amplitude and blurring.
+
+The calculation of the indices also requires an estimator -- in practice this parameter does not
+change the results much -- `JansenEstimator` being recommended.
+
+Finally the exploration of the manifold exploration is made using a sampling method, several samplers are proposed: Quasi-Monte
+Carlo (`ScipySobolSequence`, recommended) using Scipy's sobol sequence, Latin hypercubes
+ -- `LHSAmpler` -- or Halton's sequences `HaltonSequence`.
+
+
+!!!tip
+ For quick a faithful explanations, we recommend to use `grid_size` in $[7, 12)$,
+ `nb_design` in $\{16, 32, 64\}$ (more is useless), and a QMC sampler.
+ (see `SobolAttributionMethod` documentation below for detail on those parameters).
+
+## Example
+
+```python
+from xplique.attributions import SobolAttributionMethod
+from xplique.attributions.global_sensitivity_analysis import (
+ JansenEstimator, GlenEstimator,
+ LHSampler, ScipySobolSequence,
+ HaltonSequence)
+
+# load images, labels and model
+# ...
+
+# default explainer (recommended)
+explainer = SobolAttributionMethod(model, grid_size=8, nb_design=32)
+explanations = method(images, labels) # one-hot encoded labels
+```
+
+If you want to change the estimator or the sampling:
+
+```python
+from xplique.attributions import SobolAttributionMethod
+from xplique.attributions.global_sensitivity_analysis import (
+ JansenEstimator, GlenEstimator,
+ LHSampler, ScipySobolSequence,
+ HaltonSequence)
+
+# load images, labels and model
+# ...
+
+explainer_lhs = SobolAttributionMethod(model, grid_size=8, nb_design=32,
+ sampler=LHSampler(),
+ estimator=GlenEstimator())
+explanations_lhs = explainer_lhs(images, labels)
+```
+
+## Notebooks
+
+- [**Attribution Methods**: Getting started](https://colab.research.google.com/drive
+/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2)
+
+
+{{xplique.attributions.global_sensitivity_analysis.sobol_attribution_method.SobolAttributionMethod}}
+
+[^1]:[Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis (2021)](https://arxiv.org/pdf/2111.04138.pdf)
diff --git a/docs/api/attributions/square_grad.md b/docs/api/attributions/methods/square_grad.md
similarity index 100%
rename from docs/api/attributions/square_grad.md
rename to docs/api/attributions/methods/square_grad.md
diff --git a/docs/api/attributions/vargrad.md b/docs/api/attributions/methods/vargrad.md
similarity index 100%
rename from docs/api/attributions/vargrad.md
rename to docs/api/attributions/methods/vargrad.md
diff --git a/docs/api/metrics/api_metrics.md b/docs/api/attributions/metrics/api_metrics.md
similarity index 60%
rename from docs/api/metrics/api_metrics.md
rename to docs/api/attributions/metrics/api_metrics.md
index d43a9c2d..a1ecae3a 100644
--- a/docs/api/metrics/api_metrics.md
+++ b/docs/api/attributions/metrics/api_metrics.md
@@ -2,6 +2,10 @@
- [**Attribution Methods**: Metrics](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) [](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg)
+
+
+
+
## Context
As the XAI field continues on being trendy, the quantity of materials at disposal to explain DL models keeps on growing. Especially, there is an increasing need to benchmark and evaluate those different approaches. Mainly, there is an urge to evaluate the quality of explanations provided by attribution methods.
@@ -9,32 +13,33 @@ As the XAI field continues on being trendy, the quantity of materials at disposa
!!!info
Note that, even though some work exists for other tasks, this challenge has been mainly tackled in the context of Computer Vision tasks.
-As pointed out by [Petsiuk et al.](http://arxiv.org/abs/1806.07421) most explanations approaches are used to be evaluated in a human-centred way. For instance, an attribution method was considered as good if it pointed out the same relevant pixels as the ones highlighted by human users. While this kind of evaluation allows giving some user trust it can easily be biased. Therefore, the authors introduced two automatic evaluation metrics that rely solely on the drop or rise in the probability of a class as important pixels (defined by the saliency map) are removed or added. Those are not the only available metrics and we propose here to present the API we used as common ground and then to dive into more specifity.
+As pointed out by [Petsiuk et al.](http://arxiv.org/abs/1806.07421), most explainability approaches used to be evaluated in a human-centered way. For instance, an attribution method was considered good if it pointed at the same relevant pixels as the ones highlighted by human users. While this kind of evaluation allows giving some users trust, it can easily be biased. Therefore, the authors introduced two automatic evaluation metrics that rely solely on the drop or rise in the probability of a class as important pixels (defined by the saliency map) are removed or added. Those are not the only available metrics and we propose here to present the API we used as common ground.
-## Common API
-All metrics inherits from the base class `BaseAttributionMetric` which has the following `__init__` arguments:
-- `model`: The model from which we want to obtain explanations
-- `inputs`: Input samples to be explained
- !!!info
- Inputs should be the same as defined in the [model's documentation](../../attributions/model)
-- `targets`: Specify the kind of explanations we want depending on the task at end (e.g. a one-hot encoding of a class of interest, a difference to a ground-truth value..)
+## Common API
- !!!info
- Targets should be the same as defined in the [model's documentation](../../attributions/model)
+!!!info
+ Metrics described on this page are metrics for attribution methods and explanations. Therefore, the user should first get familiar with the [attributions methods API](../../api_attributions/) as many parameters are common between both API. For instance, `model`, `inputs`, `targets`, and `operator` should match for methods and their metrics.
-- `batch_size`
+All metrics inherit from the base class `BaseAttributionMetric` which has the following `__init__` arguments:
-- `activation`: A string that belongs to [None, 'sigmoid', 'softmax']. See the [dedicated section](#activation) for details
+- `model`: The model from which we want to obtain explanations
+- `inputs`: Input samples to be explained
+- `targets`: Specify the kind of explanations we want depending on the task at end (e.g. a one-hot encoding of a class of interest, a difference to a ground-truth value..)
+- `batch_size`: an integer which allows to either process inputs per batch or process perturbed samples of an input per batch (inputs are therefore processed one by one). It is most of the time overwritten by the explanation method `batch_size`.
+- `activation`: A string that belongs to [None, 'sigmoid', 'softmax']. See the [dedicated section](#activation) for details
Then we can distinguish two category of metrics:
-- Those which only need the attribution ouputs of an explainer: `ExplanationMetric`, namely those which evaluate Fidelity ([MuFidelity](../mu_fidelity), [Deletion](../deletion), [Insertion](../insertion))
+- Those which only need the attribution outputs of an explainer: `ExplanationMetric`, namely those which evaluate Fidelity ([MuFidelity](../mu_fidelity), [Deletion](../deletion), [Insertion](../insertion))
- Those which need the explainer: `ExplainerMetric` ([AverageStability](../avg_stability))
+
+
+
### `ExplanationMetric`
Those metrics are agnostic of the explainer used and rely only on the attributions mappings it gives.
@@ -44,7 +49,7 @@ Those metrics are agnostic of the explainer used and rely only on the attributio
All metrics inheriting from this class have another argument in their `__init__` method:
-- `operator`: Optionnal function wrapping the model. It can be seen as a metric which allow to evaluate model evolution. For more details, see the attribution's [API Description](../../attributions/api_attributions/) and the [operator documentation](../../attributions/operator/).
+- `operator`: Optional function wrapping the model. It can be seen as a metric which allows to evaluate model evolution. For more details, see the attribution's [API Description section on `operator`](../../api_attributions/#tasks-and-operator).
!!!info
The `operator` used here should match the one used to compute the explanations!
@@ -52,20 +57,26 @@ All metrics inheriting from this class have another argument in their `__init__`
All metrics inheriting from this class have to define a method `evaluate` which will take as input the `attributions` given by an explainer. Those attributions should correspond to the `model`, `inputs` and `targets` used to build the metric object.
+
### `ExplainerMetric`
-Those metrics will not assess the quality of the explanations provided but (also) the explainer itself.
+These metrics will not assess the quality of the explanations provided but (also) the explainer itself.
All metrics inheriting from this class have to define a method `evaluate` which will take as input the `explainer` evaluated.
!!!info
- It is even more important that `inputs` and `targets` are the same as defined in the attribution's [API Description](../../attributions/api_attributions/)
+ It is even more important that `inputs` and `targets` be the same as defined in the attribution's [API Description](../../api_attributions/#inputs).
Currently, there is only one Stability metric inheriting from this class:
+
+
+
+
+
## Activation
-This parameter specify if an additional activation layer should be added once a model has been called on the inputs when you have to compute the metric.
+This parameter specifies if an additional activation layer should be added once a model has been called on the inputs when you have to compute the metric.
Indeed, most of the times it is recommended when you instantiate an **explainer** (*i.e.* an attribution methods) to provide a model which gives logits as explaining the logits is to explain the class, while explaining the softmax is to explain why this class rather than another.
@@ -78,12 +89,20 @@ The default behavior is to compute the metric without adding any activation laye
!!!note
There does not appear to be a consensus on the activation function to be used for metrics. Some papers use logits values (e.g., with mu-fidelity), while others use sigmoid or softmax (with deletion and insertion). We can only observe that changing the activation function has an effect on the ranking of the best methods.
+
+
+
+
## Other Metrics
A Representatibity metric: [MeGe](https://arxiv.org/abs/2009.04521) is also available. Documentation about it should be added soon.
+
+
+
+
## Notebooks
- [**Metrics**: Getting started](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) [](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg)
-- [**Metrics**: With Pytorch's model](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) [](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP)
+- [**Metrics**: With PyTorch models](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) [](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP)
diff --git a/docs/api/metrics/avg_stability.md b/docs/api/attributions/metrics/avg_stability.md
similarity index 100%
rename from docs/api/metrics/avg_stability.md
rename to docs/api/attributions/metrics/avg_stability.md
diff --git a/docs/api/metrics/deletion.md b/docs/api/attributions/metrics/deletion.md
similarity index 100%
rename from docs/api/metrics/deletion.md
rename to docs/api/attributions/metrics/deletion.md
diff --git a/docs/api/metrics/insertion.md b/docs/api/attributions/metrics/insertion.md
similarity index 100%
rename from docs/api/metrics/insertion.md
rename to docs/api/attributions/metrics/insertion.md
diff --git a/docs/api/metrics/mu_fidelity.md b/docs/api/attributions/metrics/mu_fidelity.md
similarity index 100%
rename from docs/api/metrics/mu_fidelity.md
rename to docs/api/attributions/metrics/mu_fidelity.md
diff --git a/docs/api/attributions/model.md b/docs/api/attributions/model.md
deleted file mode 100644
index e984e2ba..00000000
--- a/docs/api/attributions/model.md
+++ /dev/null
@@ -1,212 +0,0 @@
-# Model expectations: What should be the model provided ?
-
-Even though we tried to cover a wide-range of models for the XAI methods to work we based our frameworks on some assumptions which we propose to see here.
-
-## The inputs have expected shape
-
-As a reminder any attribution methods are instanciated with at least three parameters:
-
-- `model`: the model from which we want to obtain attributions (e.g: InceptionV3, ResNet, ...)
-- `batch_size`: an integer which allows to either process inputs per batch (gradient-based methods) or process perturbed samples of an input per batch (inputs are therefore process one by one)
-- `operator`: function g to explain, see the [Operator documentation](../operator/) for more details
-
-And an explainer is called with the `explain` method that takes as parameters:
-
-- `inputs`: One of the following: a `tf.data.Dataset` (in which case you should not provide `targets`), a `tf.Tensor` or a `np.ndarray`
-
-- `targets`: One of the following: a `tf.Tensor` or a `np.ndarray`
-
-!!!info
- `targets`' format is defined depending on the task you are doing. See the [Tasks section](#tasks).
-
-
-### General
-
-In practice, we expect the `model` to be callable for the `inputs` parameters -- *i.e.* we can do `model(inputs)`. We expect this call to produce the `outputs` variables that is the predictions of the model on those inputs. As for most attribution methods we need to manipulate and/or link the `outputs` to the `inputs` we assume that the latter have conventional shape described in the sections below.
-
-### Inputs Format
-
-#### Images data
-
-If inputs are images, the expected shape of `inputs` is $(N, H, W, C)$ following the TF's conventions where:
-
-- $N$ is the number of inputs
-- $H$ is the height of the images
-- $W$ is the width of the images
-- $C$ is the number of channels (works for $C=3$ or $C=1$, other values might not work or need further customization)
-
-In the case where `inputs` is a `tf.data.Dataset` with images, then we expect each sample of the dataset to be a tuple `(image, target)` with `image` having $(H, W, C)$ shape and target being defined as explained in the [Tasks section](#tasks)
-
-!!!warning
- If your model is not following the same conventions it might lead to poor results or yield errors.
-
-### Tabular data
-
-If inputs are tabular data, the expected shape of `inputs` is $(N, W)$ where:
-
-- $N$ is the number of inputs
-- $W$ is the feature dimension of a single input
-
-In the case where `inputs` is a `tf.data.Dataset` with tabular data, then we expect each sample of the dataset to be a tuple `(features, target)` with `features` having $W$ shape and target being a one-hot encoding of the output you want an explanation of.
-
-!!!info
- All attribution methods does not work well with tabular data.
-
-!!!tip
- Please refer to the [table](../../../#whats-included) to see which methods might work with Tabular Data
-
-### Time-Series data
-
-If inputs are Time Series, the expected shape of `inputs` is $(N, T, W)$
-
-- $N$ is the number of inputs
-- $T$ is the temporal dimension of a single input
-- $W$ is the feature dimension of a single input
-
-!!!warning
- By default `Lime` & `KernelShap` will treat such inputs as grey images. You will need to define a custom `map_to_interpret_space` function when instantiating those methods in order to create a meaningful mapping of Time-Series data into an interpretable space when building such explainers. Such an example is provided at the end of the [Lime's documentation](../lime/).
-
-## Tasks
-
-### Classification Tasks
-
-!!!tip
- In general, if you are doing classification tasks it is better to not include the final softmax layer in your model but to work with logits instead!
-
-For classification tasks, it is expected for the user to use the `predictions_operator` (see the [Operator documentation](../operator/)) when initializing an explainer:
-
-```python
-@tf.function
-def predictions_operator(model: Callable,
- inputs: tf.Tensor,
- targets: tf.Tensor) -> tf.Tensor:
- """
- Compute predictions scores, only for the label class, for a batch of samples.
-
- Parameters
- ----------
- model
- Model used for computing predictions.
- inputs
- Input samples to be explained.
- targets
- One-hot encoded labels or regression target (e.g {+1, -1}), one for each sample.
-
- Returns
- -------
- scores
- Predictions scores computed, only for the label class.
- """
- scores = tf.reduce_sum(model(inputs) * targets, axis=-1)
- return scores
-```
-
-#### `model(inputs)`
-Consequently, we expect `model(inputs)` to yield a $(N, C)$ tensor or array where $N$ is the number of input samples and $C$ is the number of classes.
-
-#### `targets`
-If you use the default operator for classification task we expect `targets` to be a $(N, C)$ tensor or array which is a one-hot encoding of **the class you want to explain** where $N$ is the number of input samples and $C$ is the number of classes.
-
-### Regression Tasks
-
-If the task at end is regression, then the user should instantiate the explainer with the `regression_operator` (see the [Operator documentation](../operator/)):
-
-```python
-@tf.function
-def regression_operator(model: Callable,
- inputs: tf.Tensor,
- targets: tf.Tensor) -> tf.Tensor:
- """
- Compute the the mean absolute error between model prediction and the target.
- Target should the model prediction on non-perturbed input.
- This operator can be used to compute attributions for all outputs of a regression model.
-
- Parameters
- ----------
- model
- Model used for computing predictions.
- inputs
- Input samples to be explained.
- targets
- Model prediction on non-perturbed inputs.
-
- Returns
- -------
- scores
- MAE between model prediction and targets.
- """
- scores = tf.reduce_mean(tf.abs(model(inputs) - targets), axis=-1)
- return scores
-```
-
-#### `model(inputs)`
-Consequently, we expect `model(inputs)` to yield a $(N, D)$ tensor or array where $N$ is the number of input samples and $D$ is the number of variables the model should predict (possibly one).
-
-#### `targets`
-If you are using the `regression_operator`, it is expected that `targets` to be a $(N, D)$ tensor or array of the expected multi-variate output where $N$ is the number of input samples and $D$ is the number of variables (possibly one).
-
-### Object-Detection Tasks
-
-**Work In Progress**
-
-### Segmentation Tasks
-
-**Work In Progress**
-
-## What if my inputs and/or targets and/or my model does not follow the previous assumptions ?
-
-!!!warning
- In any case, when you are out of the scope of the original API, you should take a deep look at the source code to be sure that your Use Case will make sense.
-
-### My inputs follow a different shape convention
-In the case where you want to handle images or time series data that does not follow the previous conventions, it is recommended to reshape the data to the expected shape for the explainers (attribution methods) to handle them correctly. Then, you can simply define a wrapper of your model so that data is reshape to your model convenience when it is called.
-
-For example, if you have a `model` that classifies images but want the images to be channel-first (*i.e.* with $(N, C, H, W)$ shape) then you should:
-
-- Move the axis so inputs are $(N, H, W, C)$ for the explainers
-- Write the following wrapper for your model:
-
-```python
-class ModelWrapper(tf.keras.models.Model):
- def __init__(self, nchw_model):
- super(ModelWrapper, self).__init__()
- self.model = nchw_model
-
- def __call__(self, nhwc_inputs):
- # transform the NHWC inputs (wanted for the explainers) back to NCHW inputs
- nchw_inputs = self._transform_inputs(nhwc_inputs)
- # make predictions
- outputs = self.nchw_model(nchw_inputs)
-
- return outputs
-
- def _transform_inputs(self, nhwc_inputs):
- # include in this function all transformation
- # needed for your model to work with NHWC inputs
- # , here for example we moveaxis from channels last
- # to channels first
- nchw_inputs = np.moveaxis(nhwc_inputs, [3, 1, 2], [1, 2, 3])
-
- return nchw_inputs
-
-wrapped_model = ModelWrapper(model)
-explainer = Saliency(wrapped_model)
-# images should be (N, H, W, C) for the explain call
-explanations = explainer.explain(images, labels)
-```
-
-### My inputs are a dictionnary (ex: Attention Model)
-
-**Work In Progress**
-
-### My model is neither for classification nor regression tasks
-
-Then you could define your own operator! (see the [Operator documentation](../operator/))
-
-### I have a PyTorch model
-
-Then you should definetely have a look on the [dedicated documentation](../../../pytorch/)!
-
-### I have a model that is neither a tf.keras.Model nor a torch.nn.Module
-
-Then you should take a look on the [Callable documentation](../../../callable/) or you could take inspiration on the [PyTorch Wrapper](../../../pytorch/) to write a wrapper that will integrate your model into our API!
diff --git a/docs/api/attributions/object_detection.md b/docs/api/attributions/object_detection.md
new file mode 100644
index 00000000..8ce0aa7c
--- /dev/null
+++ b/docs/api/attributions/object_detection.md
@@ -0,0 +1,244 @@
+# Object detection with Xplique
+
+[Attributions: Object Detection tutorial](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL)
+
+
+
+
+
+## Which kind of tasks are supported by Xplique?
+
+With the [operator's api](../api/attributions/operator) you can treat many different problems with Xplique. There is one operator for each task.
+
+| Task and Documentation link | `operator` parameter value from `xplique.Tasks` Enum | Tutorial link | +| :------------------------------------------------- | :---------------------------------------------------------- | :------------ | +| [Classification](../classification/) | `CLASSIFICATION` | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | +| **Object Detection** | `OBJECT_DETECTION` | [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) | +| [Regression](../regression/) | `REGRESSION` | [](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) | +| [Semantic Segmentation](../semantic_segmentation/) | `SEMANTIC_SEGMENTATION` | [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) | + +!!!info + They all share the [API for Xplique attribution methods](../api_attributions/). + + + + + +## Simple example + +```python +import xplique +from xplique.attributions import Saliency +from xplique.metrics import Deletion + +# load images and model +# ... + +predictions = model(images) +explainer = Saliency(model, operator=xplique.Tasks.OBJECT_DETECTION) + +# explain each image - bounding-box pair separately +for all_bbx_for_one_image, image in zip(predictions, images): + # an image is needed per bounding box, so we tile them + repeated_image = tf.tile(tf.expand_dims(image, axis=0), + (tf.shape(all_bbx_for_one_image)[0], 1, 1, 1)) + + explanations = explainer(repeated_image, all_bbx_for_one_image) + + # either compute several score or + # concatenate repeated images and corresponding boxes in one tensor + metric_for_one_image = Deletion(model, repeated_image, all_bbx_for_one_image, + operator=xplique.Tasks.OBJECT_DETECTION) + score_saliency = metric(explanations) +``` + + + + + +## How to use it? + +To apply attribution methods, the [**common API documentation**](../api_attributions/) describes the parameters and how to fix them. However, depending on the task and thus on the `operator`, there are three points that vary: + +- **[The `operator` parameter](#the-operator)** value, it is an Enum or a string identifying the task, + +- **[The model's output](#models-output)** specification, as `model(inputs)` is used in the computation of the operators, and + +- **[The `targets` parameter](#the-targets-parameter)** format, indeed, the `targets` parameter specifies what to explain and the format of such specification depends on the task. + + + + + +## The `operator` ## + +### How to specify it + +In Xplique, to adapt attribution methods, you should specify the task to the `operator` parameter. In the case of object detection, with either: +```python +Method(model, operator="object detection") +# or +Method(model, operator=xplique.Tasks.OBJECT_DETECTION) +``` + +!!!info + There are several [variants of the object detection operator](#the-different-operators-variant-and-what-they-explain) to explain part of the prediction. + + + +### The computation + +This operator is a generalization of DRise method introduced by Petsiuk & al. [^1] to most attribution methods. The computation is the same as the one described in the DRise paper. The DRise can be divided into two principles: + +- **The matching**: DRise extends Rise (described in detail in [the Rise tutorial](https://colab.research.google.com/drive/1icu2b1JGfpTRa-ic8tBSXnqqfuCGW2mO)) to explain object detection. Rise is a perturbation-based method, hence current predictions are compared to predictions on perturbed inputs. However, object detectors predict several boxes with no consistency in the order, thus DRise chooses to match the current bounding box to the most similar one and use the similarity metric as the perturbation score. +- **The similarity metric**: This is the score used by DRise to match bounding boxes. It uses the three parts of a bounding box prediction, the position of the box, the box objectness, and the associated class. A score is computed for each of those three parts and these scores are multiplied: + +$$ +score = intersection\_score * detection\_probability * classification\_score +$$ + +With: +$$ +intersection\_score = IOU(coordinates_{ref}, coordinates_{pred}) +$$ + +$$ +detection\_probability = objectness_{pred} +$$ + +$$ +classification\_score = \frac{\sum(classes_{ref} * classes_{pred})}{||classes_{ref}|| * ||classes_{pred}||} +$$ + +!!!info + The intersection score of the operator is the IOU (Intersection Over Union) by default but can be modified by specifying as [custom intersection score](#custom-intersection-score). + +!!!info + With the DRise formula the methods explain the box position, the box objectness, and the class prediction at the same time. However, the user may want to explain them separately, therefore several variants of this operator are available in Xplique and described in [What can we explain and how? section](#what-can-we-explain-and-how). + + + +### The behavior + +- In the case of [perturbation-based methods](../api_attributions/#gradient-based-approaches), the perturbation score is the similarity metric aforementioned. +- For [gradient-based methods](../api_attributions/#perturbation-based-approaches), the gradient of the similarity metric is given, but no matching is necessary as no perturbation is made. + + + + + +## Model's output ## + +We expect `model(inputs)` to yield a $(n, nb\_boxes, 4 + 1 + nb\_classes)$ tensors or array where: + +- $n$: the number of inputs, it should match the first dimension of `inputs`. +- $nb\_boxes$: a fixed number of bounding boxes predicted for a given image (no NMS). +- $(4 + 1 + nb\_classes)$: the encoding of a bounding box prediction +- $4$: the bounding box coordinates $(x_{top\_left}, y_{top\_left}, x_{bottom\_right}, y_{bottom\_right})$, with $x_{top\_left} < x_{bottom\_right}$ and $y_{top\_left} < y_{bottom\_right}$. +- $1$: the objectness or detection probability of the bounding box, +- $nb\_classes$: the class of the bounding box, a soft class predictions not a one-hot encoding. + +!!!warning + Object detection models provided to the explainer should not include NMS and classification should be soft classification not one-hot encoding. Furthermore, if the model does not match the expected format, a wrapper may be needed. (see [the tutorial](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) for an example). + +!!!info + PyTorch models are not natively treated by Xplique, however, a simple wrapper is available in [pytorch documentation](../pytorch/). + + + + + +## The `targets` parameter ## + +### Role + +The `targets` parameter specifies what is to explain in the `inputs`, it is passed to the `explain` or to the `__call__` method of an explainer or metric and used by the operators. In the case of object detection, it indicates which box to explain, furthermore, it gives the initial predictions to the operator as the reference for perturbation-based methods. + + + +### Format + +The `targets` parameter in the case of semantic segmentation should have the same shape as the [model's output](#models-output) as the same computation are made. Concretely, the `targets` parameter should have a shape of $(n, 4 + 1 + nb\_classes)$ to explain a bounding box for each input (detail in [model's output description](#models-output)). + +Additionally, there is a possibility to explain a group of bounding boxes at the same time described in the [explaining several bounding boxes section](#explain-several-bounding-boxes-simultaneously) which requires a different shape. + + + +### In practice + +To explain each bounding box individually, the images need to be repeated. Indeed, object detector predict several bounding boxes per image and the first dimension of `inputs` and `targets` should match as it corresponds to the sample dimension. Therefore, the easiest way to obtain this is for each image to repeat it so that it matches the number of bounding boxes to explain for this image. + +In the [simple example](#simple-example), there is a loop on the images - predictions pair, then images are repeated to match the number of predicted bounding boxes, and finally, the `targets` parameter takes the predicted bounding boxes. + +!!!tip + AS specified in the [model's output specification](#models-output), the NMS (Non Maximum Suppression) should not be included in the model. However, it can be used to select the bounding boxes to explain. + +!!!warning + Repeating images may create a tensor that exceeds memory for large images and/or when many bounding boxes are to be explained. In this case, we advise to make a loop on the images, then a loop on the boxes. + + + +### Explain several bounding boxes simultaneously + +The user may not want to explain each bounding box individually but several bounding boxes at the same time (*i.e* a set of pedestrian bounding boxes on a sidewalk). In this case, the `targets` parameter shape will not be $(n, 4 + 1 + nb\_classes)$ but $(n, nb\_boxes, 4 + 1 + nb\_classes)$, with $nb\_boxes$ the number of boxes to explain simultaneously. In this case, $nb\_boxes$ bounding boxes are associated to each sample and a single attribution map is returned. However, for different images, $nb\_boxes$ may not be fix and it may not be possible to make a single tensor in this case. Thus, we recommend to treat each group of bounding boxes with a different call to the attribution method with $n=1$. + +To return one explanation for several bounding boxes, Xplique takes the mean of the bounding boxes individual explanations and returns it. + +For a concrete example, please refer to the [Attributions: Object detection](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) tutorial. + + + + + +## What can be explained and how? + +### The different elements in object detection + +In object detection, the prediction for a given bounding box include several pieces of information: The **box position**, the **box probability of containing something**, and the **class of the detected object**. Therefore we may want to explain each of them separately, however, the DRise method of matching bounding boxes should be kept in mind. Indeed, the box position cannot be removed from the score, otherwise, the explanation may not correspond to the same object. + + + +### The different operator's variants and what they explain + +The Xplique library allows the specification of which part of the prediction to explain via a set 4 operators: the one as defined by the DRise formula and three variants: + +- `"object detection"`: the one described in [the operator section](#the-operator): + + $$score = intersection\_score * detection\_probability * classification\_score$$ + +- `"object detection box position"`: explains only the bounding box position: + + $$score = intersection\_score$$ + +- `"object detection box proba"`: explains the probability of a bounding box to contain something: + + $$score = intersection\_score * detection\_probability$$ + +- `"object detection box class"`: explains the class of a bounding box: + + $$score = intersection\_score * classification\_score$$ + + + + + +## Custom intersection score + +The default intersection score is IOU, but it is possible to define a custom intersection score. The only constraint is that it should follow `xplique.commons.object_detection_operator._box_iou` signature for it to work. + +```python +from xplique.attributions import Saliency +from xplique.commons.operators import object_detection_operator + +custom_intersection_score = ... + +custom_operator = lambda model, inputs, targets: object_detection_operator( + model, inputs, targets, intersection_score=custom_intersection_score +) + +explainer = Saliency(model, operator=custom_operator) + +... # All following steps are the same as the examples +``` + +[^1] [Black-box Explanation of Object Detectors via Saliency Maps (2021)](https://arxiv.org/pdf/2006.03204.pdf) \ No newline at end of file diff --git a/docs/api/attributions/object_detector.md b/docs/api/attributions/object_detector.md deleted file mode 100644 index e488ab11..00000000 --- a/docs/api/attributions/object_detector.md +++ /dev/null @@ -1 +0,0 @@ -### Work In Progress diff --git a/docs/api/attributions/operator.md b/docs/api/attributions/operator.md deleted file mode 100644 index 12b04c8e..00000000 --- a/docs/api/attributions/operator.md +++ /dev/null @@ -1,250 +0,0 @@ -# Operator - -`operator` is one of the main parameters for both attribution methods and metrics. It defines the function $g$ that we want to explain. *E.g.*: In the case we have a classifier model the function that we might want to explain is the one that given a target gives us the score of the model for that specific target -- *i.e* $model(input)[target]$. - -!!!note - The `operator` parameter is a feature avaible for version > $1.$. The `operator` default values are the ones used before the introduction of this new feature! - -## Leitmotiv - -The `operator` parameter was introduced to offer users a flexible way to adapt current attribution methods or metrics. It should help them to empirically tackle new use-cases/new tasks. Broadly speaking, it should amplify the user's ability to experiment. However, this also imply that it is the user responsability to make sure that its derivations are in-scope of the original method and make sense. - -## Operators' Signature - -An `operator` is a function $g$ that we want to explain. This function take as input $3$ parameters: - -- `model`, the model under investigation -- `inputs`: One of the following: a `tf.data.Dataset` (in which case you should not provide `targets`), a `tf.Tensor` or a `np.ndarray` -- `targets`: One of the following: a `tf.Tensor` or a `np.ndarray` - -!!!info - More specification concerning `model` or `inputs` can be found in the [model's documentation](../model/). More information on `targets` can be found [here](#tasks) or also in the [model's documentation](../model/#tasks) - -This function $g$ should return a **vector of scalar value** of size $(N,)$ where $N$ is the number of input in `inputs` -- *i.e* a scalar score per input. - -## How is the operator used in Xplique ? - -### Black-box attribution methods - -For attribution approaches that do not require gradient computation we mostly need to query the model. Thus, those methods need an inference function. If you provide an `operator`, it will be the inference function. - -More concretely, for this kind of approach you want to compare some valued function for an original input and perturbed version of it: - -```python -original_scores = operator(model, original_inputs, original_targets) - -# depending on the attribution method this `perturbation_function` is different -perturbed_inputs, perturbed_targets = perturbation_function(original_inputs, original_targets) -perturbed_scores = operator(model, perturbed_inputs, perturbed_targets) - -# exemple of comparison of interest -diff_scores = math.sqrt((original_scores - perturbed_scores)**2) -``` - -### White-box attribution methods - -Those methods usually require some gradients computation. The gradients that will be used are the one of the operator function (see the `get_gradient_of_operator` method in the [Providing custom operator](#providing-custom-operator) section). - -## Default Behavior - -A lot of attribution methods are initially intended for classification tasks. Thus, the default operator `predictions_operator` assume such a setting - -```python -@tf.function -def predictions_operator(model: Callable, - inputs: tf.Tensor, - targets: tf.Tensor) -> tf.Tensor: - """ - Compute predictions scores, only for the label class, for a batch of samples. - - Parameters - ---------- - model - Model used for computing predictions. - inputs - Input samples to be explained. - targets - One-hot encoded labels, one for each sample. - - Returns - ------- - scores - Predictions scores computed, only for the label class. - """ - scores = tf.reduce_sum(model(inputs) * targets, axis=-1) - return scores -``` - -That is a setting where the variable `model(inputs)` is a vector of size $(N, C)$ where: $N$ is the number of input and $C$ is the number of class. - -!!!info - Explaining the logits is to explain the class, while explaining the softmax is to explain why this class is more likely. Thus, it is recommended to explain the logit and exclude the softmax layer if any. - -## Existing operators and how to use them - -At present, there are at present 2 operators available (and 2 others should be released soon) in the library that tackle different tasks. - -### Tasks - -#### Classification Tasks - -!!!tip - In general, if you are doing classification tasks it is better to not include the final softmax layer in your model but to work with logits instead! - -For classification tasks, it is expected for the user to use the `predictions_operator` when initializing an explainer: - -```python -@tf.function -def predictions_operator(model: Callable, - inputs: tf.Tensor, - targets: tf.Tensor) -> tf.Tensor: - """ - Compute predictions scores, only for the label class, for a batch of samples. - - Parameters - ---------- - model - Model used for computing predictions. - inputs - Input samples to be explained. - targets - One-hot encoded labels or regression target (e.g {+1, -1}), one for each sample. - - Returns - ------- - scores - Predictions scores computed, only for the label class. - """ - scores = tf.reduce_sum(model(inputs) * targets, axis=-1) - return scores -``` - -- `model(inputs)` -Consequently, we expect `model(inputs)` to yield a $(N, C)$ tensor or array where $N$ is the number of input samples and $C$ is the number of classes. - -- `targets` -If you use the default operator for classification task we expect `targets` to be a $(N, C)$ tensor or array which is a one-hot encoding of **the class you want to explain** where $N$ is the number of input samples and $C$ is the number of classes. - -#### Regression Tasks - -If the task at end is regression, then the user should instantiate the explainer with the `regression_operator`: - -```python -@tf.function -def regression_operator(model: Callable, - inputs: tf.Tensor, - targets: tf.Tensor) -> tf.Tensor: - """ - Compute the the mean absolute error between model prediction and the target. - Target should the model prediction on non-perturbed input. - This operator can be used to compute attributions for all outputs of a regression model. - - Parameters - ---------- - model - Model used for computing predictions. - inputs - Input samples to be explained. - targets - Model prediction on non-perturbed inputs. - - Returns - ------- - scores - MAE between model prediction and targets. - """ - scores = tf.reduce_mean(tf.abs(model(inputs) - targets), axis=-1) - return scores -``` - -- `model(inputs)`: -Consequently, we expect `model(inputs)` to yield a $(N, D)$ tensor or array where $N$ is the number of input samples and $D$ is the number of variables the model should predict (possibly one). - -- `targets`: -If you are using the `regression_operator`, it is expected that `targets` to be a $(N, D)$ tensor or array of the expected multi-variate output where $N$ is the number of input samples and $D$ is the number of variables (possibly one). - -#### Object-Detection Tasks - -**Work In Progress** - -#### Segmentation Tasks - -**Work In Progress** - -### How to use them with an explainer ? - -You can build attribution methods with those operator in three ways: - -- Explicitly importing them - -```python -from xplique.attributions import Saliency -from xplique.commons.operators import regression_operator - -explainer = Saliency(model, operator=regression_operator) -explanations = explainer(inputs, targets) -``` - -At present, the available operators are: `predictions_operator` and `regression_operator` - -- Use their name - -```python -from xplique.attributions import Saliency - -explainer = Saliency(model, operator='regression') -explanations = explainer(inputs, targets) -``` - -At present you can select a name in ["classification", "regression"] - -- Use the `Tasks` enumeration - -```python -from xplique.commons import Tasks -from xplique.attributions import Saliency - -explainer = Saliency(model, operator=Tasks.REGRESSION) -explanations = explainer(inputs, targets) -``` - -At present the `Tasks` enum has two members: `CLASSIFICATION` and `REGRESSION` - -## Providing custom operator - -If you provide a custom operator you should be aware that: - -- An assertion will be made to ensure it respects the signature describe in the previous section -- If you use any white-box explainer your operator will go through the `get_gradient_of_operator` function - -```python -def get_gradient_of_operator(operator): - """ - Get the gradient of an operator. - - Parameters - ---------- - operator - Operator to compute the gradient of. - - Returns - ------- - gradient - Gradient of the operator. - """ - @tf.function - def gradient(model, inputs, targets): - with tf.GradientTape() as tape: - tape.watch(inputs) - scores = operator(model, inputs, targets) - - return tape.gradient(scores, inputs) - - return gradient -``` - -!!!tip - Writing your operator with only tensorflow functions should increase your chance that this method does not yield any errors. In addition, providing a `@tf.function` decorator is also welcomed! - -!!!warning - The `targets` parameter is the key to specifying what to explain and differs greatly depending on the operator. diff --git a/docs/pytorch.md b/docs/api/attributions/pytorch.md similarity index 67% rename from docs/pytorch.md rename to docs/api/attributions/pytorch.md index 63674a57..e50311ca 100644 --- a/docs/pytorch.md +++ b/docs/api/attributions/pytorch.md @@ -1,15 +1,21 @@ -# PyTorch's model with Xplique +# PyTorch models with Xplique -- [**PyTorch's model**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) +- [**PyTorch models**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) -- [**Metrics**: With Pytorch's model](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) [](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) +- [**Metrics**: With PyTorch models](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) [](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) + +- Other tutorials applying Xplique to PyTorch models: [Attributions: Object Detection](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL), [Attributions: Semantic Segmentation](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) !!!note - We should point out that what we did with Pytorch should be possible for other frameworks. Do not hesitate to give it a try and to make a PR if you have been successful! + We should point out that what we did with PyTorch should be possible for other frameworks. Do not hesitate to give it a try and to make a PR if you have been successful! + + + -## Is it possible to use Xplique with PyTorch's model ? -**Yes**, it is! Even though the library was mainly designed to be a Tensorflow toolbox we have been working on a very practical wrapper to facilitate the integration of your PyTorch's model into Xplique's framework! +## Is it possible to use Xplique with PyTorch models? + +**Yes**, it is! Even though the library was mainly designed to be a Tensorflow toolbox we have been working on a very practical wrapper to facilitate the integration of your PyTorch models into Xplique's framework! ### Quickstart ```python @@ -25,18 +31,26 @@ from xplique.metrics import Deletion device = 'cuda' if torch.cuda.is_available() else 'cpu' wrapped_model = TorchWrapper(torch_model, device) -explainer = Saliency(wrapped_model) +explainer = Saliency(wrapped_model, operator="classification") explanations = explainer(inputs, targets) -metric = Deletion(wrapped_model, inputs, targets) +metric = Deletion(wrapped_model, inputs, targets, operator="classification") score_saliency = metric(explanations) ``` -## Does it work for every modules ? + + + + +## Does it work for every module? It has been tested on both the `attributions` and the `metrics` modules. -## Does it work for all attribution methods ? + + + + +## Does it work for all attribution methods? Not yet, but it works for most of them (even for gradient-based ones!): @@ -59,18 +73,29 @@ Not yet, but it works for most of them (even for gradient-based ones!): | SquareGrad | β | | VarGrad | β | -## Steps to make Xplique work on pytorch + + + + +## Does it work for all tasks? + +It works for all tasks covered by Xplique, see [the tasks covered and how to specify them](../api_attributions/#the-tasks-covered). + + + + +## Steps to make Xplique work on PyTorch ### 1. Make sure the inputs follow the Xplique API (and not what the model expects). -One thing to keep in mind is that **attribution methods expect a specific inputs format** as described in the [API Description](api/attributions/api_attributions.md). Especially, for images `inputs` should be $(N, H, W, C)$ following the TF's conventions where: +One thing to keep in mind is that **attribution methods expect a specific inputs format** as described in the [API Description](../api_attributions/#inputs). Especially, for images `inputs` should be $(N, H, W, C)$ following the TF's conventions where: - $N$ is the number of inputs - $H$ is the height of the images - $W$ is the width of the images - $C$ is the number of channels -However, if you are using a PyTorch's model it is most likely expecting images' shape to be $(N, C, H, W)$. So what should you do ? +However, if you are using a PyTorch models it is most likely expecting images' shape to be $(N, C, H, W)$. So what should you do? If you are using PyTorch's preprocessing functions what you should do is: @@ -82,7 +107,9 @@ If you are using PyTorch's preprocessing functions what you should do is: The third step is necessary only if your data has a `channel` dimension which is not in the place expected with Tensorflow !!!tip - If you want to be sure how this work you can look at the [**PyTorch's model**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) notebook and compare it to the [**Attribution methods**:Getting Started](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) + If you want to be sure how this work you can look at the [**PyTorch models**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) notebook and compare it to the [**Attribution methods**:Getting Started](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) + + ### 2. Wrap your model @@ -97,15 +124,21 @@ The last parameter is the one that needs special care. Indeed, if it is set to ` !!!info It is possible that you used special treatments for your models or that it does not follow typical convention. In that case, we encourage you to have a look at the
[Source Code](https://github.com/deel-ai/xplique/blob/master/xplique/wrappers/pytorch.py) to adapt it to your needs.
+
+
### 3. Use this wrapped model as a TF's one
-## What are the limitations ?
+
+
+
+
+## What are the limitations?
As it was previously mentionned this does not work with: Deconvolution, Grad-CAM, Grad-CAM++ and Guided Backpropagation.
Furthermore, when one use any white-box explainers one have the possibility to provide an `output_layer` parameter. This functionnality will not work with PyTorch models. The user will have to manipulate itself its model!
!!!warning
- The `output_layer` parameter does not work for PyTorch's model!
+ The `output_layer` parameter does not work for PyTorch models!
It is possible that all failure cases were not covered in the tests, in that case please open an issue so the team will work on it!
\ No newline at end of file
diff --git a/docs/api/attributions/regression.md b/docs/api/attributions/regression.md
new file mode 100644
index 00000000..9131a19c
--- /dev/null
+++ b/docs/api/attributions/regression.md
@@ -0,0 +1,124 @@
+# Regression explanations with Xplique
+
+[Attributions: Regression and Tabular data tutorial](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) [](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq)
+
+
+
+
+
+## Which kind of tasks are supported by Xplique?
+
+With the [operator's api](../api/attributions/operator) you can treat many different problems with Xplique. There is one operator for each task.
+
+| Task and Documentation link | `operator` parameter value from `xplique.Tasks` Enum | Tutorial link | +| :------------------------------------------------- | :---------------------------------------------------------- | :------------ | +| [Classification](../classification/) | `CLASSIFICATION` | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | +| [Object Detection](../object_detection/) | `OBJECT_DETECTION` | [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) | +| **Regression** | `REGRESSION` | [](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) | +| [Semantic Segmentation](../semantic_segmentation/) | `SEMANTIC_SEGMENTATION` | [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) | + +!!!info + They all share the [API for Xplique attribution methods](../api_attributions/). + +!!!warning + In Xplique, for now with regression, predictions can only be explained output by output. Indeed, explaining several output simultaneously brings new problematic and we are currently working on an operator to solve this. + + + + + +## Simple example + +```python +import xplique +from xplique.attributions import Saliency +from xplique.metrics import Deletion + +# load inputs and model +# ... + +# for regression, `targets` indicates the output of interest, here output 3 +targets = tf.one_hot([2], depth=nb_outputs, axis=-1) + +# compute explanations by specifying the regression operator +explainer = Saliency(model, operator=xplique.Tasks.REGRESSION) +explanations = explainer(inputs, targets) + +# compute metrics on these explanations +metric = Deletion(model, inputs, targets, operator=xplique.Tasks.REGRESSION) +score_saliency = metric(explanations) +``` + + + + + +## How to use it? + +To apply attribution methods, the [**common API documentation**](../api_attributions/) describes the parameters and how to fix them. However, depending on the task and thus on the `operator`, there are three points that vary: + +- **[The `operator` parameter](#the-operator)** value, it is an Enum or a string identifying the task, + +- **[The model's output](#models-output)** specification, as `model(inputs)` is used in the computation of the operators, and + +- **[The `targets` parameter](#the-targets-parameter)** format, indeed, the `targets` parameter specifies what to explain and the format of such specification depends on the task. + + + + + +## The `operator` ## + +### How to specify it + +In Xplique, to adapt attribution methods, you should specify the task to the `operator` parameter. In the case of regression, with either: +```python +Method(model, operator="regression") +# or +Method(model, operator=xplique.Tasks.REGRESSION) +``` + + + +### The computation + +The regression operator works similarly to the classification operator, it asks for the output of interest via `targets` and returns this output. See [targets section](#the-targets-parameter) for more detail. +```python +scores = tf.reduce_sum(model(inputs) * targets, axis=-1) +``` + + + +### The behavior + +- In the case of [perturbation-based methods](../api_attributions/#gradient-based-approaches), the perturbation score corresponds to the difference between the initial value of the output of interest and the same output for predictions over perturbed inputs. +- For [gradient-based methods](../api_attributions/#perturbation-based-approaches), the gradient of the model's predictions for the output of interest. + + + + + +## Model's output ## + +We expect `model(inputs)` to yield a $(n, d)$ tensor or array where $n$ is the number of input samples and $d$ is the number of variables the model should predict (possibly one). + + + + + +## The `targets` parameter ## + +### Role + +The `targets` parameter specifies what is to explain in the `inputs`, it is passed to the `explain` or to the `__call__` method of an explainer or metric and used by the operators. In the case of regression it indicates which of the output should be explained. + + + +### Format + +The `targets` parameter in the case of regression should have the same shape as the [model's output](#models-output) as they are multiplied. Hence, the shape is $(n, d)$ with $n$ the number of samples to be explained (it should match the first dimension of `inputs`) and $d$ is the number of variables (possibly one). + + +### In practice + +In the [simple example](#simple-example), the `targets` value provided is computed with `tf.one_hot`. Indeed, the regression operator takes as `targets` the one hot encoding of the index of the output to explain. diff --git a/docs/api/attributions/semantic_segmentation.md b/docs/api/attributions/semantic_segmentation.md new file mode 100644 index 00000000..8294b0ee --- /dev/null +++ b/docs/api/attributions/semantic_segmentation.md @@ -0,0 +1,322 @@ +# Semantic segmentation explanations with Xplique + +[Attributions: Semantic segmentation tutorial](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) + + + + + +## Which kind of tasks are supported by Xplique? + +With the [operator's api](../api/attributions/operator) you can treat many different problems with Xplique. There is one operator for each task. + +| Task and Documentation link | `operator` parameter value
from `xplique.Tasks` Enum | Tutorial link | +| :------------------------------------------------- | :---------------------------------------------------------- | :------------ | +| [Classification](../classification/) | `CLASSIFICATION` | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | +| [Object Detection](../object_detection/) | `OBJECT_DETECTION` | [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) | +| [Regression](../regression/) | `REGRESSION` | [](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) | +| **Semantic Segmentation** | `SEMANTIC_SEGMENTATION` | [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) | + +!!!info + They all share the [API for Xplique attribution methods](../api_attributions/). + + + + + +## Simple example + +```python +import xplique +from xplique.utils_functions.segmentation import get_connected_zone +from xplique.attributions import Saliency +from xplique.metrics import Deletion + +# load images and model +# ... + +# extract targets individually +coordinates_of_object = (42, 42) +predictions = model(image) +target = get_connected_zone(predictions, coordinates_of_object) +inputs = tf.expand_dims(image, 0) +targets = tf.expand_dims(target, 0) + +explainer = Saliency(model, operator=xplique.Tasks.SEMANTIC_SEGMENTATION) +explanations = explainer(inputs, targets) + +metric = Deletion(model, inputs, targets, operator=xplique.Tasks.SEMANTIC_SEGMENTATION) +score_saliency = metric(explanations) +``` + + + + + +## How to use it? + +To apply attribution methods, the [**common API documentation**](../api_attributions/) describes the parameters and how to fix them. However, depending on the task and thus on the `operator`, there are three points that vary: + +- **[The `operator` parameter](#the-operator)** value, it is an Enum or a string identifying the task, + +- **[The model's output](#models-output)** specification, as `model(inputs)` is used in the computation of the operators, and + +- **[The `targets` parameter](#the-targets-parameter)** format, indeed, the `targets` parameter specifies what to explain and the format of such specification depends on the task. + +!!!info + Applying attribution methods to semantic segmentation with Xplique has a particularity: a set of functions from `utils_functions.segmentation` are used to define `targets` and are documented in the a [specific section](#the-segmentation-utils-functions). + + + +## The `operator` ## + +### How to specify it + +In Xplique, to adapt attribution methods, you should specify the task to the `operator` parameter. In the case of semantic segmentation, with either: +```python +Method(model, operator="semantic segmentation") +# or +Method(model, operator=xplique.Tasks.SEMANTIC_SEGMENTATION) +``` + + + +### The computation + +The operator for semantic segmentation is similar to the classification one, but the output is not a class but a matrix of class. The operator should take this position into account, thus it manipulates two elements: + +- **The zone of interest**: it represents the zone/pixels on which we want the explanation to be made. It could be a single object like a person, a group of objects like trees, a part of an object that has been wrongly classified, or even the border of an object. Note that the concept of object here only makes sense for us as the model only classifies pixels, which is why Xplique includes the [segmentation utils function](#the-segmentation-utils-functions). + +- **The class of interest**: it represents the channel of the prediction we want to explain. Similarly to classification, we could either want to explain a cat or a dog in the same image. Note that in some case, providing several classes could make sense, see the example of applications with [explanations of the borders between two objects](#the-border-between-two-objects). + +Indeed, the semantic segmentation operator multiplies the model's predictions by the targets, which can be considered a mask. Then the operator divide the sum of the remaining predictions over the size of the mask. In some, the operator take the mean predictions over the zone and class of interest + +$$ +score = mean_{over\ the\ zone\ and\ class\ of\ interest}(model(inputs)) +$$ + +Note that the two information need to be communicated through the [`targets` parameter](#the-targets-parameter). + + + +### The behavior + +- In the case of [perturbation-based methods](../api_attributions/#gradient-based-approaches), the perturbation score is the difference between the operator's output for the studied `inputs` and the perturbed inputs. Where the operator's output is the mean logits value over the class and zone of interest. +- For [gradient-based methods](../api_attributions/#perturbation-based-approaches), the gradient of the mean of model's predictions limited to the zone and class of interest. + + + + + +## Model's output ## + +We expect `model(inputs)` to yield a $(n, h, w, c)$ tensor or array where: + +- $n$: the number of inputs, it should match the first dimension of `inputs` +- $h$: the height of the images +- $w$: the width of the images +- $c$: the number of classes + +!!!warning + The model's output for each pixel is expected to be a soft output and not the class prediction or a one hot encoding of the class. Otherwise the attribution methods will not be able to compare predictions efficiently. + +!!!warning + Contrary to classification, here a softmax or comparable last layer is necessary as zeros are interpreted by the operator as non-zone of interest. In this sense, strictly positive values are required. + + + + + +## The `targets` parameter ## + +### Role + +The `targets` parameter specifies what is to explain in the `inputs`, it is passed to the `explain` or to the `__call__` method of an explainer or metric and used by the operators. In the case of semantic segmentation, the `targets` parameter enables the communication of the two necessary information for the [semantic segmentation operator](#the-operator): + +- **The zone of interest**: to communicate the zone of interest via the `targets` parameter, the `targets` value on pixels that are not in the zone of interest should be set to zero. In this way `tf.math.sign(targets)` creates a mask of the zone of interest. This operation should be done along the $h$ and $w$ dimensions of `targets`. + +- **The class of interest**: similarly to the zone of interest, the class of interest is communicated by setting other classes along the $c$ dimension to zero. + + + +### Format + +The `targets` parameter in the case of semantic segmentation should have the same shape as the [model's output](#models-output) as a difference is made between the two. Hence, the shape is $(n, h, w,c )$ with: + - $n$ is the number of inputs, it should match the first dimension of `inputs` + - $h$ is the height of the images + - $w$ is the width of the images + - $c$ is the number of classes + +Then it should take values in $\{-1, 0, 1\}$, $1$ in the zone of interest (zone on the $h$ and $w$ dimension) and $0$ elsewhere. Similarly, values not on the channel corresponding to the class of interest (dimension $c$) should be $0$. In the case of the explanation of a border or with contrastive explanations, $-1$ values might be used. + + + +### In practice + +The `targets` parameter is computed via the [xplique.utils_functions.segmentation](#the-segmentation-utils-functions) set of functions. They manipulate model's prediction individually, as explanation requests are different between each image. Please refer to the [segmentation utils functions](#the-segmentation-utils-functions) for detail on how to design `targets`. + +!!!tip + You should not worry about such specification as the [segmentation utils functions](#the-segmentation-utils-functions) will do the work in your stead. + +!!!warning + The `targets` parameter for each sample should be defined individually. Then the batch dimension should be added manually or individual values should be stacked. + + + + + +## The segmentation utils functions ## + +[Source](https://github.com/deel-ai/xplique/tree/master/xplique/utils_functions/segmentation.py) + +The segmentation utils functions are a set a utility functions used to compute the [`targets` parameter](#the-targets-parameter) values. They should be applied to each image separately as each segmentation is different want the things to explain differs between images. Nonetheless, you could use `tf.map_fn` to apply the same function to several images. + +An example of application of those functions can be found in the [Attribution: Semantic segmentation](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) tutorial. + +For now, there are four functions: + +### `get_class_zone` + +The most simple, where the class of interest is `class_id` and the zone of interest corresponds to pixels where the class is the argmax along the classes dimension of the model's prediction. This function can be used to design `targets` to explain: + +- the [class of a crowd of objects](#the-class-of-a-crowd-of-objects) +- the [class of an object](#the-class-of-an-object), if there is only one object in the image. +- the [class of a set of objects](#the-class-of-a-set-of-object), if there are few and locally close objects of the same class. + +{{xplique.utils_functions.segmentation.get_class_zone}} + + + +### `get_connected_zone` + +Here `coordinates` is a $(h, w)$ tuple that indicates the indices of a pixel of the image. The class of interest is the argmax along the classes dimension for this given pixel. Then the zone of interest is the set of pixels with the same argmax class that forms a connected zone with the indicated pixel. This function can be seen as selecting a zone with a point in this zone. This function can be used to design `targets` to explain: + +- the [class of an object](#the-class-of-an-object). +- the [class of a set of objects](#the-class-of-a-set-of-object), if they are connected. +- the [class of part of an object](#the-class-of-part-of-an-object), if this part have been classified differently than the object and the other surrounding objects. + +{{xplique.utils_functions.segmentation.get_connected_zone}} + + + +### `list_class_connected_zones` + +A mix of `get_class_zone` and `get_connected_zone`. `class_id` indicates the class of interest and each connected zone for this class becomes a zone of interest (apart from zones with size under `zone_minimum_size`). It is useful for automatized treatment of explainability, but may generate explanations for zones we may not want to explain. Nonetheless, it can be used to design `targets` to explain similar elements as `get_connected_zone`. + +!!!warning + Contrarily to the other utils function for segmentation, here output is a list of tensors. + +{{xplique.utils_functions.segmentation.list_class_connected_zones}} + + + +### `get_in_out_border` + +This function allows to compute the `targets` needed to explain the [border of an object](#the-border-of-an-object). For this function, `class_target_mask` encodes the class and the zone of interest. From this zone, the in-border (all pixels of the zone with contact to non-zone pixels) and the out-border (all non-zone pixels with contact to pixels of the zone) are computed. Then, the in-borders pixels are set with the predictions values, and out-borders with the opposite of the predictions values. Therefore, explaining this border corresponds to explaining what increased the class predictions inside the zone and decreased it outside, but along the borders of the zone. + +{{xplique.utils_functions.segmentation.get_in_out_border}} + + + +### `get_common_border` + +This function uses two borders computed via the previous function and limits the zone of interest to the common part between both zone of interest. The classes of interest are merged, thus creating a second class of interest. Therefore, this function enables the creation of `targets` to explain the [border between two objects](#the-border-between-two-objects). + +{{xplique.utils_functions.segmentation.get_common_border}} + + + + + + +## What can be explained with it? ## + +There are many things that we may want to explain in semantic segmentation, and in this section present different possibilities. The [segmentation utils functions](#the-segmentation-utils-functions) allow the design of the [`targets` parameter](#the-targets-parameter) to specify what to explain. + +!!!warning + The concept **object** does not make sense for the model, a semantic segmentation model only classifies pixels. However, what humans want to explain are mainly objects, sets of objects or parts of them. + +!!!info + As objects do not make sense for the model, to stay coherent when manipulating objects. The only condition is that the predicted class on this connected zone is the same for all pixels. + +For a concrete example, please refer to the [Attributions: Semantic segmentation](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) tutorial. + + + +### The class of an object ### + +Here an object can be a person walking on a street, the dog by his side or a car. + +However, what humans call an object does not make sense for model, hence explaining an object corresponds to explaining a zone of interest where pixels have the same classification. + +!!!warning + The zone should be extracted from the model's prediction and not the labels. + +To explain the difference between labels and predictions there are two possibilities: + +- either the difference is a single zone with a different class than the surroundings, then this zone can be considered an object. +- or the difference is more complex or mixed with other objects. Then the zones in the union but not in the intersection of both should be iteratively considered objects and explained. It is not recommended to treat them simultaneously. + + + +### The class of a set of objects ### + +A set of objects can be a group of people walking down a street or a set of trees on one side of the road. + +There are three cases that can be considered set of objects: + +- Connected set of objects, it can be seen as only one big zone and treated the same as in [1.](#the-class-of-an-object) +- Locally close set of objects, this could also considered a big zone, but it is harder to compute. +- Set of objects dispersed on the image and hardly countable, if there are a multitude of objects then, it can be seen as a [crowd of objects](#the-class-of-a-crowd-of-objects). Otherwise, it should not be treated together. + + + +### The class of part of an object ### + +A part of an object can be the leg of a person, the head of a dog, or a person in a group of people. This is interesting when the part and the object have been classified differently by the model. It should be considered an object as in [1.](#the-class-of-an-object) + + + +### The class of a crowd of objects ### + +A crowd is a set of hardly countable objects, it can be a set of clouds, a multitude of people on the sidewalk or trees in a landscape. + + + +### The border of an object ### + +The border of an object is the limit between the pixels inside the object and those outside of it. Here the object should correspond to a connected zone of pixels where the model predicts the same class. + +It can be the contour of three people on the side walk or of trees on a landscape. It is interesting when the border is hard to define between similarly colored pixels or when the model prediction is not precise. + + + +### The border between two objects ### + +The border between two objects is the common part between two borders of objects when those two are connected. This can be the border between a person and his wrongly classified leg. + + + + + +## Binary semantic segmentation ## + +As described in the [operator description](#the-operator), the output of the model should have a shape of $(n, h, w, c)$. However, in binary semantic segmentation, the two classes are often encoded by positive and negative value along only one channel with shape $(n, h, w)$. + +The easiest way to apply xplique on such model is to wrap the model to match the expected format. If we suppose that the output of the binary semantic segmentation model have a shape of $(n, h, w)$, that negative values encode class $0$, and that positive values encode class $1$. Then the wrapper can take the form: + +```python +class Wrapper(): + def __init__(model): + self.model = model + + def __call__(inputs): + binary_segmentation = self.model(inputs) + class_0_mask = binary_segmentation < 0 + divided = tf.stack([-binary_segmentation * tf.cast(class_0_mask, tf.float32), + binary_segmentation * tf.cast(tf.logical_not(class_0_mask), tf.float32)], + axis=-1) + return tf.nn.softmax(divided, axis=-1) + +wrapped_model = wrap(binary_seg_model) +``` \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index f249772f..e4fd1372 100644 --- a/docs/index.md +++ b/docs/index.md @@ -26,10 +26,12 @@
- π¦ Xplique (pronounced \Ιks.plik\) is a Python toolkit dedicated to explainability. The goal of this library is to gather the state of the art of Explainable AI to help you understand your complex neural network models. Originally built for Tensorflow's model it also works for Pytorch's model partially.
+ π¦ Xplique (pronounced \Ιks.plik\) is a Python toolkit dedicated to explainability. The goal of this library is to gather the state of the art of Explainable AI to help you understand your complex neural network models. Originally built for Tensorflow's model it also works for PyTorch models partially.
- Explore Xplique docs Β» + π Explore Xplique docs + | + Explore Xplique tutorials π₯
Attributions @@ -66,6 +68,8 @@ Finally, the _Metrics_ module covers the current metrics used in explainability. - [**Attribution Methods**: Sanity checks paper](https://colab.research.google.com/drive/1uJOmAg6RjlOIJj6SWN9sYRamBdHAuyaS) [](https://colab.research.google.com/drive/1uJOmAg6RjlOIJj6SWN9sYRamBdHAuyaS) - [**Attribution Methods**: Tabular data and Regression](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) [](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) + - [**Attribution Methods**: Object Detection](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) + - [**Attribution Methods**: Semantic Segmentation](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) - [**FORGRad**: Gradient strikes back with FORGrad](https://colab.research.google.com/drive/1ibLzn7r9QQIEmZxApObowzx8n9ukinYB) [](https://colab.research.google.com/drive/1ibLzn7r9QQIEmZxApObowzx8n9ukinYB) - [**Attribution Methods**: Metrics](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) [](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) @@ -75,7 +79,7 @@ Finally, the _Metrics_ module covers the current metrics used in explainability. - - [**PyTorch's model**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) + - [**PyTorch models**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) - [**Concepts Methods**: Testing with Concept Activation Vectors](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE) [](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE) @@ -95,19 +99,19 @@ Finally, the _Metrics_ module covers the current metrics used in explainability. - [**Modern Feature Visualization with MaCo**: Getting started](https://colab.research.google.com/drive/1l0kag1o-qMY4NCbWuAwnuzkzd9sf92ic) [](https://colab.research.google.com/drive/1l0kag1o-qMY4NCbWuAwnuzkzd9sf92ic) - You can find a certain number of [other practical tutorials just here](tutorials/). This section is actively developed and more contents will be - included. We will try to cover all the possible usage of the library, feel free to contact us if you have any suggestions or recommandations towards tutorials you would like to see. + You can find a certain number of [**other practical tutorials just here**](tutorials/). This section is actively developed and more contents will be + included. We will try to cover all the possible usage of the library, feel free to contact us if you have any suggestions or recommendations towards tutorials you would like to see. ## π Quick Start -Xplique requires a version of python higher than 3.6 and several libraries including Tensorflow and Numpy. Installation can be done using Pypi: +Xplique requires a version of python higher than 3.7 and several libraries including Tensorflow and Numpy. Installation can be done using Pypi: ```python pip install xplique ``` -Now that Xplique is installed, here are 4 basic examples of what you can do with the available modules. +Now that Xplique is installed, here are some basic examples of what you can do with the available modules. ??? example "Attributions Methods" Let's start with a simple example, by computing Grad-CAM for several images (or a complete dataset) on a trained model. @@ -123,9 +127,7 @@ Now that Xplique is installed, here are 4 basic examples of what you can do with # or just `explainer(images, labels)` ``` - All attributions methods share a common API. You can find out more about it [here](api/attributions/api_attributions/). - - In addition, you should also look at the [model's specificities](api/attributions/model/) and the [operator parameter documentation](api/attributions/operator/) + All attributions methods share a common API described [in the attributions API documentation](api/attributions/api_attributions/). ??? example "Attributions Metrics" @@ -145,7 +147,7 @@ Now that Xplique is installed, here are 4 basic examples of what you can do with score_grad_cam = metric(explanations) ``` - All attributions metrics share a common API. You can find out more about it [here](api/metrics/api_metrics/). + All attributions metrics share a common API. You can find out more about it [here](api/attributions/metrics/api_metrics/). ??? example "Concepts Extraction" @@ -186,7 +188,7 @@ Now that Xplique is installed, here are 4 basic examples of what you can do with ??? example "PyTorch with Xplique" - Even though the library was mainly designed to be a Tensorflow toolbox we have been working on a very practical wrapper to facilitate the integration of your PyTorch's model into Xplique's framework! + Even though the library was mainly designed to be a Tensorflow toolbox we have been working on a very practical wrapper to facilitate the integration of your PyTorch models into Xplique's framework! ```python import torch @@ -208,56 +210,61 @@ Now that Xplique is installed, here are 4 basic examples of what you can do with score_saliency = metric(explanations) ``` - Want to know more ? Check the [PyTorch documentation](pytorch/) + Want to know more ? Check the [PyTorch documentation](api/attributions/pytorch/) ## π¦ What's Included +There are 4 modules in Xplique, [Attribution methods](api/attributions/api_attributions/), [Attribution metrics](api/attributions/metrics/api_metrics/), [Concepts](api/concepts/cav/), and [Feature visualization](api/feature_viz/feature_viz/). In particular, the attribution methods module supports a huge diversity of tasks for diverse data types: [Classification](api/attributions/classification/), [Regression](api/attributions/regression/), [Object Detection](api/attributions/object_detection/), and [Semantic Segmentation](api/attributions/semantic_segmentation/). The methods compatible with such task and methods compatible with Tensorflow or PyTorch are highlighted in the following table: + ??? abstract "Table of attributions available" - All the attributions method presented below handle both **Classification** and **Regression** tasks. - - | **Attribution Method** | Type of Model | Source | Tabular Data | Images | Time-Series | Tutorial | - | :--------------------- | :------------ | :---------------------------------------- | :----------------: | :----------------: | :----------------: | :----------------: | - | Deconvolution | TF | [Paper](https://arxiv.org/abs/1311.2901) | β | β | WIP | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | - | Grad-CAM | TF | [Paper](https://arxiv.org/abs/1610.02391) | | β | | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | - | Grad-CAM++ | TF | [Paper](https://arxiv.org/abs/1710.11063) | | β | | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | - | Gradient Input | TF, Pytorch** | [Paper](https://arxiv.org/abs/1704.02685) | β | β | WIP | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | - | Guided Backprop | TF | [Paper](https://arxiv.org/abs/1412.6806) | β | β | WIP | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | - | Integrated Gradients | TF, Pytorch** | [Paper](https://arxiv.org/abs/1703.01365) | β | β | WIP | [](https://colab.research.google.com/drive/1UXJYVebDVIrkTOaOl-Zk6pHG3LWkPcLo) | - | Kernel SHAP | TF, Pytorch** , Callable* | [Paper](https://arxiv.org/abs/1705.07874) | β | β | WIP | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | - | Lime | TF, Pytorch** , Callable* | [Paper](https://arxiv.org/abs/1602.04938) | β | β | WIP | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | - | Occlusion | TF, Pytorch** , Callable* | [Paper](https://arxiv.org/abs/1311.2901) | β | β | WIP | [](https://colab.research.google.com/drive/15xmmlxQkNqNuXgHO51eKogXvLgs-sG4q) | - | Rise | TF, Pytorch** , Callable* | [Paper](https://arxiv.org/abs/1806.07421) | WIP | β | | [](https://colab.research.google.com/drive/1icu2b1JGfpTRa-ic8tBSXnqqfuCGW2mO) | - | Saliency | TF, Pytorch** | [Paper](https://arxiv.org/abs/1312.6034) | β | β | WIP | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | - | SmoothGrad | TF, Pytorch** | [Paper](https://arxiv.org/abs/1706.03825) | β | β | WIP | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | - | SquareGrad | TF, Pytorch** | [Paper](https://arxiv.org/abs/1806.10758) | β | β | WIP | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | - | VarGrad | TF, Pytorch** | [Paper](https://arxiv.org/abs/1810.03292) | β | β | WIP | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | - | Sobol Attribution | TF, Pytorch** | [Paper](https://arxiv.org/abs/2111.04138) | | β | | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | - | Hsic Attribution | TF, Pytorch** | [Paper](https://arxiv.org/abs/2206.06219) | | β | | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | - | FORGrad enhancement | TF, Pytorch** | [Paper](https://arxiv.org/abs/2307.09591) | | β | | [](https://colab.research.google.com/drive/1ibLzn7r9QQIEmZxApObowzx8n9ukinYB) | + | **Attribution Method** | Type of Model | Source | Tabular Data | Images | Time-Series | Tutorial | + | :--------------------- | :----------------------- | :---------------------------------------- | :------------: | :------------------------: | :---------: | :----------------: | + | Deconvolution | TF | [Paper](https://arxiv.org/abs/1311.2901) | C:βοΈ
R:βοΈ | C:βοΈ
OD:β
SS:β | π΅ | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | + | Grad-CAM | TF | [Paper](https://arxiv.org/abs/1610.02391) | β | C:βοΈ
OD:β
SS:β | β | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | + | Grad-CAM++ | TF | [Paper](https://arxiv.org/abs/1710.11063) | β | C:βοΈ
OD:β
SS:β | β | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | + | Gradient Input | TF, PyTorch** | [Paper](https://arxiv.org/abs/1704.02685) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | + | Guided Backprop | TF | [Paper](https://arxiv.org/abs/1412.6806) | C:βοΈ
R:βοΈ | C:βοΈ
OD:β
SS:β | π΅ | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | + | Integrated Gradients | TF, PyTorch** | [Paper](https://arxiv.org/abs/1703.01365) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/1UXJYVebDVIrkTOaOl-Zk6pHG3LWkPcLo) | + | Kernel SHAP | TF, PyTorch**, Callable* | [Paper](https://arxiv.org/abs/1705.07874) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | + | Lime | TF, PyTorch**, Callable* | [Paper](https://arxiv.org/abs/1602.04938) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | + | Occlusion | TF, PyTorch**, Callable* | [Paper](https://arxiv.org/abs/1311.2901) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/15xmmlxQkNqNuXgHO51eKogXvLgs-sG4q) | + | Rise | TF, PyTorch**, Callable* | [Paper](https://arxiv.org/abs/1806.07421) | π΅ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/1icu2b1JGfpTRa-ic8tBSXnqqfuCGW2mO) | + | Saliency | TF, PyTorch** | [Paper](https://arxiv.org/abs/1312.6034) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | + | SmoothGrad | TF, PyTorch** | [Paper](https://arxiv.org/abs/1706.03825) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | + | SquareGrad | TF, PyTorch** | [Paper](https://arxiv.org/abs/1806.10758) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | + | VarGrad | TF, PyTorch** | [Paper](https://arxiv.org/abs/1810.03292) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | + | Sobol Attribution | TF, PyTorch** | [Paper](https://arxiv.org/abs/2111.04138) | π΅ | C:βοΈ
OD:βοΈ
SS:βοΈ | β | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | + | Hsic Attribution | TF, PyTorch** | [Paper](https://arxiv.org/abs/2206.06219) | π΅ | C:βοΈ
OD:βοΈ
SS:βοΈ | β | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | + | FORGrad enhancement | TF, PyTorch** | [Paper](https://arxiv.org/abs/2307.09591) | β | C:βοΈ
OD:βοΈ
SS:βοΈ | β | [](https://colab.research.google.com/drive/1ibLzn7r9QQIEmZxApObowzx8n9ukinYB) | TF : Tensorflow compatible - \* : See the [Callable documentation](callable/) + C : [Classification](api/attributions/classification/) | R : [Regression](api/attributions/regression/) | + OD : [Object Detection](api/attributions/object_detection/) | SS : [Semantic Segmentation](api/attributions/semantic_segmentation/) + + \* : See the [Callable documentation](api/attributions/callable/) + + ** : See the [Xplique for PyTorch documentation](api/attributions/pytorch/), and the [**PyTorch models**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) notebook. - ** : See the [Xplique for Pytorch documentation](pytorch/), and the [**PyTorch's model**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) notebook + βοΈ : Supported by Xplique | β : Not applicable | π΅ : Work in Progress ??? abstract "Table of attribution's metric available" | **Attribution Metrics** | Type of Model | Property | Source | | :---------------------- | :------------ | :--------------- | :---------------------------------------- | - | MuFidelity | TF, Pytorch** | Fidelity | [Paper](https://arxiv.org/abs/2005.00631) | - | Deletion | TF, Pytorch** | Fidelity | [Paper](https://arxiv.org/abs/1806.07421) | - | Insertion | TF, Pytorch** | Fidelity | [Paper](https://arxiv.org/abs/1806.07421) | - | Average Stability | TF, Pytorch** | Stability | [Paper](https://arxiv.org/abs/2005.00631) | - | MeGe | TF, Pytorch** | Representativity | [Paper](https://arxiv.org/abs/2009.04521) | - | ReCo | TF, Pytorch** | Consistency | [Paper](https://arxiv.org/abs/2009.04521) | + | MuFidelity | TF, PyTorch** | Fidelity | [Paper](https://arxiv.org/abs/2005.00631) | + | Deletion | TF, PyTorch** | Fidelity | [Paper](https://arxiv.org/abs/1806.07421) | + | Insertion | TF, PyTorch** | Fidelity | [Paper](https://arxiv.org/abs/1806.07421) | + | Average Stability | TF, PyTorch** | Stability | [Paper](https://arxiv.org/abs/2005.00631) | + | MeGe | TF, PyTorch** | Representativity | [Paper](https://arxiv.org/abs/2009.04521) | + | ReCo | TF, PyTorch** | Consistency | [Paper](https://arxiv.org/abs/2009.04521) | | (WIP) e-robustness | TF : Tensorflow compatible - ** : See the [Xplique for Pytorch documentation](pytorch/), and the [**PyTorch's model**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) notebook + ** : See the [Xplique for PyTorch documentation](api/attributions/pytorch/), and the [**PyTorch models**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) notebook. ??? abstract "Table of concept methods available" @@ -297,7 +304,7 @@ This library is one approach of many to explain your model. We don't expect it t ??? info "Other interesting tools to explain your model:" - [Lucid](https://github.com/tensorflow/lucid) the wonderful library specialized in feature visualization from OpenAI. - - [Captum](https://captum.ai/) the Pytorch library for Interpretability research + - [Captum](https://captum.ai/) the PyTorch library for Interpretability research - [Tf-explain](https://github.com/sicara/tf-explain) that implement multiples attribution methods and propose callbacks API for tensorflow. - [Alibi Explain](https://github.com/SeldonIO/alibi) for model inspection and interpretation - [SHAP](https://github.com/slundberg/shap) a very popular library to compute local explanations using the classic Shapley values from game theory and their related extensions diff --git a/docs/tutorials.md b/docs/tutorials.md index 6dd83321..f7e5645a 100644 --- a/docs/tutorials.md +++ b/docs/tutorials.md @@ -1,6 +1,6 @@ # Tutorials: Notebooks π -We propose here several tutorials to discover the different functionnalities that the library has to offer. +We propose here several tutorials to discover the different functionalities that the library has to offer. We decided to host those tutorials on [Google Colab](https://colab.research.google.com/notebooks/intro.ipynb?utm_source=scs-index) mainly because you will be able to play the notebooks with a GPU which should greatly improve your User eXperience. @@ -9,14 +9,16 @@ Here is the lists of the availables tutorial for now: ## Getting Started -| **Tutorial Name** | Notebook | -| :-------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------: | -| Getting Started | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | +| **Tutorial Name** | Notebook | +| :------------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------: | +| Getting Started | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | | Sanity checks for Saliency Maps | [](https://colab.research.google.com/drive/1uJOmAg6RjlOIJj6SWN9sYRamBdHAuyaS) | -| Tabular data and Regression | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | -| Metrics | [](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) | -| Concept Activation Vectors | [](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE) | -| Feature Visualization | [](https://colab.research.google.com/drive/1st43K9AH-UL4eZM1S4QdyrOi7Epa5K8v) | +| Tabular data and Regression | [](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) | +| Object detection | [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) | +| Semantic Segmentation | [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) | +| Metrics | [](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) | +| Concept Activation Vectors | [](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE) | +| Feature Visualization | [](https://colab.research.google.com/drive/1st43K9AH-UL4eZM1S4QdyrOi7Epa5K8v) | ## Attributions @@ -52,10 +54,12 @@ Here is the lists of the availables tutorial for now: ## PyTorch Wrapper -| **Tutorial Name** | Notebook | -| :-------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------: | -| PyTorch's model: Getting started | [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) | -| Metrics: With Pytorch's model| [](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) | +| **Tutorial Name** | Notebook | +| :------------------------------------- | :-----------------------------------------------------------------------------------------------------------------------------------------------------: | +| PyTorch models: Getting started | [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) | +| Metrics: With PyTorch models | [](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) | +| Object detection on PyTorch model | [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) | +| Semantic Segmentation on PyTorch model | [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) | ## Concepts extraction diff --git a/mkdocs.yml b/mkdocs.yml index 3a2bfdb2..92f48a15 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -8,34 +8,36 @@ nav: - Home: index.md - Attributions methods: - API Description: api/attributions/api_attributions.md - - Model Specifications: api/attributions/model.md - - Operator: api/attributions/operator.md - Methods: - - DeconvNet: api/attributions/deconvnet.md - - Grad-CAM: api/attributions/grad_cam.md - - Grad-CAM++: api/attributions/grad_cam_pp.md - - Gradient Input: api/attributions/gradient_input.md - - Guided Backprop: api/attributions/guided_backpropagation.md - - Hsic Attribution Method: api/attributions/hsic.md - - Integrated Gradient: api/attributions/integrated_gradients.md - - KernelSHAP: api/attributions/kernel_shap.md - - Lime: api/attributions/lime.md - - Occlusion sensitivity: api/attributions/occlusion.md - - Rise: api/attributions/rise.md - - Saliency: api/attributions/saliency.md - - SmoothGrad: api/attributions/smoothgrad.md - - Sobol Attribution Method: api/attributions/sobol.md - - SquareGrad: api/attributions/square_grad.md - - VarGrad: api/attributions/vargrad.md - - ForGRad: api/attributions/forgrad.md + - DeconvNet: api/attributions/methods/deconvnet.md + - ForGRad: api/attributions/methods/forgrad.md + - Grad-CAM: api/attributions/methods/grad_cam.md + - Grad-CAM++: api/attributions/methods/grad_cam_pp.md + - Gradient Input: api/attributions/methods/gradient_input.md + - Guided Backprop: api/attributions/methods/guided_backpropagation.md + - Hsic Attribution Method: api/attributions/methods/hsic.md + - Integrated Gradient: api/attributions/methods/integrated_gradients.md + - KernelSHAP: api/attributions/methods/kernel_shap.md + - Lime: api/attributions/methods/lime.md + - Occlusion sensitivity: api/attributions/methods/occlusion.md + - Rise: api/attributions/methods/rise.md + - Saliency: api/attributions/methods/saliency.md + - SmoothGrad: api/attributions/methods/smoothgrad.md + - Sobol Attribution Method: api/attributions/methods/sobol.md + - SquareGrad: api/attributions/methods/square_grad.md + - VarGrad: api/attributions/methods/vargrad.md - Metrics: - - API Description: api/metrics/api_metrics.md - - Deletion: api/metrics/deletion.md - - Insertion: api/metrics/insertion.md - - MuFidelity: api/metrics/mu_fidelity.md - - AverageStability: api/metrics/avg_stability.md - - PyTorch: pytorch.md - - Callable: callable.md + - API Description: api/attributions/metrics/api_metrics.md + - Deletion: api/attributions/metrics/deletion.md + - Insertion: api/attributions/metrics/insertion.md + - MuFidelity: api/attributions/metrics/mu_fidelity.md + - AverageStability: api/attributions/metrics/avg_stability.md + - PyTorch: api/attributions/pytorch.md + - Callable: api/attributions/callable.md + - Classification: api/attributions/classification.md + - Object Detection: api/attributions/object_detection.md + - Regression: api/attributions/regression.md + - Semantic Segmentation: api/attributions/semantic_segmentation.md - Concept based: - Cav: api/concepts/cav.md - Tcav: api/concepts/tcav.md diff --git a/requirements.txt b/requirements.txt index 125d3ba5..da889c05 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,3 +5,4 @@ scikit-image matplotlib scipy opencv-python +deprecated diff --git a/setup.cfg b/setup.cfg index d588f2e2..d88d31a0 100644 --- a/setup.cfg +++ b/setup.cfg @@ -1,5 +1,5 @@ [bumpversion] -current_version = 1.1.0 +current_version = 1.2.0 commit = True tag = True @@ -30,7 +30,7 @@ ignore-imports = no envlist = py{37,38,39,310}-lint, py{37,38,39,310}-tf{22,25,28,211}, py{38,39,310}-tf{25,28,211}-torch{111,113,200} [testenv:py{37,38,39,310}-lint] -deps = +deps = pylint -rrequirements.txt commands = diff --git a/setup.py b/setup.py index c33a4bc5..5bd1d241 100644 --- a/setup.py +++ b/setup.py @@ -5,7 +5,7 @@ setup( name="Xplique", - version="1.1.0", + version="1.2.0", description="Explanations toolbox for Tensorflow 2", long_description=README, long_description_content_type="text/markdown", @@ -13,7 +13,7 @@ author_email="thomas_fel@brown.edu", license="MIT", install_requires=['tensorflow>=2.1.0', 'numpy', 'scikit-learn', 'scikit-image', - 'matplotlib', 'scipy', 'opencv-python'], + 'matplotlib', 'scipy', 'opencv-python', 'deprecated'], extras_require={ "tests": ["pytest", "pylint"], "docs": ["mkdocs", "mkdocs-material", "numkdoc"], diff --git a/tests/attributions/test_callable.py b/tests/attributions/test_callable.py index f5457f12..c32a7d1d 100644 --- a/tests/attributions/test_callable.py +++ b/tests/attributions/test_callable.py @@ -10,7 +10,7 @@ from sklearn.ensemble import RandomForestClassifier from xplique.attributions import (Occlusion, Rise, Lime, KernelShap, SobolAttributionMethod) -from xplique.commons.operators import predictions_operator, batch_predictions,\ +from xplique.commons.operators_operations import predictions_operator, batch_predictions,\ batch_predictions_one_hot_callable from xplique.commons.callable_operations import predictions_one_hot_callable diff --git a/tests/attributions/test_object_detector.py b/tests/attributions/test_object_detector.py index 581aa6ca..aeaf5c1c 100644 --- a/tests/attributions/test_object_detector.py +++ b/tests/attributions/test_object_detector.py @@ -1,100 +1,67 @@ -import numpy as np +""" +Test object detection BoundingBoxesExplainer +""" +import os +import sys +sys.path.append(os.getcwd()) + +import unittest + import tensorflow as tf -from xplique.attributions import BoundingBoxesExplainer, Rise -from xplique.attributions.object_detector import SegmentationIouCalculator, BoxIouCalculator, \ - ImageObjectDetectorExplainer, ImageObjectDetectorScoreCalculator, IObjectFormater - -from ..utils import almost_equal, generate_model, generate_data - - -def test_iou_mask(): - """Assert the Mask IoU calculation is ok""" - dtype = np.float32 - - m1 = np.array([ - [10, 20, 30, 40] - ], dtype=dtype) - m2 = np.array([ - [15, 20, 30, 40] - ], dtype=dtype) - m3 = np.array([ - [0, 20, 10, 40] - ], dtype=dtype) - m4 = np.array([ - [0, 0, 100, 100] - ], dtype=dtype) - - iou_calculator = BoxIouCalculator() - - assert almost_equal(iou_calculator.intersect(m1, m2), 300.0 / 400.0) - assert almost_equal(iou_calculator.intersect(m1, m3), 0.0) - assert almost_equal(iou_calculator.intersect(m3, m2), 0.0) - assert almost_equal(iou_calculator.intersect(m1, m4), 400.0 / 10_000) - assert almost_equal(iou_calculator.intersect(m2, m4), 300.0 / 10_000) - assert almost_equal(iou_calculator.intersect(m3, m4), 200.0 / 10_000) - - -def test_iou_segmentation(): - """Assert the segmentation IoU computation is ok""" - - m1 = np.array([ - [0, 0], - [1, 1], - ])[None, :, :] - m2 = np.array([ - [1, 1], - [0, 0], - ])[None, :, :] - m3 = np.array([ - [1, 0], - [1, 0], - ])[None, :, :] - - iou_calculator = SegmentationIouCalculator() - - assert almost_equal(iou_calculator.intersect(m1, m2), 0.0) - assert almost_equal(iou_calculator.intersect(m1, m3), 1.0/3.0) - assert almost_equal(iou_calculator.intersect(m3, m2), 1.0/3.0) - assert almost_equal(iou_calculator.intersect(m1, m1), 1.0) - assert almost_equal(iou_calculator.intersect(m2, m2), 1.0) - - -def test_image_object_detector(): +from tests.utils import generate_data, generate_object_detection_model + +from xplique.commons.exceptions import InvalidModelException +from xplique.attributions import BoundingBoxesExplainer, Rise, Saliency + + +def test_object_detector(): """Assert input shape returned is correct""" input_shape = (8, 8, 1) + nb_samples = 3 nb_labels = 2 - x, y = generate_data(input_shape, nb_labels, nb_labels) - model = generate_model(input_shape, nb_labels) + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) method = Rise(model, nb_samples=10) obj_ref = tf.cast([ [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], ], tf.float32) - class BBoxFormater(IObjectFormater): + explainer = BoundingBoxesExplainer(method) - def format_objects(self, predictions): - if np.array_equal(predictions.numpy(), obj_ref.numpy()): - return obj_ref[:4], obj_ref[4:5], obj_ref[5:] + test_raise_assertion_error = unittest.TestCase().assertRaises + test_raise_assertion_error(InvalidModelException, explainer.gradient) + test_raise_assertion_error(InvalidModelException, explainer.batch_gradient) + phis = explainer(x, obj_ref) - bboxes = tf.cast([ - [0, 10, 20, 30], - [0, 0, 100, 100], - ], tf.float32) + assert phis.shape == (obj_ref.shape[0], input_shape[0], input_shape[1]) - proba = tf.cast([0.9, 0.1], tf.float32) - classif = tf.cast([[1.0, 0.0], [0.0, 1.0]], tf.float32) +def test_gradient_object_detector(): + """Assert input shape returned is correct""" + input_shape = (8, 8, 1) + nb_samples = 3 + nb_labels = 2 + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) - return bboxes, proba, classif + method = Saliency(model) - formater = BBoxFormater() - explainer = ImageObjectDetectorExplainer(method, formater, BoxIouCalculator()) + obj_ref = tf.cast([ + [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], + ], tf.float32) - phis = explainer(x, obj_ref) + explainer = BoundingBoxesExplainer(method) - assert phis.shape == (1, input_shape[0], input_shape[1]) + phis = explainer(x, obj_ref) + assert phis.shape == (obj_ref.shape[0], input_shape[0], input_shape[1]) \ No newline at end of file diff --git a/tests/commons/test_object_detection_operator.py b/tests/commons/test_object_detection_operator.py new file mode 100644 index 00000000..e895241a --- /dev/null +++ b/tests/commons/test_object_detection_operator.py @@ -0,0 +1,162 @@ +""" +Test object detection operator +""" +import os +import sys +sys.path.append(os.getcwd()) + +from itertools import combinations + +import numpy as np +import tensorflow as tf + +from tests.utils import almost_equal, generate_object_detection_model, generate_data + +from xplique.attributions import Occlusion, SmoothGrad +from xplique.commons.operators import object_detection_operator + + +def test_object_detector(): + """Assert input shape returned is correct""" + input_shape = (8, 8, 1) + nb_samples = 3 + nb_labels = 2 + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) + + # 3 bounding boxes, one for each input sample + obj_ref = tf.cast([ + [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], + ], tf.float32) + + explainer = Occlusion(model, operator=object_detection_operator, patch_size=4, patch_stride=2) + + # test with only one box to explain by image (3, 7) + phis = explainer(x, obj_ref) + assert phis.shape == (obj_ref.shape[0], input_shape[0], input_shape[1]) + assert phis[0, 0, 0] != np.nan + + phis2 = explainer(x, tf.expand_dims(obj_ref, axis=1)) + assert phis.shape == phis2.shape + assert almost_equal(phis, phis2) + + +def test_gradient_object_detector(): + """Assert input shape returned is correct""" + input_shape = (8, 8, 1) + nb_samples = 3 + nb_labels = 2 + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) + + explainer = SmoothGrad(model, nb_samples=10, operator=object_detection_operator) + + obj_ref = tf.cast([ + [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], + ], tf.float32) + + phis = explainer(x, obj_ref) + + assert phis.shape[:3] == (obj_ref.shape[0], input_shape[0], input_shape[1]) + + +def test_several_boxes_object_detector(): + """Assert input shape returned is correct""" + input_shape = (8, 8, 1) + nb_samples = 3 + nb_labels = 2 + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) + + explainer = SmoothGrad(model, nb_samples=10, operator=object_detection_operator) + + obj_ref = tf.cast([ + [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], + ], tf.float32) + + phis = explainer(x, obj_ref) + assert phis.shape[:3] == (obj_ref.shape[0], input_shape[0], input_shape[1]) + + several_object_refs = tf.tile(tf.expand_dims(obj_ref, axis=1), [1, 5, 1]) + + phis = explainer(x, several_object_refs) + + assert phis.shape[:3] == (several_object_refs.shape[0], input_shape[0], input_shape[1]) + + +def test_all_object_detector_operators(): + """Assert input shape returned is correct""" + input_shape = (8, 8, 1) + nb_samples = 3 + nb_labels = 2 + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) + + obj_ref = tf.cast([ + [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], + ], tf.float32) + + # set params + parameters_normal = { + "include_detection_probability": True, + "include_classification_score": True, + } + + parameters_intersection = { + "include_detection_probability": False, + "include_classification_score": False, + } + + parameters_probability = { + "include_detection_probability": True, + "include_classification_score": False, + } + + parameters_classification = { + "include_detection_probability": False, + "include_classification_score": True, + } + + # create operators + normal_op = lambda model, inputs, targets: \ + object_detection_operator(model, inputs, targets, **parameters_normal) + + intersection_op = lambda model, inputs, targets: \ + object_detection_operator(model, inputs, targets, **parameters_intersection) + + probability_op = lambda model, inputs, targets: \ + object_detection_operator(model, inputs, targets, **parameters_probability) + + classification_op = lambda model, inputs, targets: \ + object_detection_operator(model, inputs, targets, **parameters_classification) + + # compute explanations + phis = Occlusion(model, operator=object_detection_operator, patch_size=4, patch_stride=2)(x, obj_ref) + + normal_phis = Occlusion(model, operator=normal_op, patch_size=4, patch_stride=2)(x, obj_ref) + + intersection_phis = Occlusion(model, operator=intersection_op, patch_size=4, patch_stride=2)(x, obj_ref) + + probability_phis = Occlusion(model, operator=probability_op, patch_size=4, patch_stride=2)(x, obj_ref) + + classification_phis = Occlusion(model, operator=classification_op, patch_size=4, patch_stride=2)(x, obj_ref) + + for phi in [phis, normal_phis, intersection_phis, probability_phis, classification_phis]: + assert phi.shape[:3] == (obj_ref.shape[0], input_shape[0], input_shape[1]) + + assert almost_equal(phis, normal_phis) + + for phi1, phi2 in combinations([normal_phis, intersection_phis, probability_phis, classification_phis], 2): + assert not almost_equal(phi1, phi2) \ No newline at end of file diff --git a/tests/commons/test_operators.py b/tests/commons/test_operators.py index d33083d6..1e39a550 100644 --- a/tests/commons/test_operators.py +++ b/tests/commons/test_operators.py @@ -11,11 +11,8 @@ SquareGrad, GradCAM, Occlusion, Rise, GuidedBackprop, DeconvNet, GradCAMPP, Lime, KernelShap, SobolAttributionMethod, HsicAttributionMethod) -from xplique.commons.operators import (check_operator, predictions_operator, regression_operator, - binary_segmentation_operator, segmentation_operator) -from xplique.commons.operators import Tasks, get_operator -from xplique.commons.exceptions import InvalidOperatorException -from ..utils import generate_data, generate_regression_model +from xplique.commons.operators import (predictions_operator, regression_operator) +from ..utils import generate_data, generate_model, generate_regression_model, almost_equal def default_methods(model, operator): @@ -35,14 +32,6 @@ def default_methods(model, operator): ] -def get_segmentation_model(): - model = tf.keras.Sequential([ - tf.keras.layers.Input((20, 20, 1)), - ]) - model.compile() - return model - - def get_concept_model(): model = tf.keras.Sequential([ tf.keras.layers.Input((6)), @@ -52,64 +41,12 @@ def get_concept_model(): return model -def test_check_operator(): - # ensure that the check operator detects non-operator - - # operator must have at least 3 arguments - function_with_2_arguments = lambda x,y: 0 - - # operator must be Callable - not_a_function = [1, 2, 3] - - for operator in [function_with_2_arguments, not_a_function]: - try: - check_operator(operator) - assert False - except InvalidOperatorException: - pass - - -def test_proposed_operators(): - # ensure all proposed operators are operators - for operator in [predictions_operator, regression_operator, - binary_segmentation_operator, segmentation_operator]: - check_operator(operator) - - -def test_get_operator(): - tasks_name = [task.name for task in Tasks] - assert tasks_name.sort() == ['classification', 'regression'].sort() - # get by enum - assert get_operator(Tasks.CLASSIFICATION) is predictions_operator - assert get_operator(Tasks.REGRESSION) is regression_operator - - # get by string - assert get_operator("classification") is predictions_operator - assert get_operator("regression") is regression_operator - - # assert a not valid string does not work - with pytest.raises(AssertionError): - get_operator("random") - - # operator must have at least 3 arguments - function_with_2_arguments = lambda x,y: 0 - - # operator must be Callable - not_a_function = [1, 2, 3] - - for operator in [function_with_2_arguments, not_a_function]: - try: - get_operator(operator) - except InvalidOperatorException: - pass - - -def test_regression_operator(): - input_shape, nb_labels, samples = ((10, 10, 1), 10, 20) +def test_predictions_operator(): + input_shape, nb_labels, samples = ((10, 10, 3), 10, 20) x, y = generate_data(input_shape, nb_labels, samples) - regression_model = generate_regression_model(input_shape, nb_labels) + classification_model = generate_model(input_shape, nb_labels) - methods = default_methods(regression_model, regression_operator) + methods = default_methods(classification_model, regression_operator) for method in methods: assert hasattr(method, 'inference_function') @@ -120,18 +57,15 @@ def test_regression_operator(): phis = method(x, y) assert x.shape[:-1] == phis.shape[:3] - -def test_segmentation_operator(): - segmentation_model = get_segmentation_model() - x, y = generate_data((20, 20, 3), 10, 10) - def segmentation_operator(model, x, y): - # explaining channel 0 - return tf.reduce_sum(model(x)[:,:,0], (1, 2)) +def test_regression_operator(): + input_shape, nb_labels, samples = ((10, 10, 1), 10, 20) + x, y = generate_data(input_shape, nb_labels, samples) + regression_model = generate_regression_model(input_shape, nb_labels) - methods = default_methods(segmentation_model, segmentation_operator) + methods = default_methods(regression_model, regression_operator) for method in methods: assert hasattr(method, 'inference_function') @@ -153,7 +87,6 @@ def test_concept_operator(): def concept_operator(model, x, y): x = tf.reshape(x, (-1, 20*20)) - print(x.shape, random_projection.shape) ui = x @ random_projection return tf.reduce_sum(model(ui) * y, axis=-1) diff --git a/tests/commons/test_operators_operations.py b/tests/commons/test_operators_operations.py new file mode 100644 index 00000000..92e79abd --- /dev/null +++ b/tests/commons/test_operators_operations.py @@ -0,0 +1,76 @@ +""" +Ensure we can use the operator functionnality on various models +""" + +import pytest + +import xplique +from xplique.commons.operators_operations import check_operator, Tasks, get_operator +from xplique.commons.operators import (predictions_operator, regression_operator, + semantic_segmentation_operator, object_detection_operator) +from xplique.commons.exceptions import InvalidOperatorException + + +def test_check_operator(): + # ensure that the check operator detects non-operator + + # operator must have at least 3 arguments + function_with_2_arguments = lambda x,y: 0 + + # operator must be Callable + not_a_function = [1, 2, 3] + + for operator in [function_with_2_arguments, not_a_function]: + try: + check_operator(operator) + assert False + except InvalidOperatorException: + pass + + +def test_get_operator(): + possible_tasks = ["classification", "regression", "semantic segmentation", "object detection", + "object detection box position", "object detection box proba", + "object detection box class"] + + tasks_name = [task.name for task in Tasks] + assert tasks_name.sort() == possible_tasks.sort() + + # get by enum + assert get_operator(Tasks.CLASSIFICATION) is predictions_operator + assert get_operator(Tasks.REGRESSION) is predictions_operator # TODO, change when there is a real regression operator + assert get_operator(Tasks.OBJECT_DETECTION) is object_detection_operator + assert get_operator(Tasks.SEMANTIC_SEGMENTATION) is semantic_segmentation_operator + + # get by string + assert get_operator("classification") is predictions_operator + assert get_operator("regression") is predictions_operator # TODO, change when there is a real regression operator + assert get_operator("object detection") is object_detection_operator + assert get_operator("semantic segmentation") is semantic_segmentation_operator + + # assert a not valid string does not work + with pytest.raises(AssertionError): + get_operator("random") + + # operator must have at least 3 arguments + function_with_2_arguments = lambda x,y: 0 + + # operator must be Callable + not_a_function = [1, 2, 3] + + for operator in [function_with_2_arguments, not_a_function]: + try: + get_operator(operator) + except InvalidOperatorException: + pass + + +def test_proposed_operators(): + # ensure all proposed operators are operators + for operator in [task.value for task in Tasks]: + check_operator(operator) + +def test_enum_shortcut(): + # ensure all proposed operators are operators + for operator in [task.value for task in xplique.Tasks]: + check_operator(operator) diff --git a/tests/commons/test_segmentation_operator.py b/tests/commons/test_segmentation_operator.py new file mode 100644 index 00000000..c51df8c0 --- /dev/null +++ b/tests/commons/test_segmentation_operator.py @@ -0,0 +1,109 @@ +""" +Ensure we can use the operator functionality on various models +""" + +import numpy as np +import tensorflow as tf + +from xplique.attributions import (Saliency, GradientInput, IntegratedGradients, SmoothGrad, VarGrad, + SquareGrad, GradCAM, Occlusion, Rise, GuidedBackprop, DeconvNet, + GradCAMPP, Lime, KernelShap, SobolAttributionMethod, + HsicAttributionMethod) +from xplique.commons.operators_operations import semantic_segmentation_operator +from ..utils import generate_data, almost_equal + + +def default_methods(model, operator): + return [ + Saliency(model, operator=operator), + GradientInput(model, operator=operator), + SmoothGrad(model, operator=operator), + VarGrad(model, operator=operator), + SquareGrad(model, operator=operator), + IntegratedGradients(model, operator=operator), + Occlusion(model, operator=operator), + Rise(model, operator=operator, nb_samples=2), + GuidedBackprop(model, operator=operator), + DeconvNet(model, operator=operator), + SobolAttributionMethod(model, operator=operator, grid_size=2, nb_design=2), + HsicAttributionMethod(model, operator=operator, grid_size=2, nb_design=2), + ] + + +def get_segmentation_model(input_shape=(20, 20, 1)): + model = tf.keras.Sequential([ + tf.keras.layers.Input(input_shape), + ]) + model.compile() + return model + + +def test_segmentation_operator(): + input_shape = (20, 20, 3) + segmentation_model = get_segmentation_model(input_shape) + + x, _ = generate_data(input_shape, 10, 10) + y, _ = generate_data(input_shape, 10, 10) + + methods = default_methods(segmentation_model, semantic_segmentation_operator) + for method in methods: + + assert hasattr(method, 'inference_function') + assert hasattr(method, 'batch_inference_function') + assert hasattr(method, 'gradient') + assert hasattr(method, 'batch_gradient') + + phis = method(x, y) + + assert x.shape[:-1] == phis.shape[:3] + + +def test_segmentation_operator_computation(): + image = [[[[0, 0, 1.0], + [0, 0, 1.0], + [1.0, 1.0, 1.0],], + [[0, 0, 0], + [0, 1.0, 0], + [1.0, 1.0, 1.0],], + [[0, 0, 0], + [0, 0, 0], + [1.0, 0, 0],],]] + image = tf.transpose(tf.convert_to_tensor(image, tf.float32), perm=[0, 2, 3, 1]) + + model = lambda x: tf.concat([x, tf.expand_dims(tf.reduce_mean(x, axis=-1), axis=-1)], axis=-1) + + target_1 = [[[[0, 0, 1.0], + [0, 0, 1.0], + [1.0, 1.0, 1.0],], + [[0, 0, 0], + [0, 0, 0], + [0, 0, 0],], + [[0, 0, 0], + [0, 0, 0], + [0, 0, 0],], + [[0, 0, 0], + [0, 0, 0], + [0, 0, 0],],]] + target_1 = tf.transpose(tf.convert_to_tensor(target_1, tf.float32), perm=[0, 2, 3, 1]) + + target_2 = [[[[0, 0, 0], + [0, 0, 0], + [0, 0, 0],], + [[0, 0, 0], + [0, 0, 0], + [0, 0, 0],], + [[0, 0, 0], + [0, 0, 0], + [0, 0, 0],], + [[0, 0, 0], + [0, 0, 0], + [1.0, 1.0, 1.0],],]] + target_2 = tf.transpose(tf.convert_to_tensor(target_2, tf.float32), perm=[0, 2, 3, 1]) + + scores = model(np.array(image)) * target_2 + + score_1 = semantic_segmentation_operator(model, image, np.array(target_1)) + score_2 = semantic_segmentation_operator(model, np.array(image), target_2) + + assert almost_equal(score_1, 1.0) + assert almost_equal(score_2, (7.0 / 3) / 3) diff --git a/tests/utils.py b/tests/utils.py index f9457372..ebbf13bd 100644 --- a/tests/utils.py +++ b/tests/utils.py @@ -2,7 +2,8 @@ from sklearn.linear_model import LinearRegression import tensorflow as tf from tensorflow.keras.models import Sequential, Model -from tensorflow.keras.layers import Dense, Conv1D, Conv2D, Activation, GlobalAveragePooling1D, Dropout, Flatten, MaxPooling2D, Input +from tensorflow.keras.layers import (Dense, Conv1D, Conv2D, Activation, GlobalAveragePooling1D, + Dropout, Flatten, MaxPooling2D, Input, Reshape) from tensorflow.keras.utils import to_categorical def generate_data(x_shape=(32, 32, 3), num_labels=10, samples=100): @@ -114,3 +115,43 @@ def __call__(self, inputs): return tf_model +def generate_object_detection_model(input_shape=(32, 32, 3), max_nb_boxes=10, nb_labels=5, with_nmf=False): + # create a model that generates max_nb_boxes and select some randomly + output_shape = (max_nb_boxes, 5 + nb_labels) + model = Sequential() + model.add(Input(shape=input_shape)) + model.add(Conv2D(4, kernel_size=(2, 2), + activation='relu')) + model.add(MaxPooling2D(pool_size=(2, 2))) + model.add(Flatten()) + model.add(Dense(np.prod(output_shape))) + model.add(Reshape(output_shape)) + model.add(Activation('sigmoid')) + model.compile(loss='mae', optimizer='sgd') + + # ensure iou computation will work + def make_plausible_boxes(model_output): + coordinates = tf.sort(model_output[:, :, :4], axis=-1) * 200 + probabilities = model_output[:, :, 4][:, :, tf.newaxis] + classifications = tf.nn.softmax(model_output[:, :, 5:], axis=-1) + new_output = tf.concat([coordinates, probabilities, classifications], axis=-1) + return new_output + + valid_model = lambda inputs: make_plausible_boxes(model(inputs)) + + # equivalent of nmf + def randomly_select_boxes(boxes): + boxes_ids = tf.range(tf.shape(boxes)[0]) + nb_boxes = tf.experimental.numpy.random.randint(1, max_nb_boxes) + boxes_ids = tf.random.shuffle(boxes_ids)[:nb_boxes] + return tf.gather(boxes, boxes_ids) + + # model with nmf + def model_with_random_nb_boxes(input): + all_boxes = valid_model(input) + some_boxes = [randomly_select_boxes(boxes) for boxes in all_boxes] + return some_boxes + + if with_nmf: + return model_with_random_nb_boxes + return valid_model diff --git a/tests/utils_functions/__init__.py b/tests/utils_functions/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/utils_functions/test_object_detection.py b/tests/utils_functions/test_object_detection.py new file mode 100644 index 00000000..f44b1609 --- /dev/null +++ b/tests/utils_functions/test_object_detection.py @@ -0,0 +1,34 @@ +""" +Test utils functions for object detection +""" + +import numpy as np + +from xplique.utils_functions.object_detection import _box_iou + +from ..utils import almost_equal + + +def test_iou_mask(): + """Assert the Mask IoU calculation is ok""" + dtype = np.float32 + + m1 = np.array([ + [10, 20, 30, 40] + ], dtype=dtype) + m2 = np.array([ + [15, 20, 30, 40] + ], dtype=dtype) + m3 = np.array([ + [0, 20, 10, 40] + ], dtype=dtype) + m4 = np.array([ + [0, 0, 100, 100] + ], dtype=dtype) + + assert almost_equal(_box_iou(m1, m2), 300.0 / 400.0) + assert almost_equal(_box_iou(m1, m3), 0.0) + assert almost_equal(_box_iou(m3, m2), 0.0) + assert almost_equal(_box_iou(m1, m4), 400.0 / 10_000) + assert almost_equal(_box_iou(m2, m4), 300.0 / 10_000) + assert almost_equal(_box_iou(m3, m4), 200.0 / 10_000) diff --git a/tests/utils_functions/test_segmentation.py b/tests/utils_functions/test_segmentation.py new file mode 100644 index 00000000..7e3020f8 --- /dev/null +++ b/tests/utils_functions/test_segmentation.py @@ -0,0 +1,212 @@ +import tensorflow as tf + +from xplique.utils_functions.segmentation import * + +def get_prediction(): + predictions = [[[0.6, 0.6, 0.6, 0.2, 0.2], + [0.6, 0.6, 0.2, 0.2, 0.2], + [0.6, 0.2, 0.2, 0.2, 0.2], + [0.2, 0.2, 0.2, 0.2, 0.2], + [0.2, 0.2, 0.2, 0.2, 0.2],], + [[0.2, 0.2, 0.2, 0.6, 0.6], + [0.2, 0.2, 0.6, 0.6, 0.6], + [0.2, 0.6, 0.6, 0.6, 0.2], + [0.6, 0.6, 0.6, 0.2, 0.2], + [0.6, 0.6, 0.2, 0.2, 0.2],], + [[0.2, 0.2, 0.2, 0.2, 0.2], + [0.2, 0.2, 0.2, 0.2, 0.2], + [0.2, 0.2, 0.2, 0.2, 0.6], + [0.2, 0.2, 0.2, 0.6, 0.6], + [0.2, 0.2, 0.6, 0.6, 0.6],],] + + predictions = tf.convert_to_tensor(predictions, tf.float32) + + predictions = tf.transpose(predictions, perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(tf.reduce_sum(predictions, axis=-1), tf.ones((5, 5)))) + + return predictions + + +def test_get_class_zone(): + predictions = get_prediction() + + target_1 = get_class_zone(predictions, class_id=1) + + expected_target = tf.transpose(tf.convert_to_tensor( + [[[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],], + [[0, 0, 0, 1, 1], + [0, 0, 1, 1, 1], + [0, 1, 1, 1, 0], + [1, 1, 1, 0, 0], + [1, 1, 0, 0, 0],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],],tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(target_1, expected_target)) + + target_0 = get_class_zone(predictions, class_id=0) + target_2 = get_class_zone(predictions, class_id=2) + + assert tf.reduce_all(tf.equal( + tf.reduce_sum(tf.stack([target_0, target_1, target_2], axis=0), axis=[0, 3]), + tf.fill((5, 5), 1.0))) + + +def test_get_connected_zone(): + predictions = get_prediction() + + target_1 = get_connected_zone(predictions, coordinates=(2, 2)) + + expected_target = tf.transpose(tf.convert_to_tensor( + [[[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],], + [[0, 0, 0, 1, 1], + [0, 0, 1, 1, 1], + [0, 1, 1, 1, 0], + [1, 1, 1, 0, 0], + [1, 1, 0, 0, 0],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],],tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(target_1, expected_target)) + + target_0 = get_connected_zone(predictions, coordinates=(0, 0)) + target_2 = get_connected_zone(predictions, coordinates=(4, 4)) + + assert tf.reduce_all(tf.equal( + tf.reduce_sum(tf.stack([target_0, target_1, target_2], axis=0), axis=[0, 3]), + tf.fill((5, 5), 1.0))) + + +def test_list_class_connected_zones(): + predictions = get_prediction() + + predictions = tf.stack([predictions[:, :, 0] + predictions[:, :, 2], predictions[:, :, 1]], axis=-1) + + zones_0 = list_class_connected_zones(predictions, class_id=0, zone_minimum_size=1) + zones_1 = list_class_connected_zones(predictions, class_id=1, zone_minimum_size=1) + no_zones = list_class_connected_zones(predictions, class_id=0, zone_minimum_size=10) + + assert len(zones_0) == 2 + assert len(zones_1) == 1 + assert len(no_zones) == 0 + + expected_zones_1 = tf.transpose(tf.convert_to_tensor( + [[[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],], + [[0, 0, 0, 1, 1], + [0, 0, 1, 1, 1], + [0, 1, 1, 1, 0], + [1, 1, 1, 0, 0], + [1, 1, 0, 0, 0],],], tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(zones_1[0], expected_zones_1)) + + expected_zones_21 = tf.transpose(tf.convert_to_tensor( + [[[1, 1, 1, 0, 0], + [1, 1, 0, 0, 0], + [1, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],], tf.float32), perm=[1, 2, 0]) + + expected_zones_22 = tf.transpose(tf.convert_to_tensor( + [[[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 1], + [0, 0, 0, 1, 1], + [0, 0, 1, 1, 1],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],], tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(zones_0[0], expected_zones_21))\ + or tf.reduce_all(tf.equal(zones_0[0], expected_zones_22)) + + assert tf.reduce_all(tf.equal(zones_0[1], expected_zones_21))\ + or tf.reduce_all(tf.equal(zones_0[1], expected_zones_22)) + + + +def test_get_in_out_border(): + predictions = get_prediction() + + central_zone = get_connected_zone(predictions, coordinates=(2, 2)) + + borders = get_in_out_border(central_zone) + + expected_borders = tf.transpose(tf.convert_to_tensor( + [[[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],], + [[ 0, -1, -1,1, 0], + [-1, -1, 1, 1, 1], + [-1, 1, 1,1 , -1], + [ 1, 1, 1, -1, -1], + [ 0,1, -1, -1, 0],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],],tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(borders, expected_borders)) + + +def test_get_common_border(): + predictions = get_prediction() + + left_corner_zone = get_connected_zone(predictions, coordinates=(0, 0)) + central_zone = get_connected_zone(predictions, coordinates=(2, 2)) + + left_corner_borders = get_in_out_border(left_corner_zone) + central_borders = get_in_out_border(central_zone) + + common_borders_0 = get_common_border(left_corner_borders, central_borders) + common_borders_1 = get_common_border(central_borders, left_corner_borders) + + expected_common_borders = tf.transpose(tf.convert_to_tensor( + [[[ 0, 1, 1, -1, 0], + [ 1, 1, -1, -1, 0], + [ 1, -1, -1, 0, 0], + [-1, -1, 0, 0, 0], + [ 0, 0, 0, 0, 0],], + [[ 0, -1, -1, 1, 0], + [-1, -1, 1, 1, 0], + [-1, 1, 1, 0, 0], + [ 1, 1, 0, 0, 0], + [ 0, 0, 0, 0, 0],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],],tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(common_borders_0, common_borders_1)) + + assert tf.reduce_all(tf.equal(common_borders_0, expected_common_borders)) \ No newline at end of file diff --git a/xplique/__init__.py b/xplique/__init__.py index daad8dfc..d3ea3182 100644 --- a/xplique/__init__.py +++ b/xplique/__init__.py @@ -6,10 +6,12 @@ techniques """ -__version__ = '1.1.0' +__version__ = '1.2.0' from . import attributions from . import concepts from . import features_visualizations from . import commons from . import plots + +from .commons import Tasks diff --git a/xplique/attributions/object_detector.py b/xplique/attributions/object_detector.py index 760e4305..cc434806 100644 --- a/xplique/attributions/object_detector.py +++ b/xplique/attributions/object_detector.py @@ -2,268 +2,108 @@ Module related to Object detector method """ -from typing import Iterable, Tuple, Union, Optional -import abc +from deprecated import deprecated -import tensorflow as tf import numpy as np +import tensorflow as tf -from xplique.attributions.base import BlackBoxExplainer -from xplique.commons import operator_batching +from ..types import Optional, Callable, Union +from .base import BlackBoxExplainer, WhiteBoxExplainer +from ..commons import get_gradient_functions +from ..commons.operators import object_detection_operator +from ..utils_functions.object_detection import _box_iou, _format_objects -class IouCalculator: - """ - Used to compute the Intersection Over Union (IOU). - """ - @abc.abstractmethod - def intersect(self, objects_a: tf.Tensor, objects_b: tf.Tensor) -> tf.Tensor: - """ - Compute the intersection between two batched objects (e.g boxes, segmentation masks...) - Parameters - ---------- - objects_a - First batch of objects to compare. - objects_b - Second batch of objects to compare. +OLD_OBJECT_DETECTION_DEPRECATION_MESSAGE = """ +\n +The method to compute attribution explanation explanations changed drastically after version 1.0.0. +For more information please refer to the documentation: +https://deel-ai.github.io/xplique/latest/api/attributions/object_detection/ - Returns - ------- - score - Real value between [0,1] corresponding to the intersection of the 2 objects. - """ - raise NotImplementedError() +Nonetheless, here is a quick example of how it is used now: +``` +from xplique.attributions import AnyMethod +explainer = AnyMethod(model, operator="object detection", ...) +explanation = explainer(inputs, targets) # be aware, targets are specific to object detection here +``` +""" -class SegmentationIouCalculator(IouCalculator): - """ - Compute segmentation masks IOU. +class BoundingBoxesExplainer(BlackBoxExplainer): """ - def intersect(self, masks_a: tf.Tensor, masks_b: tf.Tensor) -> tf.Tensor: - """ - Compute the intersection between two batched segmentation masks. - Each segmentation is a boolean mask on the whole image - - Parameters - ---------- - masks_a - First batch of segmentation masks. - masks_b - Second batch of segmentation masks. - - Returns - ------- - iou_score - The IOU score between the first and second batch of masks. - """ - # pylint: disable=W0221,W0237 - - axis = np.arange(1, tf.rank(masks_a)) - - inter_area = tf.reduce_sum(tf.cast(tf.logical_and(masks_a, masks_b), - dtype=tf.float32), axis=axis) - union_area = tf.reduce_sum(tf.cast(tf.logical_or(masks_a, masks_b), - dtype=tf.float32), axis=axis) - - iou_score = inter_area / tf.maximum(union_area, 1.0) - - return iou_score + For a given black box explainer, this class allows to find explications of an object detector + model. The object model detector shall return a list (length of the size of the batch) + containing a tensor of 2 dimensions. + The first dimension of the tensor is the number of bounding boxes found in the image + The second dimension is: + [xmin, ymin, xmax, ymax, probability_detection, ones_hot_classif_result] + This work is a generalization of the following article at any kind of black box explainer and + also can be used for other kind of object detector (like segmentation) + Ref. Petsiuk & al., Black-box Explanation of Object Detectors via Saliency Maps (2021). + https://arxiv.org/pdf/2006.03204.pdf -class BoxIouCalculator(IouCalculator): - """ - Used to compute the Bounding Box IOU. + Parameters + ---------- + explainer + the black box explainer used to explain the object detector model + _ + inheritance from old versions + intersection_score + the iou calculator used to compare two objects. """ - EPSILON = tf.constant(1e-4) - - def intersect(self, boxes_a: tf.Tensor, boxes_b: tf.Tensor) -> tf.Tensor: - """ - Compute the intersection between two batched bounding boxes. - Each bounding box is defined by (x1, y1, x2, y2) respectively (left, bottom, right, top). - - Parameters - ---------- - boxes_a - First batch of bounding boxes. - boxes_b - Second batch of bounding boxes. - - Returns - ------- - iou_score - The IOU score between the first and second batch of bounding boxes. - """ - # pylint: disable=W0221,W0237 - - # determine the intersection rectangle - left = tf.maximum(boxes_a[..., 0], boxes_b[..., 0]) - bottom = tf.maximum(boxes_a[..., 1], boxes_b[..., 1]) - right = tf.minimum(boxes_a[..., 2], boxes_b[..., 2]) - top = tf.minimum(boxes_a[..., 3], boxes_b[..., 3]) - - intersection_area = tf.math.maximum(right - left, 0) * tf.math.maximum(top - bottom, 0) - - # determine the areas of the prediction and ground-truth rectangles - a_area = (boxes_a[..., 2] - boxes_a[..., 0]) * (boxes_a[..., 3] - boxes_a[..., 1]) - b_area = (boxes_b[..., 2] - boxes_b[..., 0]) * (boxes_b[..., 3] - boxes_b[..., 1]) - - union_area = a_area + b_area - intersection_area - - iou_score = intersection_area / (union_area + BoxIouCalculator.EPSILON) - - return iou_score - - -class IObjectFormater: - """ - Generic class to format the model prediction - """ - def format_objects(self, predictions) -> Iterable[Tuple[tf.Tensor, tf.Tensor, tf.Tensor]]: - """ - Format the model prediction of a given image to have the prediction of the following format: - objects, proba_detection, one_hots_classifications - - Parameters - ---------- - predictions - prediction of the model of a given image - - Returns - ------- - object - bounding box or mask component of the prediction - proba - existence probability component of the prediction - classification - classification component of the prediction - """ - raise NotImplementedError() + @deprecated(version="1.0.0", reason=OLD_OBJECT_DETECTION_DEPRECATION_MESSAGE) + def __init__(self, + explainer: BlackBoxExplainer, + _: Optional[Callable] = _format_objects, + intersection_score: Optional[Callable] = _box_iou): + # make operator function based on arguments + operator = lambda model, inputs, targets: \ + object_detection_operator(model, inputs, targets, intersection_score) + + # BlackBoxExplainer init to set operator + super().__init__(model=explainer.model, batch_size=explainer.batch_size, operator=operator) + self.explainer = explainer -class ImageObjectDetectorScoreCalculator: - """ - Class to compute batch score - """ - def __init__(self, object_formater: IObjectFormater, iou_calculator: IouCalculator): - self.object_formater = object_formater - self.iou_calculator = iou_calculator + # update explainer inference functions for explain method + self.explainer.inference_function = self.inference_function + self.explainer.batch_inference_function = self.batch_inference_function - self.batch_score = operator_batching(self.score) + if isinstance(self.explainer, WhiteBoxExplainer): + # check and get gradient function from model and operator + self.gradient, self.batch_gradient = get_gradient_functions(self.model, operator) + self.explainer.gradient = self.gradient + self.explainer.batch_gradient = self.batch_gradient - def score(self, model, inp, object_ref) -> tf.Tensor: + def explain(self, + inputs: Union[tf.data.Dataset, tf.Tensor, np.array], + targets: Optional[Union[tf.Tensor, np.array]] = None) -> tf.Tensor: """ - Compute the matching score between prediction and a given object + Compute the explanation of the object detection through the explainer Parameters ---------- - model - the model used for the object detection - inp - the batched image - object_ref - the object target to compare with the prediction of the model + inputs + Dataset, Tensor or Array. Input samples to be explained. + If Dataset, targets should not be provided (included in Dataset). + Expected shape (N, H, W, C). + More information in the documentation. + targets + Tensor or Array. One-hot encoding of the model's output from which an explanation + is desired. One encoding per input and only one output at a time. Therefore, + the expected shape is (N, ...). With features matching the object formatting. + See object detection operator documentation for more information + More information in the documentation. Returns ------- - score - for each image, the matching score between the object of reference and - the prediction of the model - """ - objects = model(inp) - score_values = [] - for obj, obj_ref in zip(objects, object_ref): - if obj is None or obj.shape[0] == 0: - score_values.append(tf.constant(0.0, dtype=inp.dtype)) - else: - current_boxes, proba_detection, classification = \ - self.object_formater.format_objects(obj) - - if len(tf.shape(obj_ref)) == 1: - obj_ref = tf.expand_dims(obj_ref, axis=0) - - obj_ref = self.object_formater.format_objects(obj_ref) - - scores = [] - size = tf.shape(current_boxes)[0] - for boxes_ref, proba_ref, class_ref in zip(*obj_ref): - boxes_ref = tf.repeat(tf.expand_dims(boxes_ref, axis=0), repeats=size, axis=0) - proba_ref = tf.repeat(tf.expand_dims(proba_ref, axis=0), repeats=size, axis=0) - class_ref = tf.repeat(tf.expand_dims(class_ref, axis=0), repeats=size, axis=0) - - iou = self.iou_calculator.intersect(boxes_ref, current_boxes) - classification_similarity = tf.reduce_sum(class_ref * classification, axis=1) \ - / (tf.norm(classification, axis=1) * tf.norm(class_ref, axis=1)) - - current_score = iou * tf.squeeze(proba_detection, axis=1) \ - * classification_similarity - current_score = tf.reduce_max(current_score) - scores.append(current_score) - - score_value = tf.reduce_max(tf.stack(scores)) - score_values.append(score_value) - - score_values = tf.stack(score_values) - - return score_values - - -class ImageObjectDetectorExplainer(BlackBoxExplainer): - """ - Used to define method as an object detector one - """ - - def __init__(self, explainer: BlackBoxExplainer, object_detector_formater: IObjectFormater, - iou_calculator: IouCalculator): + explanation + The resulting object detection explanation """ - Constructor - - Parameters - ---------- - explainer - the black box explainer used to explain the object detector model - object_detector_formater - the formater of the object detector model used to format the prediction - of the right format - iou_calculator - the iou calculator used to compare two objects. - """ - super().__init__(explainer.model, explainer.batch_size) - self.explainer = explainer - self.score_calculator = ImageObjectDetectorScoreCalculator(object_detector_formater, - iou_calculator) - self.explainer.inference_function = self.score_calculator.score - self.explainer.batch_inference_function = self.score_calculator.batch_score - - def explain(self, inputs: Union[tf.data.Dataset, tf.Tensor, np.array], - targets: Optional[Union[tf.Tensor, np.array]] = None) -> tf.Tensor: if len(tf.shape(targets)) == 1: targets = tf.expand_dims(targets, axis=0) return self.explainer.explain(inputs, targets) - - -class BoundingBoxesExplainer(ImageObjectDetectorExplainer, IObjectFormater): - """ - For a given black box explainer, this class allows to find explications of an object detector - model. The object model detector shall return a list (length of the size of the batch) - containing a tensor of 2 dimensions. - The first dimension of the tensor is the number of bounding boxes found in the image - The second dimension is: - [x1_box, y1_box, x2_box, y2_box, probability_detection, ones_hot_classif_result] - - This work is a generalisation of the following article at any kind of black box explainer and - also can be used for other kind of object detector (like segmentation) - - Ref. Petsiuk & al., Black-box Explanation of Object Detectors via Saliency Maps (2021). - https://arxiv.org/pdf/2006.03204.pdf - """ - - def __init__(self, explainer: BlackBoxExplainer): - super().__init__(explainer, self, BoxIouCalculator()) - - def format_objects(self, predictions) -> Iterable[Tuple[tf.Tensor, tf.Tensor, tf.Tensor]]: - boxes, proba_detection, one_hots_classifications = tf.split(predictions, - [4, 1, tf.shape(predictions[0])[0] - 5], 1) - return boxes, proba_detection, one_hots_classifications diff --git a/xplique/commons/__init__.py b/xplique/commons/__init__.py index 3dfa73af..94237f90 100644 --- a/xplique/commons/__init__.py +++ b/xplique/commons/__init__.py @@ -7,7 +7,7 @@ find_layer, open_relu_policy from .tf_operations import repeat_labels, batch_tensor from .callable_operations import predictions_one_hot_callable -from .operators import Tasks, get_operator, check_operator, operator_batching,\ - get_inference_function, get_gradient_functions +from .operators_operations import (Tasks, get_operator, check_operator, operator_batching, + get_inference_function, get_gradient_functions) from .exceptions import no_gradients_available, raise_invalid_operator from .forgrad import forgrad diff --git a/xplique/commons/operators.py b/xplique/commons/operators.py index 62af7a1f..5770420e 100644 --- a/xplique/commons/operators.py +++ b/xplique/commons/operators.py @@ -2,14 +2,13 @@ Custom tensorflow operator for Attributions """ -import inspect -from enum import Enum +from deprecated import deprecated import tensorflow as tf -from ..types import Callable, Optional, Union, OperatorSignature -from .exceptions import raise_invalid_operator, no_gradients_available -from .callable_operations import predictions_one_hot_callable +from ..types import Callable, Optional +from ..utils_functions.object_detection import _box_iou, _format_objects, _EPSILON + @tf.function def predictions_operator(model: Callable, @@ -36,6 +35,7 @@ def predictions_operator(model: Callable, return scores @tf.function +@deprecated(version="1.0.0", reason="Gradient-based explanations are zeros with this operator.") def regression_operator(model: Callable, inputs: tf.Tensor, targets: tf.Tensor) -> tf.Tensor: @@ -63,294 +63,162 @@ def regression_operator(model: Callable, @tf.function -def binary_segmentation_operator(model: Callable, - inputs: tf.Tensor, - targets: tf.Tensor) -> tf.Tensor: +def semantic_segmentation_operator(model, inputs, targets): """ - Compute the segmentation score for a batch of samples. + Explain the class of a zone of interest. Parameters ---------- model Model used for computing predictions. + The model outputs should be between 0 and 1, otherwise, applying a softmax is recommended. inputs Input samples to be explained. + Expected shape of (n, h, w, c_in), with c_in the number of channels of the input. targets - One-hot encoded labels or regression target (e.g {+1, -1}), one for each sample. + Tensor, a mask indicating the zone and class to explain. + It contains the model predictions limited to a certain zone and channel. + The zone indicates the zone of interest and the channel the class of interest. + For more detail and examples please refer to the documentation. + https://deel-ai.github.io/xplique/latest/api/attributions/semantic_segmentation/ + Expected shape of (n, h, w, c_out), with c_out the number of classes. + `targets` can also be designed to explain the border of a zone of interest. Returns ------- scores Segmentation scores computed. """ - scores = tf.reduce_sum(model(inputs) * targets, axis=(1, 2)) - return scores + # compute absolute difference between prediction and target on targets zone + scores = model(inputs) * targets + + # take mean over the zone and channel of interest + return tf.reduce_sum(scores, axis=(1, 2, 3)) /\ + tf.reduce_sum(tf.cast(tf.not_equal(targets, 0), tf.float32), axis=(1, 2, 3)) @tf.function -def segmentation_operator(model: Callable, - inputs: tf.Tensor, - targets: tf.Tensor) -> tf.Tensor: - """ - Compute the segmentation score for a batch of samples. +def object_detection_operator(model: Callable, + inputs: tf.Tensor, + targets: tf.Tensor, + intersection_score_fn: Optional[Callable] = _box_iou, + include_detection_probability: Optional[bool] = True, + include_classification_score: Optional[bool] = True,) -> tf.Tensor: + """ + Compute the object detection scores for a batch of samples. + + For a given image, there are two possibilities: + - One box per image is provided: Then, in the case of perturbation-based methods, + the model makes prediction on the perturbed image and choose the most similar predicted box. + This similarity is computed following the DRise method. + In the case of gradient-based methods, the gradient is computed from the same score. + - Several boxes are provided for one image: In this case, the attributions for each box are + computed and the mean is taken. + + Therefore, to explain each box separately, the easiest way is to call the attribution method + with a batch of the same image tiled to match the number of predicted box. + In this case, inputs and targets shapes should be: (nb_boxes, H, W, C) and (nb_boxes, (5 + nc)). + + This work is a generalization of the following article at any kind of attribution method. + Ref. Petsiuk & al., Black-box Explanation of Object Detectors via Saliency Maps (2021). + https://arxiv.org/pdf/2006.03204.pdf Parameters ---------- model - Model used for computing predictions. + Model used for computing object detection prediction. + The model should have input and output shapes of (N, H, W, C) and (N, nb_boxes, (4+1+nc)). + The model should not include the NMS computation, + it is not differentiable and drastically reduce the number of boxes for the matching. inputs - Input samples to be explained. + Batched input samples to be explained. Expected shape (N, H, W, C). + More information in the documentation. targets - One-hot encoded labels or regression target (e.g {+1, -1}), one for each sample. + Specify the box are boxes to explain for each input. Preferably, after the NMS. + It should be of shape (N, (4 + 1 + nc)) or (N, nb_boxes, (4 + 1 + nc)), + with nc the number of classes, + N the number of samples in the batch (it should match `inputs`), + and nb_boxes the number of boxes to explain simultaneously. + + (4 + 1 + nc) means: [boxes_coordinates, proba_detection, one_hots_classifications]. + + In the case the nb_boxes dimension is not 1, + several boxes will be explained at the same time. + To be more precise, explanations will be computed for each box and the mean is returned. + intersection_score_fn + Function that computes the intersection score between two bounding boxes coordinates. + This function is batched. The default value is `_box_iou` computing IOU scores. + include_detection_probability + Boolean encoding if the box objectness (or detection probability) + should be included in DRise score. + include_classification_score + Boolean encoding if the class associated to the box should be included in DRise score. Returns ------- scores - Segmentation scores computed. - """ - scores = tf.reduce_sum(model(inputs) * targets, axis=(1, 2, 3)) - return scores - -class Tasks(Enum): - """ - Enumeration of different tasks for which we have defined operators - """ - CLASSIFICATION = predictions_operator - REGRESSION = regression_operator - - @staticmethod - def from_string(operator_name: str) -> "Tasks": - """ - Restore an operator from a string - - Parameters - ---------- - operator_name - String indicating the operator to restore: must be one - of 'classification' or 'regression' - - Returns - ------- - operator - The Tasks object - """ - assert operator_name in [ - "classification", - "regression", - ], "Only 'classification' and 'regression' are supported." - - if operator_name == "regression": - return Tasks.REGRESSION - return Tasks.CLASSIFICATION - -def check_operator(operator: Callable): - """ - Check if the operator is valid g(f, x, y) -> tf.Tensor - and raise an exception and return true if so. - - Parameters - ---------- - operator - Operator to check - - Returns - ------- - is_valid - True if the operator is valid, False otherwise. - """ - # handle tf functions - # pylint: disable=protected-access - if hasattr(operator, '_python_function'): - return check_operator(operator._python_function) - - # the operator must be callable - if not hasattr(operator, '__call__'): - raise_invalid_operator() - - # the operator should take at least three arguments - args = inspect.getfullargspec(operator).args - if len(args) < 3: - raise_invalid_operator() - - return True - -def get_operator( - operator: Optional[Union[Tasks, str, OperatorSignature]]): - """ - This function allows to retrieve an operator from: a Tasks, a task name. If the operator - is a custom one, we simply check if its signature is correct - - Parameters - ---------- - operator - An operator from the Tasks enum or the task name or a custom operator. If None, use a - classification operator. - - Returns - ------- - operator - The operator requested - """ - # case when no operator is provided - if operator is None: - return predictions_operator - - # case when the query is a string - if isinstance(operator, str): - return Tasks.from_string(operator) - - # case when the query belong to the Tasks enum - if operator in [t.value for t in Tasks]: - return operator - - # case when the operator is a custom one - assert check_operator(operator) - return operator - -def get_gradient_of_operator(operator): - """ - Get the gradient of an operator. - - Parameters - ---------- - operator - Operator to compute the gradient of. - - Returns - ------- - gradient - Gradient of the operator. - """ - @tf.function - def gradient(model, inputs, targets): - with tf.GradientTape() as tape: - tape.watch(inputs) - scores = operator(model, inputs, targets) - - return tape.gradient(scores, inputs) - - return gradient - - -def operator_batching(operator: OperatorSignature) -> tf.Tensor: - """ - Take care of batching an operator: (model, inputs, labels). - - Parameters - ---------- - operator - Any callable that take model, inputs and labels as parameters. - - Returns - ------- - batched_operator - Function that apply operator by batch. + Object detection scores computed following DRise definition: + intersection_score * proba_detection * classification_similarity """ + def batch_loop(args): + # function to loop on for `tf.map_fn` + obj, obj_ref = args - def batched_operator(model, inputs, targets, batch_size=None): - if batch_size is not None: - dataset = tf.data.Dataset.from_tensor_slices((inputs, targets)) - results = tf.concat([ - operator(model, x, y) - for x, y in dataset.batch(batch_size) - ], axis=0) - else: - results = operator(model, inputs, targets) + if obj is None or obj.shape[0] == 0: + return tf.constant(0.0, dtype=inputs.dtype) - return results + # compute predicted boxes for a given image + # (nb_box_pred, 4), (nb_box_pred, 1), (nb_box_pred, nb_classes) + current_boxes, proba_detection, classification = _format_objects(obj) + size = tf.shape(current_boxes)[0] - return batched_operator + if len(tf.shape(obj_ref)) == 1: + obj_ref = tf.expand_dims(obj_ref, axis=0) + # DRise consider the reference objectness to be 1 + # (nb_box_ref, 4), _, (nb_box_ref, nb_classes) + boxes_refs, _, class_refs = _format_objects(obj_ref) -batch_predictions = operator_batching(predictions_operator) -gradients_predictions = get_gradient_of_operator(predictions_operator) -batch_gradients_predictions = operator_batching(gradients_predictions) -batch_predictions_one_hot_callable = operator_batching(predictions_one_hot_callable) + # (nb_box_ref, nb_box_pred, 4) + boxes_refs = tf.repeat(tf.expand_dims(boxes_refs, axis=1), repeats=size, axis=1) + # (nb_box_ref, nb_box_pred) + intersection_score = intersection_score_fn(boxes_refs, current_boxes) -def get_inference_function( - model: Callable, - operator: Optional[OperatorSignature] = None): - """ - Define the inference function according to the model type + # (nb_box_pred,) + detection_probability = tf.squeeze(proba_detection, axis=1) - Parameters - ---------- - model - Model used for computing explanations. - operator - Function g to explain, g take 3 parameters (f, x, y) and should return a scalar, - with f the model, x the inputs and y the targets. If None, use the standard - operator g(f, x, y) = f(x)[y]. - - Returns - ------- - inference_function - Same definition as the operator. - batch_inference_function - An inference function which treat inputs and targets by batch, - it has an additionnal parameter `batch_size`. - """ - if operator is not None: - # user specified a string, an operator from the ones available or a - # custom operator, we check if the operator is valid - # and we wrap it to generate a batching version of this operator - operator = get_operator(operator) - inference_function = operator - batch_inference_function = operator_batching(operator) + # set detection probability to 1 if it should be included + detection_probability = tf.cond(tf.cast(include_detection_probability, tf.bool), + true_fn=lambda: detection_probability, + false_fn=lambda: tf.ones_like(detection_probability)) - elif isinstance(model, (tf.keras.Model, tf.Module, tf.keras.layers.Layer)): - inference_function = predictions_operator - batch_inference_function = batch_predictions + # (nb_box_ref, nb_box_pred, nb_classes) + class_refs = tf.repeat(tf.expand_dims(class_refs, axis=1), repeats=size, axis=1) - else: - # completely unknown model (e.g. sklearn), we can't backprop through it - inference_function = predictions_one_hot_callable - batch_inference_function = batch_predictions_one_hot_callable + # (nb_box_ref, nb_box_pred) + classification_score = tf.reduce_sum(class_refs * classification, axis=-1) \ + / (tf.norm(classification, axis=-1) * tf.norm(class_refs, axis=-1)+ _EPSILON) - return inference_function, batch_inference_function + # set classification score to 1 if it should be included + classification_score = tf.cond(tf.cast(include_classification_score, tf.bool), + true_fn=lambda: classification_score, + false_fn=lambda: tf.ones_like(classification_score)) + # Compute score as defined in DRise for all possible pair of boxes + # (nb_box_ref, nb_box_pred) + boxes_pairwise_scores = intersection_score \ + * detection_probability \ + * classification_score -def get_gradient_functions( - model: Callable, - operator: Optional[OperatorSignature] = None): - """ - Define the gradient function according to the model type + # select for a reference box the most similar predicted box score + # (nb_box_ref,) + ref_boxes_scores = tf.reduce_max(boxes_pairwise_scores, axis=1) - Parameters - ---------- - model - Model used for computing explanations. - operator - Function g to explain, g take 3 parameters (f, x, y) and should return a scalar, - with f the model, x the inputs and y the targets. If None, use the standard - operator g(f, x, y) = f(x)[y]. + # get an attribution for several boxes in the same time + # () + image_score = tf.reduce_mean(ref_boxes_scores) + return image_score - Returns - ------- - gradient - Gradient function of the operator. - batch_gradient - An gradient function which treat inputs and targets by batch, - it has an additionnal parameter `batch_size`. - """ - if operator is not None: - # user specified a string, an operator from the ones available or a - # custom operator, we check if the operator is valid - # and we wrap it to generate a batching version of this operator - operator = get_operator(operator) - gradient = get_gradient_of_operator(operator) - batch_gradient = operator_batching(gradient) - - elif isinstance(model, tf.keras.Model): - # no custom operator, for keras model we can backprop through the model - gradient = gradients_predictions - batch_gradient = batch_gradients_predictions - - else: - # custom model or completely unknown model (e.g. sklearn), we can't backprop through it - gradient = no_gradients_available - batch_gradient = no_gradients_available - - return gradient, batch_gradient - \ No newline at end of file + objects = model(inputs) + return tf.map_fn(batch_loop, (objects, targets), fn_output_signature=tf.float32) diff --git a/xplique/commons/operators_operations.py b/xplique/commons/operators_operations.py new file mode 100644 index 00000000..09c815a3 --- /dev/null +++ b/xplique/commons/operators_operations.py @@ -0,0 +1,296 @@ +""" +Custom tensorflow operator for Attributions +""" + +import inspect +from enum import Enum + +import tensorflow as tf + +from ..types import Callable, Optional, Union, OperatorSignature +from .exceptions import raise_invalid_operator, no_gradients_available +from .callable_operations import predictions_one_hot_callable +from .operators import (predictions_operator, # regression_operator, + semantic_segmentation_operator, object_detection_operator) + + +class Tasks(Enum): + """ + Enumeration of different tasks for which we have defined operators + """ + CLASSIFICATION = predictions_operator + # `regression_operator` do not work for gradient-based method + # the problem is its use for multi output regression + # REGRESSION = regression_operator + REGRESSION = predictions_operator + SEMANTIC_SEGMENTATION = semantic_segmentation_operator + OBJECT_DETECTION = object_detection_operator + + # object detection operator limited to box position explanation + OBJECT_DETECTION_BOX_POSITION = lambda model, inputs, targets:( + object_detection_operator(model, inputs, targets, + include_detection_probability=False, + include_classification_score=False) + ) + + # object detection operator limited to box proba and position explanation + OBJECT_DETECTION_BOX_PROBA = lambda model, inputs, targets:( + object_detection_operator(model, inputs, targets, + include_detection_probability=True, + include_classification_score=False) + ) + + # object detection operator limited to box class and position explanation + OBJECT_DETECTION_BOX_CLASS = lambda model, inputs, targets:( + object_detection_operator(model, inputs, targets, + include_detection_probability=False, + include_classification_score=True) + ) + + @staticmethod + def from_string(operator_name: str) -> "Tasks": + """ + Restore an operator from a string + + Parameters + ---------- + operator_name + String indicating the operator to restore: must be one + of 'classification' or 'regression' + + Returns + ------- + operator + The Tasks object + """ + + string_to_tasks = { + "classification": Tasks.CLASSIFICATION, + "regression": Tasks.REGRESSION, + "semantic segmentation": Tasks.SEMANTIC_SEGMENTATION, + "object detection": Tasks.OBJECT_DETECTION, + "object detection box position": Tasks.OBJECT_DETECTION_BOX_POSITION, + "object detection box proba": Tasks.OBJECT_DETECTION_BOX_PROBA, + "object detection box class": Tasks.OBJECT_DETECTION_BOX_CLASS, + } + + assert operator_name in string_to_tasks,\ + f"Only `operator` value among {string_to_tasks.keys()} are supported,\n "+\ + f"but {operator_name} was given." + + return string_to_tasks[operator_name] + + +def check_operator(operator: Callable): + """ + Check if the operator is valid g(f, x, y) -> tf.Tensor + and raise an exception and return true if so. + + Parameters + ---------- + operator + Operator to check + + Returns + ------- + is_valid + True if the operator is valid, False otherwise. + """ + # handle tf functions + # pylint: disable=protected-access + if hasattr(operator, '_python_function'): + return check_operator(operator._python_function) + + # the operator must be callable + if not hasattr(operator, '__call__'): + raise_invalid_operator() + + # the operator should take at least three arguments + args = inspect.getfullargspec(operator).args + if len(args) < 3: + raise_invalid_operator() + + return True + + +def get_operator( + operator: Optional[Union[Tasks, str, OperatorSignature]]): + """ + This function allows to retrieve an operator from: a Tasks, a task name. If the operator + is a custom one, we simply check if its signature is correct + + Parameters + ---------- + operator + An operator from the Tasks enum or the task name or a custom operator. If None, use a + classification operator. + + Returns + ------- + operator + The operator requested + """ + # case when no operator is provided + if operator is None: + return predictions_operator + + # case when the query is a string + if isinstance(operator, str): + return Tasks.from_string(operator) + + # case when the query belong to the Tasks enum + if operator in [t.value for t in Tasks]: + return operator + + # case when the operator is a custom one + assert check_operator(operator) + return operator + + +def get_gradient_of_operator(operator): + """ + Get the gradient of an operator. + + Parameters + ---------- + operator + Operator of which to compute the gradient. + + Returns + ------- + gradient + Gradient of the operator. + """ + @tf.function + def gradient(model, inputs, targets): + with tf.GradientTape() as tape: + tape.watch(inputs) + scores = operator(model, inputs, targets) + + return tape.gradient(scores, inputs) + + return gradient + + +def operator_batching(operator: OperatorSignature) -> tf.Tensor: + """ + Take care of batching an operator: (model, inputs, labels). + + Parameters + ---------- + operator + Any callable that take model, inputs and labels as parameters. + + Returns + ------- + batched_operator + Function that apply operator by batch. + """ + + def batched_operator(model, inputs, targets, batch_size=None): + if batch_size is not None: + dataset = tf.data.Dataset.from_tensor_slices((inputs, targets)) + results = tf.concat([ + operator(model, x, y) + for x, y in dataset.batch(batch_size) + ], axis=0) + else: + results = operator(model, inputs, targets) + + return results + + return batched_operator + + +batch_predictions = operator_batching(predictions_operator) +gradients_predictions = get_gradient_of_operator(predictions_operator) +batch_gradients_predictions = operator_batching(gradients_predictions) +batch_predictions_one_hot_callable = operator_batching(predictions_one_hot_callable) + + +def get_inference_function( + model: Callable, + operator: Optional[OperatorSignature] = None): + """ + Define the inference function according to the model type + + Parameters + ---------- + model + Model used for computing explanations. + operator + Function g to explain, g take 3 parameters (f, x, y) and should return a scalar, + with f the model, x the inputs and y the targets. If None, use the standard + operator g(f, x, y) = f(x)[y]. + + Returns + ------- + inference_function + Same definition as the operator. + batch_inference_function + An inference function which treat inputs and targets by batch, + it has an additional parameter `batch_size`. + """ + if operator is not None: + # user specified a string, an operator from the ones available or a + # custom operator, we check if the operator is valid + # and we wrap it to generate a batching version of this operator + operator = get_operator(operator) + inference_function = operator + batch_inference_function = operator_batching(operator) + + elif isinstance(model, (tf.keras.Model, tf.Module, tf.keras.layers.Layer)): + inference_function = predictions_operator + batch_inference_function = batch_predictions + + else: + # completely unknown model (e.g. sklearn), we can't backprop through it + inference_function = predictions_one_hot_callable + batch_inference_function = batch_predictions_one_hot_callable + + return inference_function, batch_inference_function + + +def get_gradient_functions( + model: Callable, + operator: Optional[OperatorSignature] = None): + """ + Define the gradient function according to the model type + + Parameters + ---------- + model + Model used for computing explanations. + operator + Function g to explain, g take 3 parameters (f, x, y) and should return a scalar, + with f the model, x the inputs and y the targets. If None, use the standard + operator g(f, x, y) = f(x)[y]. + + Returns + ------- + gradient + Gradient function of the operator. + batch_gradient + An gradient function which treat inputs and targets by batch, + it has an additional parameter `batch_size`. + """ + if operator is not None: + # user specified a string, an operator from the ones available or a + # custom operator, we check if the operator is valid + # and we wrap it to generate a batching version of this operator + operator = get_operator(operator) + gradient = get_gradient_of_operator(operator) + batch_gradient = operator_batching(gradient) + + elif isinstance(model, tf.keras.Model): + # no custom operator, for keras model we can backprop through the model + gradient = gradients_predictions + batch_gradient = batch_gradients_predictions + + else: + # custom model or completely unknown model (e.g. sklearn), we can't backprop through it + gradient = no_gradients_available + batch_gradient = no_gradients_available + + return gradient, batch_gradient + \ No newline at end of file diff --git a/xplique/features_visualizations/maco.py b/xplique/features_visualizations/maco.py index 4b051e4a..f03ce173 100644 --- a/xplique/features_visualizations/maco.py +++ b/xplique/features_visualizations/maco.py @@ -71,7 +71,7 @@ def maco(objective: Objective, # default to box_size that go from 50% to 5% box_size_values = tf.cast(np.linspace(0.5, 0.05, nb_steps), tf.float32) get_box_size = lambda step_i: box_size_values[step_i] - elif isinstance(box_size, Callable): + elif hasattr(box_size, "__call__"): get_box_size = box_size elif isinstance(box_size, float): get_box_size = lambda _ : box_size @@ -82,7 +82,7 @@ def maco(objective: Objective, # default to large noise to low noise noise_values = tf.cast(np.logspace(0, -4, nb_steps), tf.float32) get_noise_intensity = lambda step_i: noise_values[step_i] - elif isinstance(noise_intensity, Callable): + elif hasattr(noise_intensity, "__call__"): get_noise_intensity = noise_intensity elif isinstance(noise_intensity, float): get_noise_intensity = lambda _ : noise_intensity diff --git a/xplique/utils_functions/__init__.py b/xplique/utils_functions/__init__.py new file mode 100644 index 00000000..4c1d8941 --- /dev/null +++ b/xplique/utils_functions/__init__.py @@ -0,0 +1,5 @@ +""" +Functions to ease attributions +""" + +from .segmentation import get_class_zone, get_connected_zone, get_in_out_border, get_common_border diff --git a/xplique/utils_functions/object_detection.py b/xplique/utils_functions/object_detection.py new file mode 100644 index 00000000..c8bbbe6b --- /dev/null +++ b/xplique/utils_functions/object_detection.py @@ -0,0 +1,72 @@ +""" +Operator for object detection +""" +from typing import Tuple +import tensorflow as tf + +_EPSILON = tf.constant(1e-4) + + +def _box_iou(boxes_a: tf.Tensor, boxes_b: tf.Tensor) -> tf.Tensor: + """ + Compute the intersection between two batched bounding boxes. + Each bounding box is defined by (x1, y1, x2, y2) respectively (left, bottom, right, top). + With left < right and bottom < top + + Parameters + ---------- + boxes_a + First batch of bounding boxes. + boxes_b + Second batch of bounding boxes. + + Returns + ------- + iou_score + The IOU score between the two batches of bounding boxes. + """ + + # determine the intersection rectangle + left = tf.maximum(boxes_a[..., 0], boxes_b[..., 0]) + bottom = tf.maximum(boxes_a[..., 1], boxes_b[..., 1]) + right = tf.minimum(boxes_a[..., 2], boxes_b[..., 2]) + top = tf.minimum(boxes_a[..., 3], boxes_b[..., 3]) + + intersection_area = tf.math.maximum(right - left, 0) * tf.math.maximum(top - bottom, 0) + + # determine the areas of the prediction and ground-truth rectangles + a_area = (boxes_a[..., 2] - boxes_a[..., 0]) * (boxes_a[..., 3] - boxes_a[..., 1]) + b_area = (boxes_b[..., 2] - boxes_b[..., 0]) * (boxes_b[..., 3] - boxes_b[..., 1]) + + union_area = a_area + b_area - intersection_area + + iou_score = intersection_area / (union_area + _EPSILON) + + return iou_score + + +def _format_objects(predictions: tf.Tensor) -> Tuple[tf.Tensor, tf.Tensor, tf.Tensor]: + """ + Format bounding boxes prediction for object detection operator. + It takes a batch of bounding boxes predictions of the model and divide it between + boxes_coordinates, proba_detection, and one_hots_classifications. + + Parameters + ---------- + predictions + Batch of bounding boxes predictions of shape (nb_boxes, (4 + 1 + nc)). + (4 + 1 + nc) means: [boxes_coordinates, proba_detection, one_hots_classifications]. + Where nc is the number of classes. + + Returns + ------- + boxes_coordinates + A Tensor of shape (nb_boxes, 4) encoding the boxes coordinates. + proba_detection + A Tensor of shape (nb_boxes, 1) encoding the detection probabilities. + one_hots_classifications + A Tensor of shape (nb_boxes, nc) encoding the class predictions. + """ + boxes_coordinates, proba_detection, one_hots_classifications = \ + tf.split(predictions, [4, 1, tf.shape(predictions[0])[0] - 5], 1) + return boxes_coordinates, proba_detection, one_hots_classifications diff --git a/xplique/utils_functions/segmentation.py b/xplique/utils_functions/segmentation.py new file mode 100644 index 00000000..4a1c6824 --- /dev/null +++ b/xplique/utils_functions/segmentation.py @@ -0,0 +1,285 @@ +""" +Functions to prepare `targets` for segmentation attributions +""" + +import cv2 +import numpy as np +import tensorflow as tf + +from ..types import Union, Tuple, List + + +def get_class_zone(predictions: Union[tf.Tensor, np.array], class_id: int) -> tf.Tensor: + """ + Extract a mask for the class `c`. + The mask correspond to the pixels where the maximum prediction correspond to the class `c`. + Other classes channels are set to zero. + + Parameters + ---------- + predictions + Output of the model, it should be the output of a softmax function. + We assume the shape (h, w, c). + class_id + Index of the channel of the class of interest. + + Returns + ------- + class_zone_mask + Mask of the zone corresponding to the class of interest. + Only the corresponding channel is non-zero. + The shape is the same as `predictions`, (h, w, c). + """ + assert len(tf.shape(predictions)) == 3, "predictions should correspond to only one image" + + class_zone = tf.cast(tf.argmax(predictions, axis=-1) == class_id, tf.float32) + class_zone_mask = tf.Variable(tf.zeros(predictions.shape)) + class_zone_mask = class_zone_mask[:, :, class_id].assign(class_zone) + + assert tf.reduce_sum(class_zone_mask) >= 1 + + return class_zone_mask + + +def get_connected_zone( + predictions: Union[tf.Tensor, np.array], coordinates: Tuple[int, int] +) -> tf.Tensor: + """ + Extract a connected mask around `coordinates`. + The mask correspond to the pixels where the maximum prediction correspond + to the maximum predicted class at `coordinates`. + This class mask is then limited to the connected zone around `coordinates`. + Other classes channels are set to zero. + + Parameters + ---------- + predictions + Output of the model, it should be the output of a softmax function. + We assume the shape (h, w, c). + coordinates + Tuple of coordinates of the point inside the zone of interest. + + Returns + ------- + connected_zone_mask + Mask of the connected zone around `coordinates` with similar class prediction. + Only the corresponding channel is non-zero. + The shape is the same as `predictions`, (h, w, c). + """ + assert len(tf.shape(predictions)) == 3, "predictions should correspond to only one image" + + assert ( + coordinates[0] < predictions.shape[0] + ), f"Coordinates should be included in the shape, i.e. {coordinates[0]}<{predictions.shape[0]}" + assert ( + coordinates[1] < predictions.shape[1] + ), f"Coordinates should be included in the shape, i.e. {coordinates[1]}<{predictions.shape[1]}" + + labels = tf.argmax(predictions, axis=-1) + class_id = labels[coordinates[0], coordinates[1]] + mask = labels == class_id + mask = np.uint8(np.array(mask)[:, :, np.newaxis] * 255) + + components_masks = cv2.connectedComponents(mask)[1] # pylint: disable=no-member + + component_id = components_masks[coordinates[0], coordinates[1]] + connected_zone = tf.cast(components_masks == component_id, tf.float32) + + connected_zone_mask = tf.Variable(tf.zeros(predictions.shape)) + connected_zone_mask = connected_zone_mask[:, :, class_id].assign(connected_zone) + + assert tf.reduce_sum(connected_zone_mask) >= 1 + assert connected_zone_mask[coordinates[0], coordinates[1], class_id] != 0 + + return connected_zone_mask + + +def list_class_connected_zones( + predictions: Union[tf.Tensor,np.array], + class_id: int, + zone_minimum_size: int = 100 +) -> List[tf.Tensor]: + """ + List all connected zones for a given class. + A connected zone is a set of pixels next to each others + where the maximum prediction correspond to the same class. + This function generate a list of connected zones, + each element of the list have a similar format to `get_connected_zone` outputs. + + Parameters + ---------- + predictions + Output of the model, it should be the output of a softmax function. + We assume the shape (h, w, c). + class_id + Index of the channel of the class of interest. + zone_minimum_size + Threshold of number of pixels under which zones are not returned. + + Returns + ------- + connected_zones_masks_list + List of the connected zones masks for a given class. + Each zone predictions shape is the same as `predictions`, (h, w, c). + Only the corresponding channel is non-zero. + """ + assert len(tf.shape(predictions)) == 3, "predictions should correspond to only one image" + + labels = tf.argmax(predictions, axis=-1) + mask = labels == class_id + mask = np.uint8(np.array(mask)[:, : , np.newaxis] * 255) + + components_masks = cv2.connectedComponents(mask)[1] # pylint: disable=no-member + + sizes = np.bincount(components_masks.ravel()) + + connected_zones_masks_list = [] + for component_id, size in enumerate(sizes[1:]): + + if size > zone_minimum_size: + + connected_zone = tf.cast(components_masks == (component_id + 1), tf.float32) + + all_channels_class_zone_mask = tf.Variable(tf.zeros(predictions.shape)) + all_channels_class_zone_mask =\ + all_channels_class_zone_mask[:, :, class_id].assign(connected_zone) + + assert tf.reduce_sum(all_channels_class_zone_mask) >= 1 + + connected_zones_masks_list.append(all_channels_class_zone_mask) + + return connected_zones_masks_list + + + +def get_in_out_border( + class_zone_mask: Union[tf.Tensor, np.array], +) -> tf.Tensor: + """ + Extract the border of a zone of interest, then put `1` on the + inside border and `-1` on the outside border. + + Examples of coefficients extracted from the class channel of the class of interest: + + ``` + # class_zone_mask[:, :, c] + [[0, 0, 0, 0, 0], + [0, 0, 0, 1, 1], + [0, 0, 1, 1, 1], + [0, 0, 1, 1, 1], + [0, 1, 1, 1, 1]] + + # border_mask + [[ 0, 0, -1, -1, -1], + [ 0, -1, -1, 1, 1], + [ 0, -1, 1, 1, 0], + [-1, -1, 1, 0, 0], + [-1, 1, 1, 0, 0]] + ``` + + Parameters + ---------- + class_zone_mask + Mask delimiting the zone of interest, for the class of interest + only one channel should have non-zero values, + the one corresponding to the class. + We assume the shape (h, w, c) same as the model output for one element. + + Returns + ------- + class_borders_masks + Mask of the borders of the zone of the class of interest. + Only the corresponding channel is non-zero. + Inside borders are set to `1` and outside borders are set to `-1`. + The shape is the same as `class_zone_mask`, (h, w, c). + """ + assert len(tf.shape(class_zone_mask)) == 3,\ + "class_zone_mask should correspond to only one image" + + # channel of the class of interest + channel_mean = tf.reduce_sum(tf.cast(class_zone_mask, tf.int32), axis=[0, 1]) + assert ( + int(tf.reduce_sum(channel_mean)) >= 1 + ), "The specified `class_target_mask` is empty." + class_id = int(tf.argmax(channel_mean)) + + # set other values to -1 on the target zone to 1 + binary_mask = 2 * tf.cast(class_zone_mask[:, :, class_id], tf.int32) - 1 + + # extend size with padding for convolution + extended_binary_mask = tf.pad( + binary_mask, + tf.constant([[1, 1], [1, 1]]), + "SYMMETRIC", + ) + + kernel = tf.convert_to_tensor( + [[-1, -1, -1], [-1, 13, -1], [-1, -1, -1]], dtype=tf.int32 + ) + kernel = kernel[:, :, tf.newaxis, tf.newaxis] + conv_result = tf.nn.conv2d( + tf.expand_dims(tf.expand_dims(extended_binary_mask, axis=0), axis=-1), + kernel, + strides=1, + padding="VALID", + )[0, :, :, 0] + + # 6 < in < 21, -21 < out < -6 + in_border = tf.logical_and( + tf.math.less(tf.constant([6]), conv_result), + tf.math.less(conv_result, tf.constant([21])), + ) + out_border = tf.logical_and( + tf.math.less(tf.constant([-21]), conv_result), + tf.math.less(conv_result, tf.constant([-6])), + ) + + border_mask = ( + tf.zeros(binary_mask.shape) + + tf.cast(in_border, tf.float32) + - tf.cast(out_border, tf.float32) + ) + + class_borders_masks = tf.Variable(tf.zeros(class_zone_mask.shape)) + class_borders_masks = class_borders_masks[:, :, class_id].assign(border_mask) + + assert int(tf.reduce_sum(tf.abs(class_borders_masks))) >= 1 + + return class_borders_masks + + +def get_common_border( + border_mask_1: Union[tf.Tensor, np.array], border_mask_2: Union[tf.Tensor, np.array] +) -> tf.Tensor: + """ + Compute the common part between `border_mask_1` and `border_mask_2` masks. + Those borders should be computed using `get_in_out_border`. + + Parameters + ---------- + border_mask_1 + Border of the first zone of interest. Computed with `get_in_out_border`. + border_mask_2 + Border of the second zone of interest. Computed with `get_in_out_border`. + + Returns + ------- + common_borders_masks + Mask of the common borders between two zones of interest. + Only the two corresponding channels are non-zero. + Inside borders are set to `1` and outside borders are set to `-1`, + Respectively on the two channels. + The shape is the same as the input border masks, (h, w, c). + """ + all_channel_border_mask_1 = tf.reduce_any(border_mask_1 != 0, axis=-1) + all_channel_border_mask_2 = tf.reduce_any(border_mask_2 != 0, axis=-1) + + common_pixels_mask = tf.logical_and( + all_channel_border_mask_1, all_channel_border_mask_2 + ) + + assert tf.reduce_any(common_pixels_mask), "No common border between the two masks." + + return (border_mask_1 + border_mask_2) * tf.expand_dims( + tf.cast(common_pixels_mask, tf.float32), -1 + ) diff --git a/xplique/wrappers/pytorch.py b/xplique/wrappers/pytorch.py index c1c912bf..78e0a1ce 100644 --- a/xplique/wrappers/pytorch.py +++ b/xplique/wrappers/pytorch.py @@ -1,5 +1,5 @@ """ -Module for having a wrapper for PyTorch's model +Module for having a wrapper for PyTorch models """ import warnings @@ -10,7 +10,7 @@ class TorchWrapper(tf.keras.Model): """ - A wrapper for PyTorch's model so that they can be used in Xplique framework + A wrapper for PyTorch models so that they can be used in Xplique framework for most attribution methods Parameters @@ -53,7 +53,7 @@ def __init__(self, torch_model: "nn.Module", device: Union["torch.device", str], self.channel_first = is_channel_first # deactivate all tf.function tf.config.run_functions_eagerly(True) - warnings.warn("TF is set to run eagerly to avoid conflict with Pytorch. Thus,\ + warnings.warn("TF is set to run eagerly to avoid conflict with PyTorch. Thus,\ TF functions might be slower") # pylint: disable=arguments-differ @@ -139,7 +139,7 @@ def np_img_to_torch(self, np_inputs: np.ndarray): def _has_conv_layers(self): """ - A method that checks if the PyTorch's model has 2D convolutional layer. + A method that checks if the PyTorch models has 2D convolutional layer. Indeed, convolution with PyTorch expects inputs in the shape (N, C, H, W) where TF expect (N, H, W, C).
- Explore Xplique docs Β» + π Explore Xplique docs + | + Explore Xplique tutorials π₯
Attributions @@ -66,6 +68,8 @@ Finally, the _Metrics_ module covers the current metrics used in explainability. - [**Attribution Methods**: Sanity checks paper](https://colab.research.google.com/drive/1uJOmAg6RjlOIJj6SWN9sYRamBdHAuyaS) [](https://colab.research.google.com/drive/1uJOmAg6RjlOIJj6SWN9sYRamBdHAuyaS) - [**Attribution Methods**: Tabular data and Regression](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) [](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) + - [**Attribution Methods**: Object Detection](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) + - [**Attribution Methods**: Semantic Segmentation](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) - [**FORGRad**: Gradient strikes back with FORGrad](https://colab.research.google.com/drive/1ibLzn7r9QQIEmZxApObowzx8n9ukinYB) [](https://colab.research.google.com/drive/1ibLzn7r9QQIEmZxApObowzx8n9ukinYB) - [**Attribution Methods**: Metrics](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) [](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) @@ -75,7 +79,7 @@ Finally, the _Metrics_ module covers the current metrics used in explainability. - - [**PyTorch's model**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) + - [**PyTorch models**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) - [**Concepts Methods**: Testing with Concept Activation Vectors](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE) [](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE) @@ -95,19 +99,19 @@ Finally, the _Metrics_ module covers the current metrics used in explainability. - [**Modern Feature Visualization with MaCo**: Getting started](https://colab.research.google.com/drive/1l0kag1o-qMY4NCbWuAwnuzkzd9sf92ic) [](https://colab.research.google.com/drive/1l0kag1o-qMY4NCbWuAwnuzkzd9sf92ic) - You can find a certain number of [other practical tutorials just here](tutorials/). This section is actively developed and more contents will be - included. We will try to cover all the possible usage of the library, feel free to contact us if you have any suggestions or recommandations towards tutorials you would like to see. + You can find a certain number of [**other practical tutorials just here**](tutorials/). This section is actively developed and more contents will be + included. We will try to cover all the possible usage of the library, feel free to contact us if you have any suggestions or recommendations towards tutorials you would like to see. ## π Quick Start -Xplique requires a version of python higher than 3.6 and several libraries including Tensorflow and Numpy. Installation can be done using Pypi: +Xplique requires a version of python higher than 3.7 and several libraries including Tensorflow and Numpy. Installation can be done using Pypi: ```python pip install xplique ``` -Now that Xplique is installed, here are 4 basic examples of what you can do with the available modules. +Now that Xplique is installed, here are some basic examples of what you can do with the available modules. ??? example "Attributions Methods" Let's start with a simple example, by computing Grad-CAM for several images (or a complete dataset) on a trained model. @@ -123,9 +127,7 @@ Now that Xplique is installed, here are 4 basic examples of what you can do with # or just `explainer(images, labels)` ``` - All attributions methods share a common API. You can find out more about it [here](api/attributions/api_attributions/). - - In addition, you should also look at the [model's specificities](api/attributions/model/) and the [operator parameter documentation](api/attributions/operator/) + All attributions methods share a common API described [in the attributions API documentation](api/attributions/api_attributions/). ??? example "Attributions Metrics" @@ -145,7 +147,7 @@ Now that Xplique is installed, here are 4 basic examples of what you can do with score_grad_cam = metric(explanations) ``` - All attributions metrics share a common API. You can find out more about it [here](api/metrics/api_metrics/). + All attributions metrics share a common API. You can find out more about it [here](api/attributions/metrics/api_metrics/). ??? example "Concepts Extraction" @@ -186,7 +188,7 @@ Now that Xplique is installed, here are 4 basic examples of what you can do with ??? example "PyTorch with Xplique" - Even though the library was mainly designed to be a Tensorflow toolbox we have been working on a very practical wrapper to facilitate the integration of your PyTorch's model into Xplique's framework! + Even though the library was mainly designed to be a Tensorflow toolbox we have been working on a very practical wrapper to facilitate the integration of your PyTorch models into Xplique's framework! ```python import torch @@ -208,56 +210,61 @@ Now that Xplique is installed, here are 4 basic examples of what you can do with score_saliency = metric(explanations) ``` - Want to know more ? Check the [PyTorch documentation](pytorch/) + Want to know more ? Check the [PyTorch documentation](api/attributions/pytorch/) ## π¦ What's Included +There are 4 modules in Xplique, [Attribution methods](api/attributions/api_attributions/), [Attribution metrics](api/attributions/metrics/api_metrics/), [Concepts](api/concepts/cav/), and [Feature visualization](api/feature_viz/feature_viz/). In particular, the attribution methods module supports a huge diversity of tasks for diverse data types: [Classification](api/attributions/classification/), [Regression](api/attributions/regression/), [Object Detection](api/attributions/object_detection/), and [Semantic Segmentation](api/attributions/semantic_segmentation/). The methods compatible with such task and methods compatible with Tensorflow or PyTorch are highlighted in the following table: + ??? abstract "Table of attributions available" - All the attributions method presented below handle both **Classification** and **Regression** tasks. - - | **Attribution Method** | Type of Model | Source | Tabular Data | Images | Time-Series | Tutorial | - | :--------------------- | :------------ | :---------------------------------------- | :----------------: | :----------------: | :----------------: | :----------------: | - | Deconvolution | TF | [Paper](https://arxiv.org/abs/1311.2901) | β | β | WIP | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | - | Grad-CAM | TF | [Paper](https://arxiv.org/abs/1610.02391) | | β | | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | - | Grad-CAM++ | TF | [Paper](https://arxiv.org/abs/1710.11063) | | β | | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | - | Gradient Input | TF, Pytorch** | [Paper](https://arxiv.org/abs/1704.02685) | β | β | WIP | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | - | Guided Backprop | TF | [Paper](https://arxiv.org/abs/1412.6806) | β | β | WIP | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | - | Integrated Gradients | TF, Pytorch** | [Paper](https://arxiv.org/abs/1703.01365) | β | β | WIP | [](https://colab.research.google.com/drive/1UXJYVebDVIrkTOaOl-Zk6pHG3LWkPcLo) | - | Kernel SHAP | TF, Pytorch** , Callable* | [Paper](https://arxiv.org/abs/1705.07874) | β | β | WIP | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | - | Lime | TF, Pytorch** , Callable* | [Paper](https://arxiv.org/abs/1602.04938) | β | β | WIP | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | - | Occlusion | TF, Pytorch** , Callable* | [Paper](https://arxiv.org/abs/1311.2901) | β | β | WIP | [](https://colab.research.google.com/drive/15xmmlxQkNqNuXgHO51eKogXvLgs-sG4q) | - | Rise | TF, Pytorch** , Callable* | [Paper](https://arxiv.org/abs/1806.07421) | WIP | β | | [](https://colab.research.google.com/drive/1icu2b1JGfpTRa-ic8tBSXnqqfuCGW2mO) | - | Saliency | TF, Pytorch** | [Paper](https://arxiv.org/abs/1312.6034) | β | β | WIP | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | - | SmoothGrad | TF, Pytorch** | [Paper](https://arxiv.org/abs/1706.03825) | β | β | WIP | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | - | SquareGrad | TF, Pytorch** | [Paper](https://arxiv.org/abs/1806.10758) | β | β | WIP | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | - | VarGrad | TF, Pytorch** | [Paper](https://arxiv.org/abs/1810.03292) | β | β | WIP | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | - | Sobol Attribution | TF, Pytorch** | [Paper](https://arxiv.org/abs/2111.04138) | | β | | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | - | Hsic Attribution | TF, Pytorch** | [Paper](https://arxiv.org/abs/2206.06219) | | β | | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | - | FORGrad enhancement | TF, Pytorch** | [Paper](https://arxiv.org/abs/2307.09591) | | β | | [](https://colab.research.google.com/drive/1ibLzn7r9QQIEmZxApObowzx8n9ukinYB) | + | **Attribution Method** | Type of Model | Source | Tabular Data | Images | Time-Series | Tutorial | + | :--------------------- | :----------------------- | :---------------------------------------- | :------------: | :------------------------: | :---------: | :----------------: | + | Deconvolution | TF | [Paper](https://arxiv.org/abs/1311.2901) | C:βοΈ
R:βοΈ | C:βοΈ
OD:β
SS:β | π΅ | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | + | Grad-CAM | TF | [Paper](https://arxiv.org/abs/1610.02391) | β | C:βοΈ
OD:β
SS:β | β | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | + | Grad-CAM++ | TF | [Paper](https://arxiv.org/abs/1710.11063) | β | C:βοΈ
OD:β
SS:β | β | [](https://colab.research.google.com/drive/1nsB7xdQbU0zeYQ1-aB_D-M67-RAnvt4X) | + | Gradient Input | TF, PyTorch** | [Paper](https://arxiv.org/abs/1704.02685) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | + | Guided Backprop | TF | [Paper](https://arxiv.org/abs/1412.6806) | C:βοΈ
R:βοΈ | C:βοΈ
OD:β
SS:β | π΅ | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | + | Integrated Gradients | TF, PyTorch** | [Paper](https://arxiv.org/abs/1703.01365) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/1UXJYVebDVIrkTOaOl-Zk6pHG3LWkPcLo) | + | Kernel SHAP | TF, PyTorch**, Callable* | [Paper](https://arxiv.org/abs/1705.07874) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | + | Lime | TF, PyTorch**, Callable* | [Paper](https://arxiv.org/abs/1602.04938) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/1frholXRE4XQQ3W5yZuPQ2-xqc-LTczfT) | + | Occlusion | TF, PyTorch**, Callable* | [Paper](https://arxiv.org/abs/1311.2901) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/15xmmlxQkNqNuXgHO51eKogXvLgs-sG4q) | + | Rise | TF, PyTorch**, Callable* | [Paper](https://arxiv.org/abs/1806.07421) | π΅ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/1icu2b1JGfpTRa-ic8tBSXnqqfuCGW2mO) | + | Saliency | TF, PyTorch** | [Paper](https://arxiv.org/abs/1312.6034) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/19eB3uwAtCKZgkoWtMzrF0LTJ-htF_KE7) | + | SmoothGrad | TF, PyTorch** | [Paper](https://arxiv.org/abs/1706.03825) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | + | SquareGrad | TF, PyTorch** | [Paper](https://arxiv.org/abs/1806.10758) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | + | VarGrad | TF, PyTorch** | [Paper](https://arxiv.org/abs/1810.03292) | C:βοΈ
R:βοΈ | C:βοΈ
OD:βοΈ
SS:βοΈ | π΅ | [](https://colab.research.google.com/drive/12-tlM_TdZ12oc5lNL2S2g-hcMJV8tZUD) | + | Sobol Attribution | TF, PyTorch** | [Paper](https://arxiv.org/abs/2111.04138) | π΅ | C:βοΈ
OD:βοΈ
SS:βοΈ | β | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | + | Hsic Attribution | TF, PyTorch** | [Paper](https://arxiv.org/abs/2206.06219) | π΅ | C:βοΈ
OD:βοΈ
SS:βοΈ | β | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | + | FORGrad enhancement | TF, PyTorch** | [Paper](https://arxiv.org/abs/2307.09591) | β | C:βοΈ
OD:βοΈ
SS:βοΈ | β | [](https://colab.research.google.com/drive/1ibLzn7r9QQIEmZxApObowzx8n9ukinYB) | TF : Tensorflow compatible - \* : See the [Callable documentation](callable/) + C : [Classification](api/attributions/classification/) | R : [Regression](api/attributions/regression/) | + OD : [Object Detection](api/attributions/object_detection/) | SS : [Semantic Segmentation](api/attributions/semantic_segmentation/) + + \* : See the [Callable documentation](api/attributions/callable/) + + ** : See the [Xplique for PyTorch documentation](api/attributions/pytorch/), and the [**PyTorch models**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) notebook. - ** : See the [Xplique for Pytorch documentation](pytorch/), and the [**PyTorch's model**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) notebook + βοΈ : Supported by Xplique | β : Not applicable | π΅ : Work in Progress ??? abstract "Table of attribution's metric available" | **Attribution Metrics** | Type of Model | Property | Source | | :---------------------- | :------------ | :--------------- | :---------------------------------------- | - | MuFidelity | TF, Pytorch** | Fidelity | [Paper](https://arxiv.org/abs/2005.00631) | - | Deletion | TF, Pytorch** | Fidelity | [Paper](https://arxiv.org/abs/1806.07421) | - | Insertion | TF, Pytorch** | Fidelity | [Paper](https://arxiv.org/abs/1806.07421) | - | Average Stability | TF, Pytorch** | Stability | [Paper](https://arxiv.org/abs/2005.00631) | - | MeGe | TF, Pytorch** | Representativity | [Paper](https://arxiv.org/abs/2009.04521) | - | ReCo | TF, Pytorch** | Consistency | [Paper](https://arxiv.org/abs/2009.04521) | + | MuFidelity | TF, PyTorch** | Fidelity | [Paper](https://arxiv.org/abs/2005.00631) | + | Deletion | TF, PyTorch** | Fidelity | [Paper](https://arxiv.org/abs/1806.07421) | + | Insertion | TF, PyTorch** | Fidelity | [Paper](https://arxiv.org/abs/1806.07421) | + | Average Stability | TF, PyTorch** | Stability | [Paper](https://arxiv.org/abs/2005.00631) | + | MeGe | TF, PyTorch** | Representativity | [Paper](https://arxiv.org/abs/2009.04521) | + | ReCo | TF, PyTorch** | Consistency | [Paper](https://arxiv.org/abs/2009.04521) | | (WIP) e-robustness | TF : Tensorflow compatible - ** : See the [Xplique for Pytorch documentation](pytorch/), and the [**PyTorch's model**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) notebook + ** : See the [Xplique for PyTorch documentation](api/attributions/pytorch/), and the [**PyTorch models**: Getting started](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) notebook. ??? abstract "Table of concept methods available" @@ -297,7 +304,7 @@ This library is one approach of many to explain your model. We don't expect it t ??? info "Other interesting tools to explain your model:" - [Lucid](https://github.com/tensorflow/lucid) the wonderful library specialized in feature visualization from OpenAI. - - [Captum](https://captum.ai/) the Pytorch library for Interpretability research + - [Captum](https://captum.ai/) the PyTorch library for Interpretability research - [Tf-explain](https://github.com/sicara/tf-explain) that implement multiples attribution methods and propose callbacks API for tensorflow. - [Alibi Explain](https://github.com/SeldonIO/alibi) for model inspection and interpretation - [SHAP](https://github.com/slundberg/shap) a very popular library to compute local explanations using the classic Shapley values from game theory and their related extensions diff --git a/docs/tutorials.md b/docs/tutorials.md index 6dd83321..f7e5645a 100644 --- a/docs/tutorials.md +++ b/docs/tutorials.md @@ -1,6 +1,6 @@ # Tutorials: Notebooks π -We propose here several tutorials to discover the different functionnalities that the library has to offer. +We propose here several tutorials to discover the different functionalities that the library has to offer. We decided to host those tutorials on [Google Colab](https://colab.research.google.com/notebooks/intro.ipynb?utm_source=scs-index) mainly because you will be able to play the notebooks with a GPU which should greatly improve your User eXperience. @@ -9,14 +9,16 @@ Here is the lists of the availables tutorial for now: ## Getting Started -| **Tutorial Name** | Notebook | -| :-------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------: | -| Getting Started | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | +| **Tutorial Name** | Notebook | +| :------------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------: | +| Getting Started | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | | Sanity checks for Saliency Maps | [](https://colab.research.google.com/drive/1uJOmAg6RjlOIJj6SWN9sYRamBdHAuyaS) | -| Tabular data and Regression | [](https://colab.research.google.com/drive/1XproaVxXjO9nrBSyyy7BuKJ1vy21iHs2) | -| Metrics | [](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) | -| Concept Activation Vectors | [](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE) | -| Feature Visualization | [](https://colab.research.google.com/drive/1st43K9AH-UL4eZM1S4QdyrOi7Epa5K8v) | +| Tabular data and Regression | [](https://colab.research.google.com/drive/1pjDJmAa9oeSquYtbYh6tksU6eTmObIcq) | +| Object detection | [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) | +| Semantic Segmentation | [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) | +| Metrics | [](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg) | +| Concept Activation Vectors | [](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE) | +| Feature Visualization | [](https://colab.research.google.com/drive/1st43K9AH-UL4eZM1S4QdyrOi7Epa5K8v) | ## Attributions @@ -52,10 +54,12 @@ Here is the lists of the availables tutorial for now: ## PyTorch Wrapper -| **Tutorial Name** | Notebook | -| :-------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------: | -| PyTorch's model: Getting started | [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) | -| Metrics: With Pytorch's model| [](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) | +| **Tutorial Name** | Notebook | +| :------------------------------------- | :-----------------------------------------------------------------------------------------------------------------------------------------------------: | +| PyTorch models: Getting started | [](https://colab.research.google.com/drive/1bMlO29_0K3YnTQBbbyKQyRfo8YjvDbhe) | +| Metrics: With PyTorch models | [](https://colab.research.google.com/drive/16bEmYXzLEkUWLRInPU17QsodAIbjdhGP) | +| Object detection on PyTorch model | [](https://colab.research.google.com/drive/1X3Yq7BduMKqTA0XEheoVIpOo3IvOrzWL) | +| Semantic Segmentation on PyTorch model | [](https://colab.research.google.com/drive/1AHg7KO1fCOX5nZLGZfxkZ2-DLPPdSfbX) | ## Concepts extraction diff --git a/mkdocs.yml b/mkdocs.yml index 3a2bfdb2..92f48a15 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -8,34 +8,36 @@ nav: - Home: index.md - Attributions methods: - API Description: api/attributions/api_attributions.md - - Model Specifications: api/attributions/model.md - - Operator: api/attributions/operator.md - Methods: - - DeconvNet: api/attributions/deconvnet.md - - Grad-CAM: api/attributions/grad_cam.md - - Grad-CAM++: api/attributions/grad_cam_pp.md - - Gradient Input: api/attributions/gradient_input.md - - Guided Backprop: api/attributions/guided_backpropagation.md - - Hsic Attribution Method: api/attributions/hsic.md - - Integrated Gradient: api/attributions/integrated_gradients.md - - KernelSHAP: api/attributions/kernel_shap.md - - Lime: api/attributions/lime.md - - Occlusion sensitivity: api/attributions/occlusion.md - - Rise: api/attributions/rise.md - - Saliency: api/attributions/saliency.md - - SmoothGrad: api/attributions/smoothgrad.md - - Sobol Attribution Method: api/attributions/sobol.md - - SquareGrad: api/attributions/square_grad.md - - VarGrad: api/attributions/vargrad.md - - ForGRad: api/attributions/forgrad.md + - DeconvNet: api/attributions/methods/deconvnet.md + - ForGRad: api/attributions/methods/forgrad.md + - Grad-CAM: api/attributions/methods/grad_cam.md + - Grad-CAM++: api/attributions/methods/grad_cam_pp.md + - Gradient Input: api/attributions/methods/gradient_input.md + - Guided Backprop: api/attributions/methods/guided_backpropagation.md + - Hsic Attribution Method: api/attributions/methods/hsic.md + - Integrated Gradient: api/attributions/methods/integrated_gradients.md + - KernelSHAP: api/attributions/methods/kernel_shap.md + - Lime: api/attributions/methods/lime.md + - Occlusion sensitivity: api/attributions/methods/occlusion.md + - Rise: api/attributions/methods/rise.md + - Saliency: api/attributions/methods/saliency.md + - SmoothGrad: api/attributions/methods/smoothgrad.md + - Sobol Attribution Method: api/attributions/methods/sobol.md + - SquareGrad: api/attributions/methods/square_grad.md + - VarGrad: api/attributions/methods/vargrad.md - Metrics: - - API Description: api/metrics/api_metrics.md - - Deletion: api/metrics/deletion.md - - Insertion: api/metrics/insertion.md - - MuFidelity: api/metrics/mu_fidelity.md - - AverageStability: api/metrics/avg_stability.md - - PyTorch: pytorch.md - - Callable: callable.md + - API Description: api/attributions/metrics/api_metrics.md + - Deletion: api/attributions/metrics/deletion.md + - Insertion: api/attributions/metrics/insertion.md + - MuFidelity: api/attributions/metrics/mu_fidelity.md + - AverageStability: api/attributions/metrics/avg_stability.md + - PyTorch: api/attributions/pytorch.md + - Callable: api/attributions/callable.md + - Classification: api/attributions/classification.md + - Object Detection: api/attributions/object_detection.md + - Regression: api/attributions/regression.md + - Semantic Segmentation: api/attributions/semantic_segmentation.md - Concept based: - Cav: api/concepts/cav.md - Tcav: api/concepts/tcav.md diff --git a/requirements.txt b/requirements.txt index 125d3ba5..da889c05 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,3 +5,4 @@ scikit-image matplotlib scipy opencv-python +deprecated diff --git a/setup.cfg b/setup.cfg index d588f2e2..d88d31a0 100644 --- a/setup.cfg +++ b/setup.cfg @@ -1,5 +1,5 @@ [bumpversion] -current_version = 1.1.0 +current_version = 1.2.0 commit = True tag = True @@ -30,7 +30,7 @@ ignore-imports = no envlist = py{37,38,39,310}-lint, py{37,38,39,310}-tf{22,25,28,211}, py{38,39,310}-tf{25,28,211}-torch{111,113,200} [testenv:py{37,38,39,310}-lint] -deps = +deps = pylint -rrequirements.txt commands = diff --git a/setup.py b/setup.py index c33a4bc5..5bd1d241 100644 --- a/setup.py +++ b/setup.py @@ -5,7 +5,7 @@ setup( name="Xplique", - version="1.1.0", + version="1.2.0", description="Explanations toolbox for Tensorflow 2", long_description=README, long_description_content_type="text/markdown", @@ -13,7 +13,7 @@ author_email="thomas_fel@brown.edu", license="MIT", install_requires=['tensorflow>=2.1.0', 'numpy', 'scikit-learn', 'scikit-image', - 'matplotlib', 'scipy', 'opencv-python'], + 'matplotlib', 'scipy', 'opencv-python', 'deprecated'], extras_require={ "tests": ["pytest", "pylint"], "docs": ["mkdocs", "mkdocs-material", "numkdoc"], diff --git a/tests/attributions/test_callable.py b/tests/attributions/test_callable.py index f5457f12..c32a7d1d 100644 --- a/tests/attributions/test_callable.py +++ b/tests/attributions/test_callable.py @@ -10,7 +10,7 @@ from sklearn.ensemble import RandomForestClassifier from xplique.attributions import (Occlusion, Rise, Lime, KernelShap, SobolAttributionMethod) -from xplique.commons.operators import predictions_operator, batch_predictions,\ +from xplique.commons.operators_operations import predictions_operator, batch_predictions,\ batch_predictions_one_hot_callable from xplique.commons.callable_operations import predictions_one_hot_callable diff --git a/tests/attributions/test_object_detector.py b/tests/attributions/test_object_detector.py index 581aa6ca..aeaf5c1c 100644 --- a/tests/attributions/test_object_detector.py +++ b/tests/attributions/test_object_detector.py @@ -1,100 +1,67 @@ -import numpy as np +""" +Test object detection BoundingBoxesExplainer +""" +import os +import sys +sys.path.append(os.getcwd()) + +import unittest + import tensorflow as tf -from xplique.attributions import BoundingBoxesExplainer, Rise -from xplique.attributions.object_detector import SegmentationIouCalculator, BoxIouCalculator, \ - ImageObjectDetectorExplainer, ImageObjectDetectorScoreCalculator, IObjectFormater - -from ..utils import almost_equal, generate_model, generate_data - - -def test_iou_mask(): - """Assert the Mask IoU calculation is ok""" - dtype = np.float32 - - m1 = np.array([ - [10, 20, 30, 40] - ], dtype=dtype) - m2 = np.array([ - [15, 20, 30, 40] - ], dtype=dtype) - m3 = np.array([ - [0, 20, 10, 40] - ], dtype=dtype) - m4 = np.array([ - [0, 0, 100, 100] - ], dtype=dtype) - - iou_calculator = BoxIouCalculator() - - assert almost_equal(iou_calculator.intersect(m1, m2), 300.0 / 400.0) - assert almost_equal(iou_calculator.intersect(m1, m3), 0.0) - assert almost_equal(iou_calculator.intersect(m3, m2), 0.0) - assert almost_equal(iou_calculator.intersect(m1, m4), 400.0 / 10_000) - assert almost_equal(iou_calculator.intersect(m2, m4), 300.0 / 10_000) - assert almost_equal(iou_calculator.intersect(m3, m4), 200.0 / 10_000) - - -def test_iou_segmentation(): - """Assert the segmentation IoU computation is ok""" - - m1 = np.array([ - [0, 0], - [1, 1], - ])[None, :, :] - m2 = np.array([ - [1, 1], - [0, 0], - ])[None, :, :] - m3 = np.array([ - [1, 0], - [1, 0], - ])[None, :, :] - - iou_calculator = SegmentationIouCalculator() - - assert almost_equal(iou_calculator.intersect(m1, m2), 0.0) - assert almost_equal(iou_calculator.intersect(m1, m3), 1.0/3.0) - assert almost_equal(iou_calculator.intersect(m3, m2), 1.0/3.0) - assert almost_equal(iou_calculator.intersect(m1, m1), 1.0) - assert almost_equal(iou_calculator.intersect(m2, m2), 1.0) - - -def test_image_object_detector(): +from tests.utils import generate_data, generate_object_detection_model + +from xplique.commons.exceptions import InvalidModelException +from xplique.attributions import BoundingBoxesExplainer, Rise, Saliency + + +def test_object_detector(): """Assert input shape returned is correct""" input_shape = (8, 8, 1) + nb_samples = 3 nb_labels = 2 - x, y = generate_data(input_shape, nb_labels, nb_labels) - model = generate_model(input_shape, nb_labels) + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) method = Rise(model, nb_samples=10) obj_ref = tf.cast([ [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], ], tf.float32) - class BBoxFormater(IObjectFormater): + explainer = BoundingBoxesExplainer(method) - def format_objects(self, predictions): - if np.array_equal(predictions.numpy(), obj_ref.numpy()): - return obj_ref[:4], obj_ref[4:5], obj_ref[5:] + test_raise_assertion_error = unittest.TestCase().assertRaises + test_raise_assertion_error(InvalidModelException, explainer.gradient) + test_raise_assertion_error(InvalidModelException, explainer.batch_gradient) + phis = explainer(x, obj_ref) - bboxes = tf.cast([ - [0, 10, 20, 30], - [0, 0, 100, 100], - ], tf.float32) + assert phis.shape == (obj_ref.shape[0], input_shape[0], input_shape[1]) - proba = tf.cast([0.9, 0.1], tf.float32) - classif = tf.cast([[1.0, 0.0], [0.0, 1.0]], tf.float32) +def test_gradient_object_detector(): + """Assert input shape returned is correct""" + input_shape = (8, 8, 1) + nb_samples = 3 + nb_labels = 2 + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) - return bboxes, proba, classif + method = Saliency(model) - formater = BBoxFormater() - explainer = ImageObjectDetectorExplainer(method, formater, BoxIouCalculator()) + obj_ref = tf.cast([ + [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], + ], tf.float32) - phis = explainer(x, obj_ref) + explainer = BoundingBoxesExplainer(method) - assert phis.shape == (1, input_shape[0], input_shape[1]) + phis = explainer(x, obj_ref) + assert phis.shape == (obj_ref.shape[0], input_shape[0], input_shape[1]) \ No newline at end of file diff --git a/tests/commons/test_object_detection_operator.py b/tests/commons/test_object_detection_operator.py new file mode 100644 index 00000000..e895241a --- /dev/null +++ b/tests/commons/test_object_detection_operator.py @@ -0,0 +1,162 @@ +""" +Test object detection operator +""" +import os +import sys +sys.path.append(os.getcwd()) + +from itertools import combinations + +import numpy as np +import tensorflow as tf + +from tests.utils import almost_equal, generate_object_detection_model, generate_data + +from xplique.attributions import Occlusion, SmoothGrad +from xplique.commons.operators import object_detection_operator + + +def test_object_detector(): + """Assert input shape returned is correct""" + input_shape = (8, 8, 1) + nb_samples = 3 + nb_labels = 2 + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) + + # 3 bounding boxes, one for each input sample + obj_ref = tf.cast([ + [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], + ], tf.float32) + + explainer = Occlusion(model, operator=object_detection_operator, patch_size=4, patch_stride=2) + + # test with only one box to explain by image (3, 7) + phis = explainer(x, obj_ref) + assert phis.shape == (obj_ref.shape[0], input_shape[0], input_shape[1]) + assert phis[0, 0, 0] != np.nan + + phis2 = explainer(x, tf.expand_dims(obj_ref, axis=1)) + assert phis.shape == phis2.shape + assert almost_equal(phis, phis2) + + +def test_gradient_object_detector(): + """Assert input shape returned is correct""" + input_shape = (8, 8, 1) + nb_samples = 3 + nb_labels = 2 + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) + + explainer = SmoothGrad(model, nb_samples=10, operator=object_detection_operator) + + obj_ref = tf.cast([ + [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], + ], tf.float32) + + phis = explainer(x, obj_ref) + + assert phis.shape[:3] == (obj_ref.shape[0], input_shape[0], input_shape[1]) + + +def test_several_boxes_object_detector(): + """Assert input shape returned is correct""" + input_shape = (8, 8, 1) + nb_samples = 3 + nb_labels = 2 + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) + + explainer = SmoothGrad(model, nb_samples=10, operator=object_detection_operator) + + obj_ref = tf.cast([ + [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], + ], tf.float32) + + phis = explainer(x, obj_ref) + assert phis.shape[:3] == (obj_ref.shape[0], input_shape[0], input_shape[1]) + + several_object_refs = tf.tile(tf.expand_dims(obj_ref, axis=1), [1, 5, 1]) + + phis = explainer(x, several_object_refs) + + assert phis.shape[:3] == (several_object_refs.shape[0], input_shape[0], input_shape[1]) + + +def test_all_object_detector_operators(): + """Assert input shape returned is correct""" + input_shape = (8, 8, 1) + nb_samples = 3 + nb_labels = 2 + max_nb_boxes = 4 + x, _ = generate_data(input_shape, nb_labels, nb_samples) + model = generate_object_detection_model(input_shape, max_nb_boxes=max_nb_boxes, nb_labels=nb_labels) + + obj_ref = tf.cast([ + [0, 0, 100, 100, 0.9, 1.0, 0.0], + [50, 50, 150, 150, 0.5, 1.0, 0.0], + [0, 10, 20, 30, 0.7, 0.0, 1.0], + ], tf.float32) + + # set params + parameters_normal = { + "include_detection_probability": True, + "include_classification_score": True, + } + + parameters_intersection = { + "include_detection_probability": False, + "include_classification_score": False, + } + + parameters_probability = { + "include_detection_probability": True, + "include_classification_score": False, + } + + parameters_classification = { + "include_detection_probability": False, + "include_classification_score": True, + } + + # create operators + normal_op = lambda model, inputs, targets: \ + object_detection_operator(model, inputs, targets, **parameters_normal) + + intersection_op = lambda model, inputs, targets: \ + object_detection_operator(model, inputs, targets, **parameters_intersection) + + probability_op = lambda model, inputs, targets: \ + object_detection_operator(model, inputs, targets, **parameters_probability) + + classification_op = lambda model, inputs, targets: \ + object_detection_operator(model, inputs, targets, **parameters_classification) + + # compute explanations + phis = Occlusion(model, operator=object_detection_operator, patch_size=4, patch_stride=2)(x, obj_ref) + + normal_phis = Occlusion(model, operator=normal_op, patch_size=4, patch_stride=2)(x, obj_ref) + + intersection_phis = Occlusion(model, operator=intersection_op, patch_size=4, patch_stride=2)(x, obj_ref) + + probability_phis = Occlusion(model, operator=probability_op, patch_size=4, patch_stride=2)(x, obj_ref) + + classification_phis = Occlusion(model, operator=classification_op, patch_size=4, patch_stride=2)(x, obj_ref) + + for phi in [phis, normal_phis, intersection_phis, probability_phis, classification_phis]: + assert phi.shape[:3] == (obj_ref.shape[0], input_shape[0], input_shape[1]) + + assert almost_equal(phis, normal_phis) + + for phi1, phi2 in combinations([normal_phis, intersection_phis, probability_phis, classification_phis], 2): + assert not almost_equal(phi1, phi2) \ No newline at end of file diff --git a/tests/commons/test_operators.py b/tests/commons/test_operators.py index d33083d6..1e39a550 100644 --- a/tests/commons/test_operators.py +++ b/tests/commons/test_operators.py @@ -11,11 +11,8 @@ SquareGrad, GradCAM, Occlusion, Rise, GuidedBackprop, DeconvNet, GradCAMPP, Lime, KernelShap, SobolAttributionMethod, HsicAttributionMethod) -from xplique.commons.operators import (check_operator, predictions_operator, regression_operator, - binary_segmentation_operator, segmentation_operator) -from xplique.commons.operators import Tasks, get_operator -from xplique.commons.exceptions import InvalidOperatorException -from ..utils import generate_data, generate_regression_model +from xplique.commons.operators import (predictions_operator, regression_operator) +from ..utils import generate_data, generate_model, generate_regression_model, almost_equal def default_methods(model, operator): @@ -35,14 +32,6 @@ def default_methods(model, operator): ] -def get_segmentation_model(): - model = tf.keras.Sequential([ - tf.keras.layers.Input((20, 20, 1)), - ]) - model.compile() - return model - - def get_concept_model(): model = tf.keras.Sequential([ tf.keras.layers.Input((6)), @@ -52,64 +41,12 @@ def get_concept_model(): return model -def test_check_operator(): - # ensure that the check operator detects non-operator - - # operator must have at least 3 arguments - function_with_2_arguments = lambda x,y: 0 - - # operator must be Callable - not_a_function = [1, 2, 3] - - for operator in [function_with_2_arguments, not_a_function]: - try: - check_operator(operator) - assert False - except InvalidOperatorException: - pass - - -def test_proposed_operators(): - # ensure all proposed operators are operators - for operator in [predictions_operator, regression_operator, - binary_segmentation_operator, segmentation_operator]: - check_operator(operator) - - -def test_get_operator(): - tasks_name = [task.name for task in Tasks] - assert tasks_name.sort() == ['classification', 'regression'].sort() - # get by enum - assert get_operator(Tasks.CLASSIFICATION) is predictions_operator - assert get_operator(Tasks.REGRESSION) is regression_operator - - # get by string - assert get_operator("classification") is predictions_operator - assert get_operator("regression") is regression_operator - - # assert a not valid string does not work - with pytest.raises(AssertionError): - get_operator("random") - - # operator must have at least 3 arguments - function_with_2_arguments = lambda x,y: 0 - - # operator must be Callable - not_a_function = [1, 2, 3] - - for operator in [function_with_2_arguments, not_a_function]: - try: - get_operator(operator) - except InvalidOperatorException: - pass - - -def test_regression_operator(): - input_shape, nb_labels, samples = ((10, 10, 1), 10, 20) +def test_predictions_operator(): + input_shape, nb_labels, samples = ((10, 10, 3), 10, 20) x, y = generate_data(input_shape, nb_labels, samples) - regression_model = generate_regression_model(input_shape, nb_labels) + classification_model = generate_model(input_shape, nb_labels) - methods = default_methods(regression_model, regression_operator) + methods = default_methods(classification_model, regression_operator) for method in methods: assert hasattr(method, 'inference_function') @@ -120,18 +57,15 @@ def test_regression_operator(): phis = method(x, y) assert x.shape[:-1] == phis.shape[:3] - -def test_segmentation_operator(): - segmentation_model = get_segmentation_model() - x, y = generate_data((20, 20, 3), 10, 10) - def segmentation_operator(model, x, y): - # explaining channel 0 - return tf.reduce_sum(model(x)[:,:,0], (1, 2)) +def test_regression_operator(): + input_shape, nb_labels, samples = ((10, 10, 1), 10, 20) + x, y = generate_data(input_shape, nb_labels, samples) + regression_model = generate_regression_model(input_shape, nb_labels) - methods = default_methods(segmentation_model, segmentation_operator) + methods = default_methods(regression_model, regression_operator) for method in methods: assert hasattr(method, 'inference_function') @@ -153,7 +87,6 @@ def test_concept_operator(): def concept_operator(model, x, y): x = tf.reshape(x, (-1, 20*20)) - print(x.shape, random_projection.shape) ui = x @ random_projection return tf.reduce_sum(model(ui) * y, axis=-1) diff --git a/tests/commons/test_operators_operations.py b/tests/commons/test_operators_operations.py new file mode 100644 index 00000000..92e79abd --- /dev/null +++ b/tests/commons/test_operators_operations.py @@ -0,0 +1,76 @@ +""" +Ensure we can use the operator functionnality on various models +""" + +import pytest + +import xplique +from xplique.commons.operators_operations import check_operator, Tasks, get_operator +from xplique.commons.operators import (predictions_operator, regression_operator, + semantic_segmentation_operator, object_detection_operator) +from xplique.commons.exceptions import InvalidOperatorException + + +def test_check_operator(): + # ensure that the check operator detects non-operator + + # operator must have at least 3 arguments + function_with_2_arguments = lambda x,y: 0 + + # operator must be Callable + not_a_function = [1, 2, 3] + + for operator in [function_with_2_arguments, not_a_function]: + try: + check_operator(operator) + assert False + except InvalidOperatorException: + pass + + +def test_get_operator(): + possible_tasks = ["classification", "regression", "semantic segmentation", "object detection", + "object detection box position", "object detection box proba", + "object detection box class"] + + tasks_name = [task.name for task in Tasks] + assert tasks_name.sort() == possible_tasks.sort() + + # get by enum + assert get_operator(Tasks.CLASSIFICATION) is predictions_operator + assert get_operator(Tasks.REGRESSION) is predictions_operator # TODO, change when there is a real regression operator + assert get_operator(Tasks.OBJECT_DETECTION) is object_detection_operator + assert get_operator(Tasks.SEMANTIC_SEGMENTATION) is semantic_segmentation_operator + + # get by string + assert get_operator("classification") is predictions_operator + assert get_operator("regression") is predictions_operator # TODO, change when there is a real regression operator + assert get_operator("object detection") is object_detection_operator + assert get_operator("semantic segmentation") is semantic_segmentation_operator + + # assert a not valid string does not work + with pytest.raises(AssertionError): + get_operator("random") + + # operator must have at least 3 arguments + function_with_2_arguments = lambda x,y: 0 + + # operator must be Callable + not_a_function = [1, 2, 3] + + for operator in [function_with_2_arguments, not_a_function]: + try: + get_operator(operator) + except InvalidOperatorException: + pass + + +def test_proposed_operators(): + # ensure all proposed operators are operators + for operator in [task.value for task in Tasks]: + check_operator(operator) + +def test_enum_shortcut(): + # ensure all proposed operators are operators + for operator in [task.value for task in xplique.Tasks]: + check_operator(operator) diff --git a/tests/commons/test_segmentation_operator.py b/tests/commons/test_segmentation_operator.py new file mode 100644 index 00000000..c51df8c0 --- /dev/null +++ b/tests/commons/test_segmentation_operator.py @@ -0,0 +1,109 @@ +""" +Ensure we can use the operator functionality on various models +""" + +import numpy as np +import tensorflow as tf + +from xplique.attributions import (Saliency, GradientInput, IntegratedGradients, SmoothGrad, VarGrad, + SquareGrad, GradCAM, Occlusion, Rise, GuidedBackprop, DeconvNet, + GradCAMPP, Lime, KernelShap, SobolAttributionMethod, + HsicAttributionMethod) +from xplique.commons.operators_operations import semantic_segmentation_operator +from ..utils import generate_data, almost_equal + + +def default_methods(model, operator): + return [ + Saliency(model, operator=operator), + GradientInput(model, operator=operator), + SmoothGrad(model, operator=operator), + VarGrad(model, operator=operator), + SquareGrad(model, operator=operator), + IntegratedGradients(model, operator=operator), + Occlusion(model, operator=operator), + Rise(model, operator=operator, nb_samples=2), + GuidedBackprop(model, operator=operator), + DeconvNet(model, operator=operator), + SobolAttributionMethod(model, operator=operator, grid_size=2, nb_design=2), + HsicAttributionMethod(model, operator=operator, grid_size=2, nb_design=2), + ] + + +def get_segmentation_model(input_shape=(20, 20, 1)): + model = tf.keras.Sequential([ + tf.keras.layers.Input(input_shape), + ]) + model.compile() + return model + + +def test_segmentation_operator(): + input_shape = (20, 20, 3) + segmentation_model = get_segmentation_model(input_shape) + + x, _ = generate_data(input_shape, 10, 10) + y, _ = generate_data(input_shape, 10, 10) + + methods = default_methods(segmentation_model, semantic_segmentation_operator) + for method in methods: + + assert hasattr(method, 'inference_function') + assert hasattr(method, 'batch_inference_function') + assert hasattr(method, 'gradient') + assert hasattr(method, 'batch_gradient') + + phis = method(x, y) + + assert x.shape[:-1] == phis.shape[:3] + + +def test_segmentation_operator_computation(): + image = [[[[0, 0, 1.0], + [0, 0, 1.0], + [1.0, 1.0, 1.0],], + [[0, 0, 0], + [0, 1.0, 0], + [1.0, 1.0, 1.0],], + [[0, 0, 0], + [0, 0, 0], + [1.0, 0, 0],],]] + image = tf.transpose(tf.convert_to_tensor(image, tf.float32), perm=[0, 2, 3, 1]) + + model = lambda x: tf.concat([x, tf.expand_dims(tf.reduce_mean(x, axis=-1), axis=-1)], axis=-1) + + target_1 = [[[[0, 0, 1.0], + [0, 0, 1.0], + [1.0, 1.0, 1.0],], + [[0, 0, 0], + [0, 0, 0], + [0, 0, 0],], + [[0, 0, 0], + [0, 0, 0], + [0, 0, 0],], + [[0, 0, 0], + [0, 0, 0], + [0, 0, 0],],]] + target_1 = tf.transpose(tf.convert_to_tensor(target_1, tf.float32), perm=[0, 2, 3, 1]) + + target_2 = [[[[0, 0, 0], + [0, 0, 0], + [0, 0, 0],], + [[0, 0, 0], + [0, 0, 0], + [0, 0, 0],], + [[0, 0, 0], + [0, 0, 0], + [0, 0, 0],], + [[0, 0, 0], + [0, 0, 0], + [1.0, 1.0, 1.0],],]] + target_2 = tf.transpose(tf.convert_to_tensor(target_2, tf.float32), perm=[0, 2, 3, 1]) + + scores = model(np.array(image)) * target_2 + + score_1 = semantic_segmentation_operator(model, image, np.array(target_1)) + score_2 = semantic_segmentation_operator(model, np.array(image), target_2) + + assert almost_equal(score_1, 1.0) + assert almost_equal(score_2, (7.0 / 3) / 3) diff --git a/tests/utils.py b/tests/utils.py index f9457372..ebbf13bd 100644 --- a/tests/utils.py +++ b/tests/utils.py @@ -2,7 +2,8 @@ from sklearn.linear_model import LinearRegression import tensorflow as tf from tensorflow.keras.models import Sequential, Model -from tensorflow.keras.layers import Dense, Conv1D, Conv2D, Activation, GlobalAveragePooling1D, Dropout, Flatten, MaxPooling2D, Input +from tensorflow.keras.layers import (Dense, Conv1D, Conv2D, Activation, GlobalAveragePooling1D, + Dropout, Flatten, MaxPooling2D, Input, Reshape) from tensorflow.keras.utils import to_categorical def generate_data(x_shape=(32, 32, 3), num_labels=10, samples=100): @@ -114,3 +115,43 @@ def __call__(self, inputs): return tf_model +def generate_object_detection_model(input_shape=(32, 32, 3), max_nb_boxes=10, nb_labels=5, with_nmf=False): + # create a model that generates max_nb_boxes and select some randomly + output_shape = (max_nb_boxes, 5 + nb_labels) + model = Sequential() + model.add(Input(shape=input_shape)) + model.add(Conv2D(4, kernel_size=(2, 2), + activation='relu')) + model.add(MaxPooling2D(pool_size=(2, 2))) + model.add(Flatten()) + model.add(Dense(np.prod(output_shape))) + model.add(Reshape(output_shape)) + model.add(Activation('sigmoid')) + model.compile(loss='mae', optimizer='sgd') + + # ensure iou computation will work + def make_plausible_boxes(model_output): + coordinates = tf.sort(model_output[:, :, :4], axis=-1) * 200 + probabilities = model_output[:, :, 4][:, :, tf.newaxis] + classifications = tf.nn.softmax(model_output[:, :, 5:], axis=-1) + new_output = tf.concat([coordinates, probabilities, classifications], axis=-1) + return new_output + + valid_model = lambda inputs: make_plausible_boxes(model(inputs)) + + # equivalent of nmf + def randomly_select_boxes(boxes): + boxes_ids = tf.range(tf.shape(boxes)[0]) + nb_boxes = tf.experimental.numpy.random.randint(1, max_nb_boxes) + boxes_ids = tf.random.shuffle(boxes_ids)[:nb_boxes] + return tf.gather(boxes, boxes_ids) + + # model with nmf + def model_with_random_nb_boxes(input): + all_boxes = valid_model(input) + some_boxes = [randomly_select_boxes(boxes) for boxes in all_boxes] + return some_boxes + + if with_nmf: + return model_with_random_nb_boxes + return valid_model diff --git a/tests/utils_functions/__init__.py b/tests/utils_functions/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/utils_functions/test_object_detection.py b/tests/utils_functions/test_object_detection.py new file mode 100644 index 00000000..f44b1609 --- /dev/null +++ b/tests/utils_functions/test_object_detection.py @@ -0,0 +1,34 @@ +""" +Test utils functions for object detection +""" + +import numpy as np + +from xplique.utils_functions.object_detection import _box_iou + +from ..utils import almost_equal + + +def test_iou_mask(): + """Assert the Mask IoU calculation is ok""" + dtype = np.float32 + + m1 = np.array([ + [10, 20, 30, 40] + ], dtype=dtype) + m2 = np.array([ + [15, 20, 30, 40] + ], dtype=dtype) + m3 = np.array([ + [0, 20, 10, 40] + ], dtype=dtype) + m4 = np.array([ + [0, 0, 100, 100] + ], dtype=dtype) + + assert almost_equal(_box_iou(m1, m2), 300.0 / 400.0) + assert almost_equal(_box_iou(m1, m3), 0.0) + assert almost_equal(_box_iou(m3, m2), 0.0) + assert almost_equal(_box_iou(m1, m4), 400.0 / 10_000) + assert almost_equal(_box_iou(m2, m4), 300.0 / 10_000) + assert almost_equal(_box_iou(m3, m4), 200.0 / 10_000) diff --git a/tests/utils_functions/test_segmentation.py b/tests/utils_functions/test_segmentation.py new file mode 100644 index 00000000..7e3020f8 --- /dev/null +++ b/tests/utils_functions/test_segmentation.py @@ -0,0 +1,212 @@ +import tensorflow as tf + +from xplique.utils_functions.segmentation import * + +def get_prediction(): + predictions = [[[0.6, 0.6, 0.6, 0.2, 0.2], + [0.6, 0.6, 0.2, 0.2, 0.2], + [0.6, 0.2, 0.2, 0.2, 0.2], + [0.2, 0.2, 0.2, 0.2, 0.2], + [0.2, 0.2, 0.2, 0.2, 0.2],], + [[0.2, 0.2, 0.2, 0.6, 0.6], + [0.2, 0.2, 0.6, 0.6, 0.6], + [0.2, 0.6, 0.6, 0.6, 0.2], + [0.6, 0.6, 0.6, 0.2, 0.2], + [0.6, 0.6, 0.2, 0.2, 0.2],], + [[0.2, 0.2, 0.2, 0.2, 0.2], + [0.2, 0.2, 0.2, 0.2, 0.2], + [0.2, 0.2, 0.2, 0.2, 0.6], + [0.2, 0.2, 0.2, 0.6, 0.6], + [0.2, 0.2, 0.6, 0.6, 0.6],],] + + predictions = tf.convert_to_tensor(predictions, tf.float32) + + predictions = tf.transpose(predictions, perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(tf.reduce_sum(predictions, axis=-1), tf.ones((5, 5)))) + + return predictions + + +def test_get_class_zone(): + predictions = get_prediction() + + target_1 = get_class_zone(predictions, class_id=1) + + expected_target = tf.transpose(tf.convert_to_tensor( + [[[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],], + [[0, 0, 0, 1, 1], + [0, 0, 1, 1, 1], + [0, 1, 1, 1, 0], + [1, 1, 1, 0, 0], + [1, 1, 0, 0, 0],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],],tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(target_1, expected_target)) + + target_0 = get_class_zone(predictions, class_id=0) + target_2 = get_class_zone(predictions, class_id=2) + + assert tf.reduce_all(tf.equal( + tf.reduce_sum(tf.stack([target_0, target_1, target_2], axis=0), axis=[0, 3]), + tf.fill((5, 5), 1.0))) + + +def test_get_connected_zone(): + predictions = get_prediction() + + target_1 = get_connected_zone(predictions, coordinates=(2, 2)) + + expected_target = tf.transpose(tf.convert_to_tensor( + [[[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],], + [[0, 0, 0, 1, 1], + [0, 0, 1, 1, 1], + [0, 1, 1, 1, 0], + [1, 1, 1, 0, 0], + [1, 1, 0, 0, 0],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],],tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(target_1, expected_target)) + + target_0 = get_connected_zone(predictions, coordinates=(0, 0)) + target_2 = get_connected_zone(predictions, coordinates=(4, 4)) + + assert tf.reduce_all(tf.equal( + tf.reduce_sum(tf.stack([target_0, target_1, target_2], axis=0), axis=[0, 3]), + tf.fill((5, 5), 1.0))) + + +def test_list_class_connected_zones(): + predictions = get_prediction() + + predictions = tf.stack([predictions[:, :, 0] + predictions[:, :, 2], predictions[:, :, 1]], axis=-1) + + zones_0 = list_class_connected_zones(predictions, class_id=0, zone_minimum_size=1) + zones_1 = list_class_connected_zones(predictions, class_id=1, zone_minimum_size=1) + no_zones = list_class_connected_zones(predictions, class_id=0, zone_minimum_size=10) + + assert len(zones_0) == 2 + assert len(zones_1) == 1 + assert len(no_zones) == 0 + + expected_zones_1 = tf.transpose(tf.convert_to_tensor( + [[[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],], + [[0, 0, 0, 1, 1], + [0, 0, 1, 1, 1], + [0, 1, 1, 1, 0], + [1, 1, 1, 0, 0], + [1, 1, 0, 0, 0],],], tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(zones_1[0], expected_zones_1)) + + expected_zones_21 = tf.transpose(tf.convert_to_tensor( + [[[1, 1, 1, 0, 0], + [1, 1, 0, 0, 0], + [1, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],], tf.float32), perm=[1, 2, 0]) + + expected_zones_22 = tf.transpose(tf.convert_to_tensor( + [[[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 1], + [0, 0, 0, 1, 1], + [0, 0, 1, 1, 1],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],], tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(zones_0[0], expected_zones_21))\ + or tf.reduce_all(tf.equal(zones_0[0], expected_zones_22)) + + assert tf.reduce_all(tf.equal(zones_0[1], expected_zones_21))\ + or tf.reduce_all(tf.equal(zones_0[1], expected_zones_22)) + + + +def test_get_in_out_border(): + predictions = get_prediction() + + central_zone = get_connected_zone(predictions, coordinates=(2, 2)) + + borders = get_in_out_border(central_zone) + + expected_borders = tf.transpose(tf.convert_to_tensor( + [[[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],], + [[ 0, -1, -1,1, 0], + [-1, -1, 1, 1, 1], + [-1, 1, 1,1 , -1], + [ 1, 1, 1, -1, -1], + [ 0,1, -1, -1, 0],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],],tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(borders, expected_borders)) + + +def test_get_common_border(): + predictions = get_prediction() + + left_corner_zone = get_connected_zone(predictions, coordinates=(0, 0)) + central_zone = get_connected_zone(predictions, coordinates=(2, 2)) + + left_corner_borders = get_in_out_border(left_corner_zone) + central_borders = get_in_out_border(central_zone) + + common_borders_0 = get_common_border(left_corner_borders, central_borders) + common_borders_1 = get_common_border(central_borders, left_corner_borders) + + expected_common_borders = tf.transpose(tf.convert_to_tensor( + [[[ 0, 1, 1, -1, 0], + [ 1, 1, -1, -1, 0], + [ 1, -1, -1, 0, 0], + [-1, -1, 0, 0, 0], + [ 0, 0, 0, 0, 0],], + [[ 0, -1, -1, 1, 0], + [-1, -1, 1, 1, 0], + [-1, 1, 1, 0, 0], + [ 1, 1, 0, 0, 0], + [ 0, 0, 0, 0, 0],], + [[0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0], + [0, 0, 0, 0, 0],],],tf.float32), perm=[1, 2, 0]) + + assert tf.reduce_all(tf.equal(common_borders_0, common_borders_1)) + + assert tf.reduce_all(tf.equal(common_borders_0, expected_common_borders)) \ No newline at end of file diff --git a/xplique/__init__.py b/xplique/__init__.py index daad8dfc..d3ea3182 100644 --- a/xplique/__init__.py +++ b/xplique/__init__.py @@ -6,10 +6,12 @@ techniques """ -__version__ = '1.1.0' +__version__ = '1.2.0' from . import attributions from . import concepts from . import features_visualizations from . import commons from . import plots + +from .commons import Tasks diff --git a/xplique/attributions/object_detector.py b/xplique/attributions/object_detector.py index 760e4305..cc434806 100644 --- a/xplique/attributions/object_detector.py +++ b/xplique/attributions/object_detector.py @@ -2,268 +2,108 @@ Module related to Object detector method """ -from typing import Iterable, Tuple, Union, Optional -import abc +from deprecated import deprecated -import tensorflow as tf import numpy as np +import tensorflow as tf -from xplique.attributions.base import BlackBoxExplainer -from xplique.commons import operator_batching +from ..types import Optional, Callable, Union +from .base import BlackBoxExplainer, WhiteBoxExplainer +from ..commons import get_gradient_functions +from ..commons.operators import object_detection_operator +from ..utils_functions.object_detection import _box_iou, _format_objects -class IouCalculator: - """ - Used to compute the Intersection Over Union (IOU). - """ - @abc.abstractmethod - def intersect(self, objects_a: tf.Tensor, objects_b: tf.Tensor) -> tf.Tensor: - """ - Compute the intersection between two batched objects (e.g boxes, segmentation masks...) - Parameters - ---------- - objects_a - First batch of objects to compare. - objects_b - Second batch of objects to compare. +OLD_OBJECT_DETECTION_DEPRECATION_MESSAGE = """ +\n +The method to compute attribution explanation explanations changed drastically after version 1.0.0. +For more information please refer to the documentation: +https://deel-ai.github.io/xplique/latest/api/attributions/object_detection/ - Returns - ------- - score - Real value between [0,1] corresponding to the intersection of the 2 objects. - """ - raise NotImplementedError() +Nonetheless, here is a quick example of how it is used now: +``` +from xplique.attributions import AnyMethod +explainer = AnyMethod(model, operator="object detection", ...) +explanation = explainer(inputs, targets) # be aware, targets are specific to object detection here +``` +""" -class SegmentationIouCalculator(IouCalculator): - """ - Compute segmentation masks IOU. +class BoundingBoxesExplainer(BlackBoxExplainer): """ - def intersect(self, masks_a: tf.Tensor, masks_b: tf.Tensor) -> tf.Tensor: - """ - Compute the intersection between two batched segmentation masks. - Each segmentation is a boolean mask on the whole image - - Parameters - ---------- - masks_a - First batch of segmentation masks. - masks_b - Second batch of segmentation masks. - - Returns - ------- - iou_score - The IOU score between the first and second batch of masks. - """ - # pylint: disable=W0221,W0237 - - axis = np.arange(1, tf.rank(masks_a)) - - inter_area = tf.reduce_sum(tf.cast(tf.logical_and(masks_a, masks_b), - dtype=tf.float32), axis=axis) - union_area = tf.reduce_sum(tf.cast(tf.logical_or(masks_a, masks_b), - dtype=tf.float32), axis=axis) - - iou_score = inter_area / tf.maximum(union_area, 1.0) - - return iou_score + For a given black box explainer, this class allows to find explications of an object detector + model. The object model detector shall return a list (length of the size of the batch) + containing a tensor of 2 dimensions. + The first dimension of the tensor is the number of bounding boxes found in the image + The second dimension is: + [xmin, ymin, xmax, ymax, probability_detection, ones_hot_classif_result] + This work is a generalization of the following article at any kind of black box explainer and + also can be used for other kind of object detector (like segmentation) + Ref. Petsiuk & al., Black-box Explanation of Object Detectors via Saliency Maps (2021). + https://arxiv.org/pdf/2006.03204.pdf -class BoxIouCalculator(IouCalculator): - """ - Used to compute the Bounding Box IOU. + Parameters + ---------- + explainer + the black box explainer used to explain the object detector model + _ + inheritance from old versions + intersection_score + the iou calculator used to compare two objects. """ - EPSILON = tf.constant(1e-4) - - def intersect(self, boxes_a: tf.Tensor, boxes_b: tf.Tensor) -> tf.Tensor: - """ - Compute the intersection between two batched bounding boxes. - Each bounding box is defined by (x1, y1, x2, y2) respectively (left, bottom, right, top). - - Parameters - ---------- - boxes_a - First batch of bounding boxes. - boxes_b - Second batch of bounding boxes. - - Returns - ------- - iou_score - The IOU score between the first and second batch of bounding boxes. - """ - # pylint: disable=W0221,W0237 - - # determine the intersection rectangle - left = tf.maximum(boxes_a[..., 0], boxes_b[..., 0]) - bottom = tf.maximum(boxes_a[..., 1], boxes_b[..., 1]) - right = tf.minimum(boxes_a[..., 2], boxes_b[..., 2]) - top = tf.minimum(boxes_a[..., 3], boxes_b[..., 3]) - - intersection_area = tf.math.maximum(right - left, 0) * tf.math.maximum(top - bottom, 0) - - # determine the areas of the prediction and ground-truth rectangles - a_area = (boxes_a[..., 2] - boxes_a[..., 0]) * (boxes_a[..., 3] - boxes_a[..., 1]) - b_area = (boxes_b[..., 2] - boxes_b[..., 0]) * (boxes_b[..., 3] - boxes_b[..., 1]) - - union_area = a_area + b_area - intersection_area - - iou_score = intersection_area / (union_area + BoxIouCalculator.EPSILON) - - return iou_score - - -class IObjectFormater: - """ - Generic class to format the model prediction - """ - def format_objects(self, predictions) -> Iterable[Tuple[tf.Tensor, tf.Tensor, tf.Tensor]]: - """ - Format the model prediction of a given image to have the prediction of the following format: - objects, proba_detection, one_hots_classifications - - Parameters - ---------- - predictions - prediction of the model of a given image - - Returns - ------- - object - bounding box or mask component of the prediction - proba - existence probability component of the prediction - classification - classification component of the prediction - """ - raise NotImplementedError() + @deprecated(version="1.0.0", reason=OLD_OBJECT_DETECTION_DEPRECATION_MESSAGE) + def __init__(self, + explainer: BlackBoxExplainer, + _: Optional[Callable] = _format_objects, + intersection_score: Optional[Callable] = _box_iou): + # make operator function based on arguments + operator = lambda model, inputs, targets: \ + object_detection_operator(model, inputs, targets, intersection_score) + + # BlackBoxExplainer init to set operator + super().__init__(model=explainer.model, batch_size=explainer.batch_size, operator=operator) + self.explainer = explainer -class ImageObjectDetectorScoreCalculator: - """ - Class to compute batch score - """ - def __init__(self, object_formater: IObjectFormater, iou_calculator: IouCalculator): - self.object_formater = object_formater - self.iou_calculator = iou_calculator + # update explainer inference functions for explain method + self.explainer.inference_function = self.inference_function + self.explainer.batch_inference_function = self.batch_inference_function - self.batch_score = operator_batching(self.score) + if isinstance(self.explainer, WhiteBoxExplainer): + # check and get gradient function from model and operator + self.gradient, self.batch_gradient = get_gradient_functions(self.model, operator) + self.explainer.gradient = self.gradient + self.explainer.batch_gradient = self.batch_gradient - def score(self, model, inp, object_ref) -> tf.Tensor: + def explain(self, + inputs: Union[tf.data.Dataset, tf.Tensor, np.array], + targets: Optional[Union[tf.Tensor, np.array]] = None) -> tf.Tensor: """ - Compute the matching score between prediction and a given object + Compute the explanation of the object detection through the explainer Parameters ---------- - model - the model used for the object detection - inp - the batched image - object_ref - the object target to compare with the prediction of the model + inputs + Dataset, Tensor or Array. Input samples to be explained. + If Dataset, targets should not be provided (included in Dataset). + Expected shape (N, H, W, C). + More information in the documentation. + targets + Tensor or Array. One-hot encoding of the model's output from which an explanation + is desired. One encoding per input and only one output at a time. Therefore, + the expected shape is (N, ...). With features matching the object formatting. + See object detection operator documentation for more information + More information in the documentation. Returns ------- - score - for each image, the matching score between the object of reference and - the prediction of the model - """ - objects = model(inp) - score_values = [] - for obj, obj_ref in zip(objects, object_ref): - if obj is None or obj.shape[0] == 0: - score_values.append(tf.constant(0.0, dtype=inp.dtype)) - else: - current_boxes, proba_detection, classification = \ - self.object_formater.format_objects(obj) - - if len(tf.shape(obj_ref)) == 1: - obj_ref = tf.expand_dims(obj_ref, axis=0) - - obj_ref = self.object_formater.format_objects(obj_ref) - - scores = [] - size = tf.shape(current_boxes)[0] - for boxes_ref, proba_ref, class_ref in zip(*obj_ref): - boxes_ref = tf.repeat(tf.expand_dims(boxes_ref, axis=0), repeats=size, axis=0) - proba_ref = tf.repeat(tf.expand_dims(proba_ref, axis=0), repeats=size, axis=0) - class_ref = tf.repeat(tf.expand_dims(class_ref, axis=0), repeats=size, axis=0) - - iou = self.iou_calculator.intersect(boxes_ref, current_boxes) - classification_similarity = tf.reduce_sum(class_ref * classification, axis=1) \ - / (tf.norm(classification, axis=1) * tf.norm(class_ref, axis=1)) - - current_score = iou * tf.squeeze(proba_detection, axis=1) \ - * classification_similarity - current_score = tf.reduce_max(current_score) - scores.append(current_score) - - score_value = tf.reduce_max(tf.stack(scores)) - score_values.append(score_value) - - score_values = tf.stack(score_values) - - return score_values - - -class ImageObjectDetectorExplainer(BlackBoxExplainer): - """ - Used to define method as an object detector one - """ - - def __init__(self, explainer: BlackBoxExplainer, object_detector_formater: IObjectFormater, - iou_calculator: IouCalculator): + explanation + The resulting object detection explanation """ - Constructor - - Parameters - ---------- - explainer - the black box explainer used to explain the object detector model - object_detector_formater - the formater of the object detector model used to format the prediction - of the right format - iou_calculator - the iou calculator used to compare two objects. - """ - super().__init__(explainer.model, explainer.batch_size) - self.explainer = explainer - self.score_calculator = ImageObjectDetectorScoreCalculator(object_detector_formater, - iou_calculator) - self.explainer.inference_function = self.score_calculator.score - self.explainer.batch_inference_function = self.score_calculator.batch_score - - def explain(self, inputs: Union[tf.data.Dataset, tf.Tensor, np.array], - targets: Optional[Union[tf.Tensor, np.array]] = None) -> tf.Tensor: if len(tf.shape(targets)) == 1: targets = tf.expand_dims(targets, axis=0) return self.explainer.explain(inputs, targets) - - -class BoundingBoxesExplainer(ImageObjectDetectorExplainer, IObjectFormater): - """ - For a given black box explainer, this class allows to find explications of an object detector - model. The object model detector shall return a list (length of the size of the batch) - containing a tensor of 2 dimensions. - The first dimension of the tensor is the number of bounding boxes found in the image - The second dimension is: - [x1_box, y1_box, x2_box, y2_box, probability_detection, ones_hot_classif_result] - - This work is a generalisation of the following article at any kind of black box explainer and - also can be used for other kind of object detector (like segmentation) - - Ref. Petsiuk & al., Black-box Explanation of Object Detectors via Saliency Maps (2021). - https://arxiv.org/pdf/2006.03204.pdf - """ - - def __init__(self, explainer: BlackBoxExplainer): - super().__init__(explainer, self, BoxIouCalculator()) - - def format_objects(self, predictions) -> Iterable[Tuple[tf.Tensor, tf.Tensor, tf.Tensor]]: - boxes, proba_detection, one_hots_classifications = tf.split(predictions, - [4, 1, tf.shape(predictions[0])[0] - 5], 1) - return boxes, proba_detection, one_hots_classifications diff --git a/xplique/commons/__init__.py b/xplique/commons/__init__.py index 3dfa73af..94237f90 100644 --- a/xplique/commons/__init__.py +++ b/xplique/commons/__init__.py @@ -7,7 +7,7 @@ find_layer, open_relu_policy from .tf_operations import repeat_labels, batch_tensor from .callable_operations import predictions_one_hot_callable -from .operators import Tasks, get_operator, check_operator, operator_batching,\ - get_inference_function, get_gradient_functions +from .operators_operations import (Tasks, get_operator, check_operator, operator_batching, + get_inference_function, get_gradient_functions) from .exceptions import no_gradients_available, raise_invalid_operator from .forgrad import forgrad diff --git a/xplique/commons/operators.py b/xplique/commons/operators.py index 62af7a1f..5770420e 100644 --- a/xplique/commons/operators.py +++ b/xplique/commons/operators.py @@ -2,14 +2,13 @@ Custom tensorflow operator for Attributions """ -import inspect -from enum import Enum +from deprecated import deprecated import tensorflow as tf -from ..types import Callable, Optional, Union, OperatorSignature -from .exceptions import raise_invalid_operator, no_gradients_available -from .callable_operations import predictions_one_hot_callable +from ..types import Callable, Optional +from ..utils_functions.object_detection import _box_iou, _format_objects, _EPSILON + @tf.function def predictions_operator(model: Callable, @@ -36,6 +35,7 @@ def predictions_operator(model: Callable, return scores @tf.function +@deprecated(version="1.0.0", reason="Gradient-based explanations are zeros with this operator.") def regression_operator(model: Callable, inputs: tf.Tensor, targets: tf.Tensor) -> tf.Tensor: @@ -63,294 +63,162 @@ def regression_operator(model: Callable, @tf.function -def binary_segmentation_operator(model: Callable, - inputs: tf.Tensor, - targets: tf.Tensor) -> tf.Tensor: +def semantic_segmentation_operator(model, inputs, targets): """ - Compute the segmentation score for a batch of samples. + Explain the class of a zone of interest. Parameters ---------- model Model used for computing predictions. + The model outputs should be between 0 and 1, otherwise, applying a softmax is recommended. inputs Input samples to be explained. + Expected shape of (n, h, w, c_in), with c_in the number of channels of the input. targets - One-hot encoded labels or regression target (e.g {+1, -1}), one for each sample. + Tensor, a mask indicating the zone and class to explain. + It contains the model predictions limited to a certain zone and channel. + The zone indicates the zone of interest and the channel the class of interest. + For more detail and examples please refer to the documentation. + https://deel-ai.github.io/xplique/latest/api/attributions/semantic_segmentation/ + Expected shape of (n, h, w, c_out), with c_out the number of classes. + `targets` can also be designed to explain the border of a zone of interest. Returns ------- scores Segmentation scores computed. """ - scores = tf.reduce_sum(model(inputs) * targets, axis=(1, 2)) - return scores + # compute absolute difference between prediction and target on targets zone + scores = model(inputs) * targets + + # take mean over the zone and channel of interest + return tf.reduce_sum(scores, axis=(1, 2, 3)) /\ + tf.reduce_sum(tf.cast(tf.not_equal(targets, 0), tf.float32), axis=(1, 2, 3)) @tf.function -def segmentation_operator(model: Callable, - inputs: tf.Tensor, - targets: tf.Tensor) -> tf.Tensor: - """ - Compute the segmentation score for a batch of samples. +def object_detection_operator(model: Callable, + inputs: tf.Tensor, + targets: tf.Tensor, + intersection_score_fn: Optional[Callable] = _box_iou, + include_detection_probability: Optional[bool] = True, + include_classification_score: Optional[bool] = True,) -> tf.Tensor: + """ + Compute the object detection scores for a batch of samples. + + For a given image, there are two possibilities: + - One box per image is provided: Then, in the case of perturbation-based methods, + the model makes prediction on the perturbed image and choose the most similar predicted box. + This similarity is computed following the DRise method. + In the case of gradient-based methods, the gradient is computed from the same score. + - Several boxes are provided for one image: In this case, the attributions for each box are + computed and the mean is taken. + + Therefore, to explain each box separately, the easiest way is to call the attribution method + with a batch of the same image tiled to match the number of predicted box. + In this case, inputs and targets shapes should be: (nb_boxes, H, W, C) and (nb_boxes, (5 + nc)). + + This work is a generalization of the following article at any kind of attribution method. + Ref. Petsiuk & al., Black-box Explanation of Object Detectors via Saliency Maps (2021). + https://arxiv.org/pdf/2006.03204.pdf Parameters ---------- model - Model used for computing predictions. + Model used for computing object detection prediction. + The model should have input and output shapes of (N, H, W, C) and (N, nb_boxes, (4+1+nc)). + The model should not include the NMS computation, + it is not differentiable and drastically reduce the number of boxes for the matching. inputs - Input samples to be explained. + Batched input samples to be explained. Expected shape (N, H, W, C). + More information in the documentation. targets - One-hot encoded labels or regression target (e.g {+1, -1}), one for each sample. + Specify the box are boxes to explain for each input. Preferably, after the NMS. + It should be of shape (N, (4 + 1 + nc)) or (N, nb_boxes, (4 + 1 + nc)), + with nc the number of classes, + N the number of samples in the batch (it should match `inputs`), + and nb_boxes the number of boxes to explain simultaneously. + + (4 + 1 + nc) means: [boxes_coordinates, proba_detection, one_hots_classifications]. + + In the case the nb_boxes dimension is not 1, + several boxes will be explained at the same time. + To be more precise, explanations will be computed for each box and the mean is returned. + intersection_score_fn + Function that computes the intersection score between two bounding boxes coordinates. + This function is batched. The default value is `_box_iou` computing IOU scores. + include_detection_probability + Boolean encoding if the box objectness (or detection probability) + should be included in DRise score. + include_classification_score + Boolean encoding if the class associated to the box should be included in DRise score. Returns ------- scores - Segmentation scores computed. - """ - scores = tf.reduce_sum(model(inputs) * targets, axis=(1, 2, 3)) - return scores - -class Tasks(Enum): - """ - Enumeration of different tasks for which we have defined operators - """ - CLASSIFICATION = predictions_operator - REGRESSION = regression_operator - - @staticmethod - def from_string(operator_name: str) -> "Tasks": - """ - Restore an operator from a string - - Parameters - ---------- - operator_name - String indicating the operator to restore: must be one - of 'classification' or 'regression' - - Returns - ------- - operator - The Tasks object - """ - assert operator_name in [ - "classification", - "regression", - ], "Only 'classification' and 'regression' are supported." - - if operator_name == "regression": - return Tasks.REGRESSION - return Tasks.CLASSIFICATION - -def check_operator(operator: Callable): - """ - Check if the operator is valid g(f, x, y) -> tf.Tensor - and raise an exception and return true if so. - - Parameters - ---------- - operator - Operator to check - - Returns - ------- - is_valid - True if the operator is valid, False otherwise. - """ - # handle tf functions - # pylint: disable=protected-access - if hasattr(operator, '_python_function'): - return check_operator(operator._python_function) - - # the operator must be callable - if not hasattr(operator, '__call__'): - raise_invalid_operator() - - # the operator should take at least three arguments - args = inspect.getfullargspec(operator).args - if len(args) < 3: - raise_invalid_operator() - - return True - -def get_operator( - operator: Optional[Union[Tasks, str, OperatorSignature]]): - """ - This function allows to retrieve an operator from: a Tasks, a task name. If the operator - is a custom one, we simply check if its signature is correct - - Parameters - ---------- - operator - An operator from the Tasks enum or the task name or a custom operator. If None, use a - classification operator. - - Returns - ------- - operator - The operator requested - """ - # case when no operator is provided - if operator is None: - return predictions_operator - - # case when the query is a string - if isinstance(operator, str): - return Tasks.from_string(operator) - - # case when the query belong to the Tasks enum - if operator in [t.value for t in Tasks]: - return operator - - # case when the operator is a custom one - assert check_operator(operator) - return operator - -def get_gradient_of_operator(operator): - """ - Get the gradient of an operator. - - Parameters - ---------- - operator - Operator to compute the gradient of. - - Returns - ------- - gradient - Gradient of the operator. - """ - @tf.function - def gradient(model, inputs, targets): - with tf.GradientTape() as tape: - tape.watch(inputs) - scores = operator(model, inputs, targets) - - return tape.gradient(scores, inputs) - - return gradient - - -def operator_batching(operator: OperatorSignature) -> tf.Tensor: - """ - Take care of batching an operator: (model, inputs, labels). - - Parameters - ---------- - operator - Any callable that take model, inputs and labels as parameters. - - Returns - ------- - batched_operator - Function that apply operator by batch. + Object detection scores computed following DRise definition: + intersection_score * proba_detection * classification_similarity """ + def batch_loop(args): + # function to loop on for `tf.map_fn` + obj, obj_ref = args - def batched_operator(model, inputs, targets, batch_size=None): - if batch_size is not None: - dataset = tf.data.Dataset.from_tensor_slices((inputs, targets)) - results = tf.concat([ - operator(model, x, y) - for x, y in dataset.batch(batch_size) - ], axis=0) - else: - results = operator(model, inputs, targets) + if obj is None or obj.shape[0] == 0: + return tf.constant(0.0, dtype=inputs.dtype) - return results + # compute predicted boxes for a given image + # (nb_box_pred, 4), (nb_box_pred, 1), (nb_box_pred, nb_classes) + current_boxes, proba_detection, classification = _format_objects(obj) + size = tf.shape(current_boxes)[0] - return batched_operator + if len(tf.shape(obj_ref)) == 1: + obj_ref = tf.expand_dims(obj_ref, axis=0) + # DRise consider the reference objectness to be 1 + # (nb_box_ref, 4), _, (nb_box_ref, nb_classes) + boxes_refs, _, class_refs = _format_objects(obj_ref) -batch_predictions = operator_batching(predictions_operator) -gradients_predictions = get_gradient_of_operator(predictions_operator) -batch_gradients_predictions = operator_batching(gradients_predictions) -batch_predictions_one_hot_callable = operator_batching(predictions_one_hot_callable) + # (nb_box_ref, nb_box_pred, 4) + boxes_refs = tf.repeat(tf.expand_dims(boxes_refs, axis=1), repeats=size, axis=1) + # (nb_box_ref, nb_box_pred) + intersection_score = intersection_score_fn(boxes_refs, current_boxes) -def get_inference_function( - model: Callable, - operator: Optional[OperatorSignature] = None): - """ - Define the inference function according to the model type + # (nb_box_pred,) + detection_probability = tf.squeeze(proba_detection, axis=1) - Parameters - ---------- - model - Model used for computing explanations. - operator - Function g to explain, g take 3 parameters (f, x, y) and should return a scalar, - with f the model, x the inputs and y the targets. If None, use the standard - operator g(f, x, y) = f(x)[y]. - - Returns - ------- - inference_function - Same definition as the operator. - batch_inference_function - An inference function which treat inputs and targets by batch, - it has an additionnal parameter `batch_size`. - """ - if operator is not None: - # user specified a string, an operator from the ones available or a - # custom operator, we check if the operator is valid - # and we wrap it to generate a batching version of this operator - operator = get_operator(operator) - inference_function = operator - batch_inference_function = operator_batching(operator) + # set detection probability to 1 if it should be included + detection_probability = tf.cond(tf.cast(include_detection_probability, tf.bool), + true_fn=lambda: detection_probability, + false_fn=lambda: tf.ones_like(detection_probability)) - elif isinstance(model, (tf.keras.Model, tf.Module, tf.keras.layers.Layer)): - inference_function = predictions_operator - batch_inference_function = batch_predictions + # (nb_box_ref, nb_box_pred, nb_classes) + class_refs = tf.repeat(tf.expand_dims(class_refs, axis=1), repeats=size, axis=1) - else: - # completely unknown model (e.g. sklearn), we can't backprop through it - inference_function = predictions_one_hot_callable - batch_inference_function = batch_predictions_one_hot_callable + # (nb_box_ref, nb_box_pred) + classification_score = tf.reduce_sum(class_refs * classification, axis=-1) \ + / (tf.norm(classification, axis=-1) * tf.norm(class_refs, axis=-1)+ _EPSILON) - return inference_function, batch_inference_function + # set classification score to 1 if it should be included + classification_score = tf.cond(tf.cast(include_classification_score, tf.bool), + true_fn=lambda: classification_score, + false_fn=lambda: tf.ones_like(classification_score)) + # Compute score as defined in DRise for all possible pair of boxes + # (nb_box_ref, nb_box_pred) + boxes_pairwise_scores = intersection_score \ + * detection_probability \ + * classification_score -def get_gradient_functions( - model: Callable, - operator: Optional[OperatorSignature] = None): - """ - Define the gradient function according to the model type + # select for a reference box the most similar predicted box score + # (nb_box_ref,) + ref_boxes_scores = tf.reduce_max(boxes_pairwise_scores, axis=1) - Parameters - ---------- - model - Model used for computing explanations. - operator - Function g to explain, g take 3 parameters (f, x, y) and should return a scalar, - with f the model, x the inputs and y the targets. If None, use the standard - operator g(f, x, y) = f(x)[y]. + # get an attribution for several boxes in the same time + # () + image_score = tf.reduce_mean(ref_boxes_scores) + return image_score - Returns - ------- - gradient - Gradient function of the operator. - batch_gradient - An gradient function which treat inputs and targets by batch, - it has an additionnal parameter `batch_size`. - """ - if operator is not None: - # user specified a string, an operator from the ones available or a - # custom operator, we check if the operator is valid - # and we wrap it to generate a batching version of this operator - operator = get_operator(operator) - gradient = get_gradient_of_operator(operator) - batch_gradient = operator_batching(gradient) - - elif isinstance(model, tf.keras.Model): - # no custom operator, for keras model we can backprop through the model - gradient = gradients_predictions - batch_gradient = batch_gradients_predictions - - else: - # custom model or completely unknown model (e.g. sklearn), we can't backprop through it - gradient = no_gradients_available - batch_gradient = no_gradients_available - - return gradient, batch_gradient - \ No newline at end of file + objects = model(inputs) + return tf.map_fn(batch_loop, (objects, targets), fn_output_signature=tf.float32) diff --git a/xplique/commons/operators_operations.py b/xplique/commons/operators_operations.py new file mode 100644 index 00000000..09c815a3 --- /dev/null +++ b/xplique/commons/operators_operations.py @@ -0,0 +1,296 @@ +""" +Custom tensorflow operator for Attributions +""" + +import inspect +from enum import Enum + +import tensorflow as tf + +from ..types import Callable, Optional, Union, OperatorSignature +from .exceptions import raise_invalid_operator, no_gradients_available +from .callable_operations import predictions_one_hot_callable +from .operators import (predictions_operator, # regression_operator, + semantic_segmentation_operator, object_detection_operator) + + +class Tasks(Enum): + """ + Enumeration of different tasks for which we have defined operators + """ + CLASSIFICATION = predictions_operator + # `regression_operator` do not work for gradient-based method + # the problem is its use for multi output regression + # REGRESSION = regression_operator + REGRESSION = predictions_operator + SEMANTIC_SEGMENTATION = semantic_segmentation_operator + OBJECT_DETECTION = object_detection_operator + + # object detection operator limited to box position explanation + OBJECT_DETECTION_BOX_POSITION = lambda model, inputs, targets:( + object_detection_operator(model, inputs, targets, + include_detection_probability=False, + include_classification_score=False) + ) + + # object detection operator limited to box proba and position explanation + OBJECT_DETECTION_BOX_PROBA = lambda model, inputs, targets:( + object_detection_operator(model, inputs, targets, + include_detection_probability=True, + include_classification_score=False) + ) + + # object detection operator limited to box class and position explanation + OBJECT_DETECTION_BOX_CLASS = lambda model, inputs, targets:( + object_detection_operator(model, inputs, targets, + include_detection_probability=False, + include_classification_score=True) + ) + + @staticmethod + def from_string(operator_name: str) -> "Tasks": + """ + Restore an operator from a string + + Parameters + ---------- + operator_name + String indicating the operator to restore: must be one + of 'classification' or 'regression' + + Returns + ------- + operator + The Tasks object + """ + + string_to_tasks = { + "classification": Tasks.CLASSIFICATION, + "regression": Tasks.REGRESSION, + "semantic segmentation": Tasks.SEMANTIC_SEGMENTATION, + "object detection": Tasks.OBJECT_DETECTION, + "object detection box position": Tasks.OBJECT_DETECTION_BOX_POSITION, + "object detection box proba": Tasks.OBJECT_DETECTION_BOX_PROBA, + "object detection box class": Tasks.OBJECT_DETECTION_BOX_CLASS, + } + + assert operator_name in string_to_tasks,\ + f"Only `operator` value among {string_to_tasks.keys()} are supported,\n "+\ + f"but {operator_name} was given." + + return string_to_tasks[operator_name] + + +def check_operator(operator: Callable): + """ + Check if the operator is valid g(f, x, y) -> tf.Tensor + and raise an exception and return true if so. + + Parameters + ---------- + operator + Operator to check + + Returns + ------- + is_valid + True if the operator is valid, False otherwise. + """ + # handle tf functions + # pylint: disable=protected-access + if hasattr(operator, '_python_function'): + return check_operator(operator._python_function) + + # the operator must be callable + if not hasattr(operator, '__call__'): + raise_invalid_operator() + + # the operator should take at least three arguments + args = inspect.getfullargspec(operator).args + if len(args) < 3: + raise_invalid_operator() + + return True + + +def get_operator( + operator: Optional[Union[Tasks, str, OperatorSignature]]): + """ + This function allows to retrieve an operator from: a Tasks, a task name. If the operator + is a custom one, we simply check if its signature is correct + + Parameters + ---------- + operator + An operator from the Tasks enum or the task name or a custom operator. If None, use a + classification operator. + + Returns + ------- + operator + The operator requested + """ + # case when no operator is provided + if operator is None: + return predictions_operator + + # case when the query is a string + if isinstance(operator, str): + return Tasks.from_string(operator) + + # case when the query belong to the Tasks enum + if operator in [t.value for t in Tasks]: + return operator + + # case when the operator is a custom one + assert check_operator(operator) + return operator + + +def get_gradient_of_operator(operator): + """ + Get the gradient of an operator. + + Parameters + ---------- + operator + Operator of which to compute the gradient. + + Returns + ------- + gradient + Gradient of the operator. + """ + @tf.function + def gradient(model, inputs, targets): + with tf.GradientTape() as tape: + tape.watch(inputs) + scores = operator(model, inputs, targets) + + return tape.gradient(scores, inputs) + + return gradient + + +def operator_batching(operator: OperatorSignature) -> tf.Tensor: + """ + Take care of batching an operator: (model, inputs, labels). + + Parameters + ---------- + operator + Any callable that take model, inputs and labels as parameters. + + Returns + ------- + batched_operator + Function that apply operator by batch. + """ + + def batched_operator(model, inputs, targets, batch_size=None): + if batch_size is not None: + dataset = tf.data.Dataset.from_tensor_slices((inputs, targets)) + results = tf.concat([ + operator(model, x, y) + for x, y in dataset.batch(batch_size) + ], axis=0) + else: + results = operator(model, inputs, targets) + + return results + + return batched_operator + + +batch_predictions = operator_batching(predictions_operator) +gradients_predictions = get_gradient_of_operator(predictions_operator) +batch_gradients_predictions = operator_batching(gradients_predictions) +batch_predictions_one_hot_callable = operator_batching(predictions_one_hot_callable) + + +def get_inference_function( + model: Callable, + operator: Optional[OperatorSignature] = None): + """ + Define the inference function according to the model type + + Parameters + ---------- + model + Model used for computing explanations. + operator + Function g to explain, g take 3 parameters (f, x, y) and should return a scalar, + with f the model, x the inputs and y the targets. If None, use the standard + operator g(f, x, y) = f(x)[y]. + + Returns + ------- + inference_function + Same definition as the operator. + batch_inference_function + An inference function which treat inputs and targets by batch, + it has an additional parameter `batch_size`. + """ + if operator is not None: + # user specified a string, an operator from the ones available or a + # custom operator, we check if the operator is valid + # and we wrap it to generate a batching version of this operator + operator = get_operator(operator) + inference_function = operator + batch_inference_function = operator_batching(operator) + + elif isinstance(model, (tf.keras.Model, tf.Module, tf.keras.layers.Layer)): + inference_function = predictions_operator + batch_inference_function = batch_predictions + + else: + # completely unknown model (e.g. sklearn), we can't backprop through it + inference_function = predictions_one_hot_callable + batch_inference_function = batch_predictions_one_hot_callable + + return inference_function, batch_inference_function + + +def get_gradient_functions( + model: Callable, + operator: Optional[OperatorSignature] = None): + """ + Define the gradient function according to the model type + + Parameters + ---------- + model + Model used for computing explanations. + operator + Function g to explain, g take 3 parameters (f, x, y) and should return a scalar, + with f the model, x the inputs and y the targets. If None, use the standard + operator g(f, x, y) = f(x)[y]. + + Returns + ------- + gradient + Gradient function of the operator. + batch_gradient + An gradient function which treat inputs and targets by batch, + it has an additional parameter `batch_size`. + """ + if operator is not None: + # user specified a string, an operator from the ones available or a + # custom operator, we check if the operator is valid + # and we wrap it to generate a batching version of this operator + operator = get_operator(operator) + gradient = get_gradient_of_operator(operator) + batch_gradient = operator_batching(gradient) + + elif isinstance(model, tf.keras.Model): + # no custom operator, for keras model we can backprop through the model + gradient = gradients_predictions + batch_gradient = batch_gradients_predictions + + else: + # custom model or completely unknown model (e.g. sklearn), we can't backprop through it + gradient = no_gradients_available + batch_gradient = no_gradients_available + + return gradient, batch_gradient + \ No newline at end of file diff --git a/xplique/features_visualizations/maco.py b/xplique/features_visualizations/maco.py index 4b051e4a..f03ce173 100644 --- a/xplique/features_visualizations/maco.py +++ b/xplique/features_visualizations/maco.py @@ -71,7 +71,7 @@ def maco(objective: Objective, # default to box_size that go from 50% to 5% box_size_values = tf.cast(np.linspace(0.5, 0.05, nb_steps), tf.float32) get_box_size = lambda step_i: box_size_values[step_i] - elif isinstance(box_size, Callable): + elif hasattr(box_size, "__call__"): get_box_size = box_size elif isinstance(box_size, float): get_box_size = lambda _ : box_size @@ -82,7 +82,7 @@ def maco(objective: Objective, # default to large noise to low noise noise_values = tf.cast(np.logspace(0, -4, nb_steps), tf.float32) get_noise_intensity = lambda step_i: noise_values[step_i] - elif isinstance(noise_intensity, Callable): + elif hasattr(noise_intensity, "__call__"): get_noise_intensity = noise_intensity elif isinstance(noise_intensity, float): get_noise_intensity = lambda _ : noise_intensity diff --git a/xplique/utils_functions/__init__.py b/xplique/utils_functions/__init__.py new file mode 100644 index 00000000..4c1d8941 --- /dev/null +++ b/xplique/utils_functions/__init__.py @@ -0,0 +1,5 @@ +""" +Functions to ease attributions +""" + +from .segmentation import get_class_zone, get_connected_zone, get_in_out_border, get_common_border diff --git a/xplique/utils_functions/object_detection.py b/xplique/utils_functions/object_detection.py new file mode 100644 index 00000000..c8bbbe6b --- /dev/null +++ b/xplique/utils_functions/object_detection.py @@ -0,0 +1,72 @@ +""" +Operator for object detection +""" +from typing import Tuple +import tensorflow as tf + +_EPSILON = tf.constant(1e-4) + + +def _box_iou(boxes_a: tf.Tensor, boxes_b: tf.Tensor) -> tf.Tensor: + """ + Compute the intersection between two batched bounding boxes. + Each bounding box is defined by (x1, y1, x2, y2) respectively (left, bottom, right, top). + With left < right and bottom < top + + Parameters + ---------- + boxes_a + First batch of bounding boxes. + boxes_b + Second batch of bounding boxes. + + Returns + ------- + iou_score + The IOU score between the two batches of bounding boxes. + """ + + # determine the intersection rectangle + left = tf.maximum(boxes_a[..., 0], boxes_b[..., 0]) + bottom = tf.maximum(boxes_a[..., 1], boxes_b[..., 1]) + right = tf.minimum(boxes_a[..., 2], boxes_b[..., 2]) + top = tf.minimum(boxes_a[..., 3], boxes_b[..., 3]) + + intersection_area = tf.math.maximum(right - left, 0) * tf.math.maximum(top - bottom, 0) + + # determine the areas of the prediction and ground-truth rectangles + a_area = (boxes_a[..., 2] - boxes_a[..., 0]) * (boxes_a[..., 3] - boxes_a[..., 1]) + b_area = (boxes_b[..., 2] - boxes_b[..., 0]) * (boxes_b[..., 3] - boxes_b[..., 1]) + + union_area = a_area + b_area - intersection_area + + iou_score = intersection_area / (union_area + _EPSILON) + + return iou_score + + +def _format_objects(predictions: tf.Tensor) -> Tuple[tf.Tensor, tf.Tensor, tf.Tensor]: + """ + Format bounding boxes prediction for object detection operator. + It takes a batch of bounding boxes predictions of the model and divide it between + boxes_coordinates, proba_detection, and one_hots_classifications. + + Parameters + ---------- + predictions + Batch of bounding boxes predictions of shape (nb_boxes, (4 + 1 + nc)). + (4 + 1 + nc) means: [boxes_coordinates, proba_detection, one_hots_classifications]. + Where nc is the number of classes. + + Returns + ------- + boxes_coordinates + A Tensor of shape (nb_boxes, 4) encoding the boxes coordinates. + proba_detection + A Tensor of shape (nb_boxes, 1) encoding the detection probabilities. + one_hots_classifications + A Tensor of shape (nb_boxes, nc) encoding the class predictions. + """ + boxes_coordinates, proba_detection, one_hots_classifications = \ + tf.split(predictions, [4, 1, tf.shape(predictions[0])[0] - 5], 1) + return boxes_coordinates, proba_detection, one_hots_classifications diff --git a/xplique/utils_functions/segmentation.py b/xplique/utils_functions/segmentation.py new file mode 100644 index 00000000..4a1c6824 --- /dev/null +++ b/xplique/utils_functions/segmentation.py @@ -0,0 +1,285 @@ +""" +Functions to prepare `targets` for segmentation attributions +""" + +import cv2 +import numpy as np +import tensorflow as tf + +from ..types import Union, Tuple, List + + +def get_class_zone(predictions: Union[tf.Tensor, np.array], class_id: int) -> tf.Tensor: + """ + Extract a mask for the class `c`. + The mask correspond to the pixels where the maximum prediction correspond to the class `c`. + Other classes channels are set to zero. + + Parameters + ---------- + predictions + Output of the model, it should be the output of a softmax function. + We assume the shape (h, w, c). + class_id + Index of the channel of the class of interest. + + Returns + ------- + class_zone_mask + Mask of the zone corresponding to the class of interest. + Only the corresponding channel is non-zero. + The shape is the same as `predictions`, (h, w, c). + """ + assert len(tf.shape(predictions)) == 3, "predictions should correspond to only one image" + + class_zone = tf.cast(tf.argmax(predictions, axis=-1) == class_id, tf.float32) + class_zone_mask = tf.Variable(tf.zeros(predictions.shape)) + class_zone_mask = class_zone_mask[:, :, class_id].assign(class_zone) + + assert tf.reduce_sum(class_zone_mask) >= 1 + + return class_zone_mask + + +def get_connected_zone( + predictions: Union[tf.Tensor, np.array], coordinates: Tuple[int, int] +) -> tf.Tensor: + """ + Extract a connected mask around `coordinates`. + The mask correspond to the pixels where the maximum prediction correspond + to the maximum predicted class at `coordinates`. + This class mask is then limited to the connected zone around `coordinates`. + Other classes channels are set to zero. + + Parameters + ---------- + predictions + Output of the model, it should be the output of a softmax function. + We assume the shape (h, w, c). + coordinates + Tuple of coordinates of the point inside the zone of interest. + + Returns + ------- + connected_zone_mask + Mask of the connected zone around `coordinates` with similar class prediction. + Only the corresponding channel is non-zero. + The shape is the same as `predictions`, (h, w, c). + """ + assert len(tf.shape(predictions)) == 3, "predictions should correspond to only one image" + + assert ( + coordinates[0] < predictions.shape[0] + ), f"Coordinates should be included in the shape, i.e. {coordinates[0]}<{predictions.shape[0]}" + assert ( + coordinates[1] < predictions.shape[1] + ), f"Coordinates should be included in the shape, i.e. {coordinates[1]}<{predictions.shape[1]}" + + labels = tf.argmax(predictions, axis=-1) + class_id = labels[coordinates[0], coordinates[1]] + mask = labels == class_id + mask = np.uint8(np.array(mask)[:, :, np.newaxis] * 255) + + components_masks = cv2.connectedComponents(mask)[1] # pylint: disable=no-member + + component_id = components_masks[coordinates[0], coordinates[1]] + connected_zone = tf.cast(components_masks == component_id, tf.float32) + + connected_zone_mask = tf.Variable(tf.zeros(predictions.shape)) + connected_zone_mask = connected_zone_mask[:, :, class_id].assign(connected_zone) + + assert tf.reduce_sum(connected_zone_mask) >= 1 + assert connected_zone_mask[coordinates[0], coordinates[1], class_id] != 0 + + return connected_zone_mask + + +def list_class_connected_zones( + predictions: Union[tf.Tensor,np.array], + class_id: int, + zone_minimum_size: int = 100 +) -> List[tf.Tensor]: + """ + List all connected zones for a given class. + A connected zone is a set of pixels next to each others + where the maximum prediction correspond to the same class. + This function generate a list of connected zones, + each element of the list have a similar format to `get_connected_zone` outputs. + + Parameters + ---------- + predictions + Output of the model, it should be the output of a softmax function. + We assume the shape (h, w, c). + class_id + Index of the channel of the class of interest. + zone_minimum_size + Threshold of number of pixels under which zones are not returned. + + Returns + ------- + connected_zones_masks_list + List of the connected zones masks for a given class. + Each zone predictions shape is the same as `predictions`, (h, w, c). + Only the corresponding channel is non-zero. + """ + assert len(tf.shape(predictions)) == 3, "predictions should correspond to only one image" + + labels = tf.argmax(predictions, axis=-1) + mask = labels == class_id + mask = np.uint8(np.array(mask)[:, : , np.newaxis] * 255) + + components_masks = cv2.connectedComponents(mask)[1] # pylint: disable=no-member + + sizes = np.bincount(components_masks.ravel()) + + connected_zones_masks_list = [] + for component_id, size in enumerate(sizes[1:]): + + if size > zone_minimum_size: + + connected_zone = tf.cast(components_masks == (component_id + 1), tf.float32) + + all_channels_class_zone_mask = tf.Variable(tf.zeros(predictions.shape)) + all_channels_class_zone_mask =\ + all_channels_class_zone_mask[:, :, class_id].assign(connected_zone) + + assert tf.reduce_sum(all_channels_class_zone_mask) >= 1 + + connected_zones_masks_list.append(all_channels_class_zone_mask) + + return connected_zones_masks_list + + + +def get_in_out_border( + class_zone_mask: Union[tf.Tensor, np.array], +) -> tf.Tensor: + """ + Extract the border of a zone of interest, then put `1` on the + inside border and `-1` on the outside border. + + Examples of coefficients extracted from the class channel of the class of interest: + + ``` + # class_zone_mask[:, :, c] + [[0, 0, 0, 0, 0], + [0, 0, 0, 1, 1], + [0, 0, 1, 1, 1], + [0, 0, 1, 1, 1], + [0, 1, 1, 1, 1]] + + # border_mask + [[ 0, 0, -1, -1, -1], + [ 0, -1, -1, 1, 1], + [ 0, -1, 1, 1, 0], + [-1, -1, 1, 0, 0], + [-1, 1, 1, 0, 0]] + ``` + + Parameters + ---------- + class_zone_mask + Mask delimiting the zone of interest, for the class of interest + only one channel should have non-zero values, + the one corresponding to the class. + We assume the shape (h, w, c) same as the model output for one element. + + Returns + ------- + class_borders_masks + Mask of the borders of the zone of the class of interest. + Only the corresponding channel is non-zero. + Inside borders are set to `1` and outside borders are set to `-1`. + The shape is the same as `class_zone_mask`, (h, w, c). + """ + assert len(tf.shape(class_zone_mask)) == 3,\ + "class_zone_mask should correspond to only one image" + + # channel of the class of interest + channel_mean = tf.reduce_sum(tf.cast(class_zone_mask, tf.int32), axis=[0, 1]) + assert ( + int(tf.reduce_sum(channel_mean)) >= 1 + ), "The specified `class_target_mask` is empty." + class_id = int(tf.argmax(channel_mean)) + + # set other values to -1 on the target zone to 1 + binary_mask = 2 * tf.cast(class_zone_mask[:, :, class_id], tf.int32) - 1 + + # extend size with padding for convolution + extended_binary_mask = tf.pad( + binary_mask, + tf.constant([[1, 1], [1, 1]]), + "SYMMETRIC", + ) + + kernel = tf.convert_to_tensor( + [[-1, -1, -1], [-1, 13, -1], [-1, -1, -1]], dtype=tf.int32 + ) + kernel = kernel[:, :, tf.newaxis, tf.newaxis] + conv_result = tf.nn.conv2d( + tf.expand_dims(tf.expand_dims(extended_binary_mask, axis=0), axis=-1), + kernel, + strides=1, + padding="VALID", + )[0, :, :, 0] + + # 6 < in < 21, -21 < out < -6 + in_border = tf.logical_and( + tf.math.less(tf.constant([6]), conv_result), + tf.math.less(conv_result, tf.constant([21])), + ) + out_border = tf.logical_and( + tf.math.less(tf.constant([-21]), conv_result), + tf.math.less(conv_result, tf.constant([-6])), + ) + + border_mask = ( + tf.zeros(binary_mask.shape) + + tf.cast(in_border, tf.float32) + - tf.cast(out_border, tf.float32) + ) + + class_borders_masks = tf.Variable(tf.zeros(class_zone_mask.shape)) + class_borders_masks = class_borders_masks[:, :, class_id].assign(border_mask) + + assert int(tf.reduce_sum(tf.abs(class_borders_masks))) >= 1 + + return class_borders_masks + + +def get_common_border( + border_mask_1: Union[tf.Tensor, np.array], border_mask_2: Union[tf.Tensor, np.array] +) -> tf.Tensor: + """ + Compute the common part between `border_mask_1` and `border_mask_2` masks. + Those borders should be computed using `get_in_out_border`. + + Parameters + ---------- + border_mask_1 + Border of the first zone of interest. Computed with `get_in_out_border`. + border_mask_2 + Border of the second zone of interest. Computed with `get_in_out_border`. + + Returns + ------- + common_borders_masks + Mask of the common borders between two zones of interest. + Only the two corresponding channels are non-zero. + Inside borders are set to `1` and outside borders are set to `-1`, + Respectively on the two channels. + The shape is the same as the input border masks, (h, w, c). + """ + all_channel_border_mask_1 = tf.reduce_any(border_mask_1 != 0, axis=-1) + all_channel_border_mask_2 = tf.reduce_any(border_mask_2 != 0, axis=-1) + + common_pixels_mask = tf.logical_and( + all_channel_border_mask_1, all_channel_border_mask_2 + ) + + assert tf.reduce_any(common_pixels_mask), "No common border between the two masks." + + return (border_mask_1 + border_mask_2) * tf.expand_dims( + tf.cast(common_pixels_mask, tf.float32), -1 + ) diff --git a/xplique/wrappers/pytorch.py b/xplique/wrappers/pytorch.py index c1c912bf..78e0a1ce 100644 --- a/xplique/wrappers/pytorch.py +++ b/xplique/wrappers/pytorch.py @@ -1,5 +1,5 @@ """ -Module for having a wrapper for PyTorch's model +Module for having a wrapper for PyTorch models """ import warnings @@ -10,7 +10,7 @@ class TorchWrapper(tf.keras.Model): """ - A wrapper for PyTorch's model so that they can be used in Xplique framework + A wrapper for PyTorch models so that they can be used in Xplique framework for most attribution methods Parameters @@ -53,7 +53,7 @@ def __init__(self, torch_model: "nn.Module", device: Union["torch.device", str], self.channel_first = is_channel_first # deactivate all tf.function tf.config.run_functions_eagerly(True) - warnings.warn("TF is set to run eagerly to avoid conflict with Pytorch. Thus,\ + warnings.warn("TF is set to run eagerly to avoid conflict with PyTorch. Thus,\ TF functions might be slower") # pylint: disable=arguments-differ @@ -139,7 +139,7 @@ def np_img_to_torch(self, np_inputs: np.ndarray): def _has_conv_layers(self): """ - A method that checks if the PyTorch's model has 2D convolutional layer. + A method that checks if the PyTorch models has 2D convolutional layer. Indeed, convolution with PyTorch expects inputs in the shape (N, C, H, W) where TF expect (N, H, W, C).