Explain predictions of Keras image classifiers (Grad-CAM) #315

teabolt · 2019-06-10T14:04:05Z

This PR adds explanations for Keras models that are used to classify images. Specifically we implement Grad-CAM.

For example, the following piece of code:

import keras
import numpy as np
import eli5

# load model
xception = keras.applications.xception.Xception(include_top=True, weights='imagenet', classes=1000)

# load image
im = keras.preprocessing.image.load_img('../eli5_examples/motorcycle.jpg', target_size=(299, 299))
doc = keras.preprocessing.image.img_to_array(im)
doc = np.expand_dims(doc, axis=0)
keras.applications.xception.preprocess_input(doc)

# explain
eli5.show_prediction(xception, doc)

produces this explanation for the class 'motor scooter':

The following features are added:

Add keras package with explain_prediction_keras() and formatters.image module with format_as_image() (requires matplotlib and Pillow).
Add .image attribute to base.Explanation and .heatmap to base.TargetExplanation.
Make ipython.show_prediction() dispatch to an image display function for image explanations.

TODO items before this PR is finalized:

Resolve reviews
~~Coverage~~
~~Pass CI~~
~~Mypy type annotations~~
~~Docs (formatting, tutorial)~~
~~Integration and unit tests~~

… dimensions)

…is supplied

…ications preprocess_input. Fix using callable to find target layer

…ain_prediction via approximate attention over area

… be a Layer instance)

Co-Authored-By: Mikhail Korobov <[email protected]>

tests/test_keras_integration.py

kmike · 2019-07-10T17:17:24Z

eli5/sklearn/text.py

@@ -13,10 +13,11 @@


 def get_weighted_spans(doc, vec, feature_weights):
-    # type: (Any, Any, FeatureWeights) -> Optional[WeightedSpans]
+    # type: (Any, Any, Union[FeatureWeights, None]) -> Optional[WeightedSpans]


it seems this function requires FeatureWeights to be not None, so maybe it makes sense to keep type signature the same, but move assert to the caller code

Moved the assert to add_weighted_spans. Good one. Strange that https://github.com/TeamHG-Memex/eli5/search?q=get_weighted_spans did not show add_weighted_spans's call to get_weighted_spans.

913d415

kmike · 2019-07-10T17:18:25Z

eli5/keras/gradcam.py

+
+
+def _get_target_prediction(targets, estimator):
+    # type: (Union[None, list], Model) -> K.variable


Optional[List]

Ah I was thinking whether to use this. My thinking was that targets is not actually optional in the parameter list so it might be confusing. But I see that the rest of the library uses Optional (i.e. for functions that could return None) so I will change it.

e8a34f1

eli5/keras/gradcam.py

kmike · 2019-07-10T17:24:38Z

eli5/keras/explain_prediction.py

+        a valid keras layer name, layer index, or an instance of a Keras layer.
+
+        If None, a suitable layer is attempted to be retrieved. 
+        See :func:`eli5.keras._search_layer_backwards` for details.


I think if we're documenting a function, it makes sense to make it public - or just document the behavior without mentioning a function.

Good catch. Forgot that I made the function private.

bcaf7ca

kmike · 2019-07-10T17:25:50Z

eli5/keras/explain_prediction.py

+        An input image as a tensor to ``estimator``, 
+        from which prediction will be done and explained.
+
+        For example a ``numpy.ndarray``.


are there other supported data types, why "for example"?

I will explicitly mention that numpy arrays are required.

I think it is possible to use other input types (https://github.com/keras-team/keras/blob/ed07472bc5fc985982db355135d37059a1f887a9/keras/engine/training.py#L1315), i.e. tensorflow tensor. However, I haven't tested with other types and I think I have some numpy dependencies in my code. Adding more input types could be a separate GitHub issue?

9d2d22a

…etail

Co-Authored-By: Mikhail Korobov <[email protected]>

…nto keras-gradcam-img

teabolt · 2019-07-12T16:14:05Z

Currently the parameter resampling_filter (previously called interpolation) of eli5.format_as_image() takes an integer from https://pillow.readthedocs.io/en/stable/handbook/concepts.html#filters, i.e. the user passes something like resampling_filter=PIL.Image.BOX. It could be clearer to take the filter as a string, i.e. let the user say resampling_filter="BOX"?

lopuhin · 2019-07-29T08:28:54Z

resampling_filter=PIL.Image.BOX. It could be clearer to take the filter as a string, i.e. let the user say resampling_filter="BOX"?

I think both options are fine, to me a constant looks a bit better than a string, and I think it's fine to use a PIL constant here.

lopuhin

@teabolt sorry for a long review - there is just one minor thing which is preventing the merge now, updating the some docs after some attributes were moved to TargetExplanation:

eli5/keras/explain_prediction.py

docs/source/libraries/keras.rst

eli5/formatters/image.py

… separately

lopuhin

Perfect, thanks @teabolt 👍

lopuhin · 2019-08-05T15:51:59Z

Thanks for a great new feature @teabolt , and thanks for review @kmike , merging 🎉

kmike · 2019-08-05T17:36:26Z

docs/source/overview.rst

@@ -45,6 +45,9 @@ following machine learning frameworks and packages:
 * :ref:`library-sklearn-crfsuite`. ELI5 allows to check weights of
  sklearn_crfsuite.CRF models.

+* :ref:`library-keras` - explain predictions of image classifiers
+  via Grad-CAM visualizations.


Sorry for a late comment: could you please copy overview.rst changes to README file in the repo root?

Will do that in #329. Thanks

kmike · 2019-08-05T17:37:25Z

Thanks @teabolt and @lopuhin, great work!

teabolt added 30 commits June 1, 2019 14:16

Add keras module. Register implementation of explain_prediction

e27d0d2

Show input image through formatters.image

b19a631

Keras changes (not working - cycle error)

88d8197

Working wrapper around jacobgil's implementation

6a199c5

Add .image and .heatmap attributes to Explanation

2f06827

Imports and comments

2154aad

Move parts to image.py, use matplotlib and Pillow instead of cv2

7cf948f

Set transparency to 0 for low heatmap pixel values in image formatter

0fb5285

Remove VGG16 dependencies (use layers argument, resize based on input…

fbac7cb

… dimensions)

Fix image being darkened from overlaying heapmap

17626c8

Get target layer if None supplied (fixed offset from the back)

b057636

Make blended heatmap transitions smoother

eed04ef

Bug fix: Do not get the index of a layer automatically if layer name …

a499f4d

…is supplied

Add ability to pass target prediction to do Grad-CAM on

e273806

Refactor code to get target layer into a separate function

d38dd90

Get number of classes from model output shape

5c21862

(Provisional) auto option to preprocess_image tries to use keras.appl…

d1bcbbf

…ications preprocess_input. Fix using callable to find target layer

Bring wrapped jacobgil's implementation to top level

be9e5c8

Replace tensorflow operations with keras.backend operations

5a12649

Grad-CAM implementation: rename variables, move functions

0c4a917

Refactor automatically getting preprocess_input for keras.applications

e6089b0

Updated notebook, whitespace / comments

17bdf72

Typo

4cff1e3

Format explain_prediction_keras import

669222f

Move preprocessing and keras utility methods outside keras.py

322256b

Refactor image formatter into functions

7ab2768

Move utils from notebook to keras_utils.py, integration test for expl…

b8ff690

…ain_prediction via approximate attention over area

Added files

f6abdfe

Take colormap, alpha, interpolation as arguments to format_as_image

20aa52a

Refactor explain_prediction API (remove prediction_decoder, let layer…

b9af9ff

… be a Layer instance)

Update tests/utils_image.py (use count_nonzero instead of sum)

206f31a

Co-Authored-By: Mikhail Korobov <[email protected]>

kmike reviewed Jul 10, 2019

View reviewed changes

tests/test_keras_integration.py Show resolved Hide resolved

kmike reviewed Jul 10, 2019

View reviewed changes

tests/test_keras_integration.py Show resolved Hide resolved

kmike reviewed Jul 10, 2019

View reviewed changes

eli5/keras/gradcam.py Outdated Show resolved Hide resolved

kmike reviewed Jul 10, 2019

View reviewed changes

teabolt and others added 9 commits July 11, 2019 12:16

(keras lib docs) Remove mention of TargetExplanation implementation d…

359aed8

…etail

(keras no image deps test) Use addfinalizer for tear down

50f7000

Move check that feature_weights is not None to caller

913d415

(mypy) Replace Union[None,...] with Optional[...] for function params

e8a34f1

Document behaviour of automatic search for a suitable layer

bcaf7ca

Mention that a numpy array must be passed to doc arg

9d2d22a

Typo (differentiate -> differentiable)

1c5cd68

Co-Authored-By: Mikhail Korobov <[email protected]>

Merge branch 'keras-gradcam-img' of https://github.com/teabolt/eli5 i…

ff5eb75

…nto keras-gradcam-img

Rename argument: interpolation -> resampling_filter

dc0dc18

teabolt mentioned this pull request Jul 17, 2019

Explain text predictions of Keras classifiers #325

Open

teabolt mentioned this pull request Jul 29, 2019

[WIP] Explain PyTorch neural nets with Grad-CAM #327

Open

lopuhin reviewed Aug 5, 2019

View reviewed changes

eli5/keras/explain_prediction.py Outdated Show resolved Hide resolved

docs/source/libraries/keras.rst Show resolved Hide resolved

eli5/formatters/image.py Show resolved Hide resolved

eli5/formatters/image.py Outdated Show resolved Hide resolved

teabolt added 2 commits August 5, 2019 16:09

(keras docs) document attributes on Explanation and TargetExplanation…

87cafea

… separately

(keras image) Assert that we have 1 target in explanation only

0a4db99

lopuhin approved these changes Aug 5, 2019

View reviewed changes

lopuhin merged commit 2497ec3 into TeamHG-Memex:master Aug 5, 2019

teabolt mentioned this pull request Aug 5, 2019

(Keras images) Add an optional image argument, and other improvements #329

Merged

kmike reviewed Aug 5, 2019

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Explain predictions of Keras image classifiers (Grad-CAM) #315

Explain predictions of Keras image classifiers (Grad-CAM) #315

teabolt commented Jun 10, 2019 •

edited

Loading

kmike Jul 10, 2019

teabolt Jul 11, 2019

kmike Jul 10, 2019

teabolt Jul 11, 2019

kmike Jul 10, 2019

teabolt Jul 11, 2019

kmike Jul 10, 2019

teabolt Jul 11, 2019

teabolt commented Jul 12, 2019

lopuhin commented Jul 29, 2019

lopuhin left a comment

lopuhin left a comment

lopuhin commented Aug 5, 2019

kmike Aug 5, 2019

teabolt Aug 6, 2019

kmike commented Aug 5, 2019



		def _get_target_prediction(targets, estimator):
		# type: (Union[None, list], Model) -> K.variable

Explain predictions of Keras image classifiers (Grad-CAM) #315

Explain predictions of Keras image classifiers (Grad-CAM) #315

Conversation

teabolt commented Jun 10, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

teabolt commented Jul 12, 2019

lopuhin commented Jul 29, 2019

lopuhin left a comment

Choose a reason for hiding this comment

lopuhin left a comment

Choose a reason for hiding this comment

lopuhin commented Aug 5, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kmike commented Aug 5, 2019

teabolt commented Jun 10, 2019 •

edited

Loading