Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explain predictions of Keras image classifiers (Grad-CAM) #315

Merged
merged 165 commits into from
Aug 5, 2019
Merged

Explain predictions of Keras image classifiers (Grad-CAM) #315

Show file tree
Hide file tree
Changes from 163 commits
Commits
Show all changes
165 commits
Select commit Hold shift + click to select a range
e27d0d2
Add keras module. Register implementation of explain_prediction
teabolt May 27, 2019
b19a631
Show input image through formatters.image
teabolt May 27, 2019
88d8197
Keras changes (not working - cycle error)
teabolt May 29, 2019
6a199c5
Working wrapper around jacobgil's implementation
teabolt May 29, 2019
2f06827
Add .image and .heatmap attributes to Explanation
teabolt May 29, 2019
2154aad
Imports and comments
teabolt May 29, 2019
7cf948f
Move parts to image.py, use matplotlib and Pillow instead of cv2
teabolt May 29, 2019
0fb5285
Set transparency to 0 for low heatmap pixel values in image formatter
teabolt May 30, 2019
fbac7cb
Remove VGG16 dependencies (use layers argument, resize based on input…
teabolt May 30, 2019
17626c8
Fix image being darkened from overlaying heapmap
teabolt May 30, 2019
b057636
Get target layer if None supplied (fixed offset from the back)
teabolt May 30, 2019
eed04ef
Make blended heatmap transitions smoother
teabolt May 31, 2019
a499f4d
Bug fix: Do not get the index of a layer automatically if layer name …
teabolt May 31, 2019
e273806
Add ability to pass target prediction to do Grad-CAM on
teabolt May 31, 2019
d38dd90
Refactor code to get target layer into a separate function
teabolt May 31, 2019
5c21862
Get number of classes from model output shape
teabolt May 31, 2019
d1bcbbf
(Provisional) auto option to preprocess_image tries to use keras.appl…
teabolt May 31, 2019
be9e5c8
Bring wrapped jacobgil's implementation to top level
teabolt May 31, 2019
5a12649
Replace tensorflow operations with keras.backend operations
teabolt May 31, 2019
0c4a917
Grad-CAM implementation: rename variables, move functions
teabolt May 31, 2019
e6089b0
Refactor automatically getting preprocess_input for keras.applications
teabolt May 31, 2019
17bdf72
Updated notebook, whitespace / comments
teabolt May 31, 2019
4cff1e3
Typo
teabolt Jun 1, 2019
669222f
Format explain_prediction_keras import
teabolt Jun 1, 2019
322256b
Move preprocessing and keras utility methods outside keras.py
teabolt Jun 3, 2019
7ab2768
Refactor image formatter into functions
teabolt Jun 3, 2019
b8ff690
Move utils from notebook to keras_utils.py, integration test for expl…
teabolt Jun 4, 2019
f6abdfe
Added files
teabolt Jun 4, 2019
20aa52a
Take colormap, alpha, interpolation as arguments to format_as_image
teabolt Jun 4, 2019
b9af9ff
Refactor explain_prediction API (remove prediction_decoder, let layer…
teabolt Jun 5, 2019
ae90f4b
Add test input for focusing on the cat prediction in the cat_dog example
teabolt Jun 5, 2019
f03d666
Add image formatter testing to keras integration test
teabolt Jun 6, 2019
97573c2
Remove loss definition from grad_cam function, use K.l2_normalize, mo…
teabolt Jun 6, 2019
b1f94ea
Automatically get activation layer if None is passed. Move functions.
teabolt Jun 7, 2019
d398a00
Basic docstrings
teabolt Jun 7, 2019
f2585ea
Unit test for layer argument of explain_prediction
teabolt Jun 7, 2019
d080582
Add an image display method to show_prediction
teabolt Jun 7, 2019
33af812
Comments. tox.ini with new dependencies
teabolt Jun 7, 2019
0a3f7c4
Add pytest skiporimport. Skip keras and image tests in nodeps
teabolt Jun 8, 2019
d5a33f6
Replace PIL.Image with from PIL import Image (fix AttributeError)
teabolt Jun 8, 2019
5beb983
Add new Explanation attributes to format_as_dict test case (fix regre…
teabolt Jun 8, 2019
386dd08
Set kwargs defaults in show_weights AND show_prediction (fix regression)
teabolt Jun 8, 2019
831ae81
Update tox and _ci for keras and formatters/image tests
teabolt Jun 8, 2019
fdfa8d1
Add __future__.absolute_import (fix py27 import); Move ipython.py dep…
teabolt Jun 9, 2019
754a162
Comments. Variable names. Remove unneeded files
teabolt Jun 10, 2019
b625ef6
Update docs. Refactor automatic layer getter function
teabolt Jun 10, 2019
a5da085
Check for TypeError when passing incorrect layer argument (fix last c…
teabolt Jun 11, 2019
fa39f57
(mypy) Ignore third party imports
teabolt Jun 11, 2019
a236c01
Add keras docs to supported libraries
teabolt Jun 11, 2019
bcde9ca
Print prediction ID when getting top prediction automatically
teabolt Jun 11, 2019
ce06e7a
Add keras image classifier tutorial
teabolt Jun 11, 2019
2db2f01
Add keras deps to mocks (docs conf.py)
teabolt Jun 11, 2019
9cf8f9f
Update eli5/keras.py (docstring english)
teabolt Jun 12, 2019
15a7add
Remove .fuse_hidden files
teabolt Jun 12, 2019
c8d7981
Update notebook tutorial: fix images and shorten model.summary()
teabolt Jun 12, 2019
7614217
(ipython.py) Return formatted image instead of matplotlib objects
teabolt Jun 12, 2019
ccee1f5
Fix docs formatting, English. Strip TF warnings from notebook
teabolt Jun 12, 2019
2cf1636
Rephrase DESCRIPTION_KERAS
teabolt Jun 12, 2019
9744451
Update method string: Vanilla Grad-CAM -> Grad-CAM
teabolt Jun 12, 2019
2821b25
Rewrite condition on index
teabolt Jun 12, 2019
f829551
Replace FIXME with docstring comment (implementation assumptions)
teabolt Jun 12, 2019
afa5f53
Make it obvious that update_alpha is in-place
teabolt Jun 12, 2019
8c31d5e
(docstrings) replace single grave with double grave for variables
teabolt Jun 12, 2019
f187301
Mention testing incorrect prediction ID
teabolt Jun 12, 2019
9151d4c
Add eps to lmap normalization
teabolt Jun 12, 2019
00ae853
Explicitly require list of target predictions to be len 1
teabolt Jun 12, 2019
84fcba6
Try to make travis happy by copying static images in /tutorials to /_…
teabolt Jun 12, 2019
6eb8ce3
Change alpha_limit range to be between 0.0 and 1.0 (format_as_image)
teabolt Jun 13, 2019
c1d3f7e
Convert any PIL.Image to RGBA. Raise TypeError
teabolt Jun 13, 2019
c357cbd
Replace matplotlib with returned PIL image (ipython.py)
teabolt Jun 13, 2019
20d3635
Keep .heatmap as a numpy array. Clean up format_as_image
teabolt Jun 13, 2019
2ce3ac3
Validate input doc shape
teabolt Jun 13, 2019
b8550e6
Comment
teabolt Jun 13, 2019
2b63b85
Comment on doc
teabolt Jun 13, 2019
38fa865
Replace starred expression with manual indexing (py27 build fix)
teabolt Jun 13, 2019
35e76dc
mypy annotations
teabolt Jun 14, 2019
f24b0ed
Add subpackage imports to mocked doc modules (docs build failing)
teabolt Jun 14, 2019
41b7aa6
mypy for function signatures
teabolt Jun 14, 2019
382f572
Clean up submodule imports/mocks. Add comment to conf.py
teabolt Jun 14, 2019
a959b2d
Update docstrings
teabolt Jun 14, 2019
0b1493e
Update tutorial
teabolt Jun 14, 2019
9d121b2
Tutorial comments. Delete duplicate image files
teabolt Jun 15, 2019
18633d1
Add comment about image paths for tutorial rst
teabolt Jun 15, 2019
103d9f0
Do prediction once. More keras backend ops
teabolt Jun 17, 2019
2640623
Mention matplotlib and Pillow optional deps in docstrings
teabolt Jun 17, 2019
578b0bd
Test ipython image as part of keras integration tests
teabolt Jun 17, 2019
469b0df
Tests for image formatter heatmap conversion functions
teabolt Jun 17, 2019
df29072
Tests for image formatter and keras functions (coverage)
teabolt Jun 17, 2019
3ee35d6
Move image display dependencies to separate module from ipython.py. A…
teabolt Jun 17, 2019
39ba9a6
(Fix previous commit) mypy, remove test for ipython_image.py
teabolt Jun 17, 2019
960720e
Link to latest matplotlib docs
teabolt Jun 19, 2019
8b5d872
Commands for converting notebook
teabolt Jun 19, 2019
a79146a
Remove comment. Command to script
teabolt Jun 19, 2019
c58cb98
Update type annotations for .image and .heatmap
teabolt Jun 19, 2019
88417fe
Override function signature doc
teabolt Jun 19, 2019
4724ae9
(width, height) order for Pillow
teabolt Jun 19, 2019
80ff9ec
Clean up list expansion
teabolt Jun 19, 2019
a6fe6db
Print model summary if test fails
teabolt Jun 19, 2019
efe7e4f
Remove FIXME comment
teabolt Jun 19, 2019
8fbd446
Comments. __future__.print_function to test
teabolt Jun 21, 2019
143766e
(fix build) Don't install hypothesis 4.24.3
teabolt Jun 21, 2019
998d135
Replace print with warn. Move ipython_image.py to ipython.py
teabolt Jun 21, 2019
f2260b2
Remove TODO (print summary if fail)
teabolt Jun 21, 2019
0f9e98d
Comments about choice of model
teabolt Jun 21, 2019
4d4556e
Mention about cat dog image
teabolt Jun 21, 2019
d2ed75b
Show which image deps are missing (ipython)
teabolt Jun 24, 2019
34e3cee
Update tutorial (loading images, top preds)
teabolt Jun 24, 2019
5f03ab2
Update docs/source/libraries/keras.rst
teabolt Jun 24, 2019
246d780
Merge branch 'keras-gradcam-img' of https://github.com/teabolt/eli5 i…
teabolt Jun 24, 2019
d930a12
mypy type ignore (fix CI)
teabolt Jun 25, 2019
64a31a1
More about input parameter in docstring and library description
teabolt Jun 25, 2019
3d3bada
Add comment about grads list
teabolt Jun 25, 2019
684465b
Rewrite grads to raise error if not singleton
teabolt Jun 26, 2019
35b7829
Hide some functions to simplify API.
teabolt Jun 26, 2019
a923864
Remove eli5.keras.image_from_path() [tutorial has equivalent code]
teabolt Jun 26, 2019
1b16314
Keras package, modules explain_prediction and gradcam
teabolt Jun 26, 2019
ff18072
Rename function (grad_cam -> gradcam)
teabolt Jun 26, 2019
8e95d1d
Fix docs formatting. Update library description
teabolt Jun 26, 2019
ad1bbf2
Have .image as RGBA. Remove converter function
teabolt Jun 26, 2019
9334d20
Separate out gradcam and gradcam_backend
teabolt Jun 26, 2019
2885807
Remove some TODO's. TypeError for _validate_doc
teabolt Jun 26, 2019
0f65f92
Update _ci with keras package
teabolt Jun 27, 2019
c37a8b8
Refactor get_activation_layer. Add validation for retrieved layer
teabolt Jun 27, 2019
14958eb
Add validation to targets. Tests
teabolt Jun 27, 2019
64a9923
Resolve/move TODO's. Add gradcam test
teabolt Jun 27, 2019
00ab5c2
Remove image formatter TODO's
teabolt Jun 27, 2019
fc61592
TODO about png vs jpg
teabolt Jun 27, 2019
dfecd8b
Comment TODO regarding checking activations over area
teabolt Jun 27, 2019
abcb115
Tests. New method heatmap_to_image replaces others
teabolt Jun 27, 2019
4f06b85
New test images
teabolt Jun 27, 2019
80bd2f2
Docs
teabolt Jun 27, 2019
da4513c
Add TODO: testing ipython missing image deps
teabolt Jun 27, 2019
7bddb5a
Add test for missing image deps in ipython
teabolt Jun 27, 2019
5d80594
Don't assign to result of preprocess_input (is in-place)
teabolt Jun 28, 2019
60fd881
Move .heatmap to TargetExplanation. Add score and proba. Update tutorial
teabolt Jun 28, 2019
e4c8211
Update dict formatter test (Explanation attrs)
teabolt Jun 28, 2019
587bef6
Test outputs_proba
teabolt Jun 28, 2019
eb93d51
Mypy for outputs_proba. Blank line
teabolt Jun 30, 2019
a5cab1a
Add sigmoid to proba activations. Docs
teabolt Jul 1, 2019
1a88472
Try fix docs indentation (CI fail)
teabolt Jul 1, 2019
1a22dc1
Indentation in keras lib docs (fix CI)
teabolt Jul 1, 2019
730487f
Merge branch 'master' of https://github.com/teabolt/eli5 into keras-g…
teabolt Jul 1, 2019
017bf3e
Make feature_weights optional.
teabolt Jul 2, 2019
e26997d
Remove empty_feature_weights. Update notebook
teabolt Jul 3, 2019
7e81e52
Validate heatmap and image in formatters.image
teabolt Jul 3, 2019
4837808
Update ipython docstring about dispatch to image display
teabolt Jul 3, 2019
2296c2c
Fix docstring param order
teabolt Jul 3, 2019
e57c8d6
Error handling for failed gradients. grads function
teabolt Jul 4, 2019
7cadb65
Merge branch 'master' of https://github.com/teabolt/eli5 into keras-g…
teabolt Jul 5, 2019
a8237bb
Remove outputs_proba, assign value of predicted class to .score
teabolt Jul 8, 2019
5f05744
Use tuple unpacking to get image from batch
teabolt Jul 8, 2019
20cb6ec
(lib docs) Mention heatmap is part of TargetExplanation
teabolt Jul 8, 2019
d7aae22
Update lib docs, docstring (heatmap and explanation details)
teabolt Jul 10, 2019
206f31a
Update tests/utils_image.py (use count_nonzero instead of sum)
teabolt Jul 10, 2019
359aed8
(keras lib docs) Remove mention of TargetExplanation implementation d…
teabolt Jul 11, 2019
50f7000
(keras no image deps test) Use addfinalizer for tear down
teabolt Jul 11, 2019
913d415
Move check that feature_weights is not None to caller
teabolt Jul 11, 2019
e8a34f1
(mypy) Replace Union[None,...] with Optional[...] for function params
teabolt Jul 11, 2019
bcaf7ca
Document behaviour of automatic search for a suitable layer
teabolt Jul 11, 2019
9d2d22a
Mention that a numpy array must be passed to doc arg
teabolt Jul 11, 2019
1c5cd68
Typo (`differentiate` -> `differentiable`)
teabolt Jul 11, 2019
ff5eb75
Merge branch 'keras-gradcam-img' of https://github.com/teabolt/eli5 i…
teabolt Jul 11, 2019
dc0dc18
Rename argument: interpolation -> resampling_filter
teabolt Jul 12, 2019
87cafea
(keras docs) document attributes on Explanation and TargetExplanation…
teabolt Aug 5, 2019
0a4db99
(keras image) Assert that we have 1 target in explanation only
teabolt Aug 5, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions _ci/runtests_default.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,7 @@ py.test --doctest-modules \
--ignore eli5/xgboost.py \
--ignore eli5/lightgbm.py \
--ignore eli5/catboost.py \
--ignore eli5/keras \
--ignore eli5/formatters/image.py \
--ignore tests/utils_image.py \
--cov=eli5 --cov-report=html --cov-report=term "$@"
3 changes: 3 additions & 0 deletions _ci/runtests_nodeps.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,8 @@ py.test --doctest-modules \
--ignore eli5/xgboost.py \
--ignore eli5/lightgbm.py \
--ignore eli5/catboost.py \
--ignore eli5/keras \
--ignore eli5/formatters/as_dataframe.py \
--ignore eli5/formatters/image.py \
--ignore tests/utils_image.py \
--cov=eli5 --cov-report=html --cov-report=term "$@"
Loading