Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prediction probabilities not matching with the genre predicted on LIME view #29

Open
aysenurbilgin opened this issue Mar 28, 2019 · 2 comments
Labels
bug Something isn't working

Comments

@aysenurbilgin
Copy link
Member

See particular example on demo environment with UGS test ACE:
Explanation for Article 60 using BGS XGB Frog+TFIDF
BGS XGB Frog+TFIDF predicts Verslag. Actual genre is Nieuwsbericht.
Under prediction probabilities, essay is 0.47.

@aysenurbilgin aysenurbilgin added the bug Something isn't working label Mar 28, 2019
@Tommos0
Copy link
Member

Tommos0 commented Mar 29, 2019

Checked in notebook:

genre_labels[skp.predict([article.raw_text])[0]]
'Verslag'
-
sorted(zip(genre_labels, skp.predict_proba([article.raw_text])[0]), key=lambda x: -x[1])
[('Verslag', 0.73057395),
 ('Afbeelding', 0.046490025),
 ('Mededeling', 0.043383565),
 ('Brief', 0.04182354),
 ('Portret', 0.026247267),
 ('Nieuwsbericht', 0.023750637),
 ('Overzicht', 0.020018548),
 ('Fictie', 0.018087009),
 ('Opiniestuk', 0.009474231),
 ('Interview', 0.008421486),
 ('Recensie', 0.007908742),
 ('Essay', 0.0057582003),
 ('Column', 0.005404688),
 ('Reportage/feature', 0.0050709583),
 ('Achtergrond', 0.004480537),
 ('Service', 0.0031066334)]

@Tommos0
Copy link
Member

Tommos0 commented Mar 29, 2019

https://gist.github.com/Tommos0/47a65b627bfe2bcf95228e0c0ef538a9#file-issue29-ipynb
(final plot doesn't show, but it's the same as on the platform).

What's happening is that LIME's prediction probability doesn't align with a simple predict_proba call (at all).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants