From 6512d2411cbd9a3f2f860813042b2b3a67995c67 Mon Sep 17 00:00:00 2001 From: Frederic Boisnard Date: Thu, 3 Aug 2023 03:01:43 +0200 Subject: [PATCH] docs: introduce Craft documentation Signed-off-by: Frederic Boisnard --- README.md | 1 + docs/api/concepts/craft.md | 50 +++++++++++++++++++++++++++++++++++++ docs/assets/craft.jpeg | Bin 0 -> 862851 bytes docs/index.md | 13 ++++++++++ docs/tutorials.md | 5 +++- mkdocs.yml | 1 + 6 files changed, 69 insertions(+), 1 deletion(-) create mode 100644 docs/api/concepts/craft.md create mode 100644 docs/assets/craft.jpeg diff --git a/README.md b/README.md index 1f9bb6e6..e5be2895 100644 --- a/README.md +++ b/README.md @@ -237,6 +237,7 @@ All the attributions method presented below handle both **Classification** and * | Testing CAV (TCAV) | TF | [Paper](https://arxiv.org/pdf/1711.11279.pdf) | | (WIP) Robust TCAV | | | (WIP) Automatic Concept Extraction (ACE) | +| Concept Recursive Activation FacTorization for Explainability (Craft) | TF | [Paper](https://arxiv.org/pdf/2211.10154.pdf) | | **Feature Visualization** [(Paper)](https://distill.pub/2017/feature-visualization/) | Type of Model | Details | | :----------------------------------------------------------------------------------- | :------------ | :----------------------------------------------------------------------------------------------------------------- | diff --git a/docs/api/concepts/craft.md b/docs/api/concepts/craft.md new file mode 100644 index 00000000..4e628d0e --- /dev/null +++ b/docs/api/concepts/craft.md @@ -0,0 +1,50 @@ +# CRAFT + +CRAFT or Concept Recursive Activation FacTorization for Explainability is a method for automatically extracting human-interpretable concepts from deep networks. + +This concept activations factorization method aims to explain a pre-trained model's decisions both on a per-class and per-image basis by highlighting both "what" the model saw and “where” it saw it. + +It is made up from 3 ingredients: +1. a method to recursively decompose concepts into sub-concepts +2. a method to better estimate the importance of extracted concepts +3. a method to use any attribution method to create concept attribution maps, using implicit differentiation + +CRAFT requires splitting the model in two parts: $(g, h)$ such that $f(x) = (g \cdot h)(x)$. To put it simply, $g$ is the function that maps our input to the latent space (the penultimate layer of our model), and $h$ is the function that maps this penultimate layer to the output. It is important to note that if the model contains a global average pooling layer, it is strongly recommended to provide CRAFT with the layer before the global average pooling. + +!!!warning + Please keep in mind that the activations must be positives (after relu or any positive activation function) + + +## Example + +```python +from xplique.concepts import CraftTf as Craft +from xplique.plots.craft_visualizations import CraftPlot + +# cut the model in two parts (as explained in the paper) +# first part is g(.) our 'input_to_latent' model, second part is h(.) our 'latent_to_logit' model +g = tf.keras.Model(model.input, model.layers[-3].output) +h = tf.keras.Model(model.layers[-2].input, model.layers[-1].output) + +# Create a Craft concept extractor from these 2 models +craft = Craft(input_to_latent = g, + latent_to_logit = h, + number_of_concepts = 10, + patch_size = 80, + batch_size = 64) + +# Use Craft to get the crops (crops), the embedding of the crops (crops_u), and the concept bank (w) +# 330 is the rabbit class id in imagenet +crops, crops_u, w = craft.fit(images_preprocessed, class_id=rabbit_class) + +# Compute Sobol indices to understand which concept matters +importances = craft.estimate_importance(nb_most_important_concepts=5) + +# Display those concepts by showing the 10 best crops for each concept +craft.plot_concepts_crops(nb_crops=10) + +``` + +{{xplique.concepts.craft.Craft}} + +[^1]: [CRAFT: Concept Recursive Activation FacTorization for Explainability (2023).](https://arxiv.org/pdf/2211.10154.pdf) diff --git a/docs/assets/craft.jpeg b/docs/assets/craft.jpeg new file mode 100644 index 0000000000000000000000000000000000000000..743144a08f13b74e0cc1e468135e3838a20c6ec0 GIT binary patch literal 862851 zcmeEv2UL{HlJF2Dh=?d48I>pqC_#e2AV?Mv5QaP|An5`ElEYvCBnt>AX^@;pa%Lpw zoFs=K=OGR-%>1AC-nYB&?Z0Pt|Nred@13=e-%Q(6-CtLAbyanBb%Xzbp95V{Ra8*~ z5fFd~R0&i;ApAVYPr=>#B?zRh4&nuYKx80d0!GjUAV&Z^G6YQjl2;_)0TKOPP6z@8 zT7!uHgwX)Le`bN!e%AT(n<$0gPiUYlh43%XS8P&<{*oto^s^p*3nc!;%Hg$xvz5cE z8^Qv2LE>^M>cl@=2h#6(&fl|FhFL<#BS6K(YkuUPH!*U)`0pTUGLlDEY={VMfe5Jy zh^Pth%^)_Q3l|7}Prp|KJ_rbjh%ba# zr2HcVMI~hwRh_50dJugBLo;)WmzGx6HqNhIT;1F~Jm0c*z#me#iRj?TfM;gQj?ALA4A3#i4V<(1X7b@bl;!Qs&{ z=H&DzUI0D+KnwW%1GB%uiyFX-keHZ=nDi%J1cYus5vL};aPuC?)rVT7&mFJb5_&^M zBNzF#sDYeaSbLZDh10-AIu4O}PV`UIe#h*;hS=NxC(Ql@v46#D5+n~I{5=s75)xe? zA|kp#asfyrWF$Wm*+sJ7)5Sm2rQg%#pNZ;U5*}!T0BD1ln3xp!cZGtS;>v$J;imz{ zC&f>IE)fv`%tS;D0)x)6slhy;U(YWZ{NjOMJn%O>K!orApOz1g-^fxZh`Df~WT{fW zM5{#0Ue(eXkiBZI75TsR>vsZZJ?=>3@?!%?BQOt8h@poDq^0mr^r%#7+ozCfJ)W}a zNFg22S_H=Qf91gbDI7TKaMG@pSftq z6mmC4F~$%Vrn*%^6X2KB=^e!#Pf(m=nqzi_*FepTALma{{+R>&<3C~D>V5(LH>vRl zcwvl||2&lP# zaC~X%od8f_PGce?9+1G{6u3Pbd;Vi4cf%@}v;P8x4L;98zfk-Ighx z9v<|u_Rl7i8}1pQwz0MWCAe_ycp~Z(>izC)r_b;fo@xsD@$b+iU1Bc zb*E5xwQ$YrN_9#B6av(zPWSI++0(y@{tJ|dbIP+4^%V{SlJKnlf!I_vMJy031+`9U>dL>>CqB?LKN~; zA@{zgf3hE^pw<`O8kHxdp9riv9t-KR^DA1UP%Nuj$Ch-L_DTAQ)XsWJO>_%udkhm~ zEf0Fcd|A<2c|rmQxYQuoh1CUQee&s7;r~BD64aJQp-D=KJVcP5w4UR9q-0!g3MKUZ zCpCi?Hm~CSL?3-Zb6N!1=*IeqG{@ZV>@nbViSiR6dipi@ZJkDe>~(@1f*xH;?RTO88ZFri|47f>#KGu^1lbEB$&H^9pPDcD5jsBR9{m^} z6z@kqJ1(7#)qaxCQR!3$dHj!y zpOAd;{3L3bH@rSf@LE)v@~K`deKzH;@O8_$=H|QiKgOrDdmr{F%((d{hA9ul48E71k5a3bS5&3XbDBGK5W>E z$4;DQ$cWj%X{GGEo@dPz?Uu*z$(voWr0ywMvJGgP6}qB&?N|o{a=W%~;n;VrAQdL& zs1zP!{lY@&PR4cm@~R9sFBcV2i%E#j0oa=9s0;)0bkDS@7P;9n_;+?ZNLUl6 zJc9?JgR#$3(>AnaY;o+Pj(Ctvj?W+-wAeTP?o1y`EG?yl2Sq`U*cWNwGT3(=pqdg+ z#q{@Twy6;X%HRcQOSb*>xM@M$HruT`0fat5WH%M2X*lE;qF1hUuzxa zkYbhcQ8*v%vkagZUi3FGN?1|L?-75zb{}{x1eHYNWAhLXqTUe4gWyl=*I;{=n^3GM z^xr}Oh*3iReDfc@Tl;styZ$#YG_WRJEEyj36%G&+0miiWN@^*Mrn8}0OCnAV)wbjC zATHk)Jm_zr*mg%?e}&Vhe~%OFuVDb3{;IbSRK_oy{^X)xZ3fTsOxm^1N{c8lxq*!cq zCV{0l`do(8p_|H4M4Gj1`tLqKnJbeSPyc0I%u>GL}@VpWq zMOp?Vk7uMeGv_Zg>_i`Eo~)g+?WV=}5A^drT^|XEPl96)&Z#iE8N+rDkT^+LV>sl8 zfbF}P4A!!!>ZH#b+&|0U=LOGd865E-0*BqS$RmpQ@5`~paRx&Xgbr92WMi5gOj)ze zF?<%thb#Rs?y9O$aBgd1yh>=e3IdPAb~9m(CiAl=ShnLPJjkCpYLj`RYEVl*u-TJm z!x;|>ufRN!QcAn`y%G;A|Jee>G6q05Q z-mAjh_ZBGcgBP||G9gFQhjvxJTAayx-xzi~4{3JnT0*(rOf-{d({Iaoe#eBTM%CJT zK+m#hz|=sn73sg55@q5-tI4pJxq}CRvf-$>#_T!UxG@xUHV2&7{vB?7=yXJI8R{-F zut5Wf0Z7hNPq!XfTF!Xje5-u$k9%!4yE<2`y6}suoSq z6v^H7&IV81HWYWOI(@A43&R4)(7n0Ha<#**Vx5j%^vz-!c zi)CM~H}?_sMDRV;>u5WzlEi~%MH0$G-5yl)UGUcYTJAbc6akQ!kHP@+We!GVNy6C znCu_+o$=#ni^t4l%N3Pl-xtCqeRyniMbY)Uw6C|W!c ztHZl(ti;pa>fCvGBv;B|{)Jo!IvNKsG+Ja~=!Ydl!8vE4kE-KXprP#yXk68S)m?R3 zUOk)lt>ZDr1|H-+A~L#AEqTfp6WEHoo}`;ve~Brp9S_2)bJUCPmQofQ?G}Wdx8Xqx zR;u&Pd8t(*=stgr-mXKyK!1)0VdC3IWcaW$11Yvinh@n%dSv=-w@)JMuk?>pP$UN` zc=SZHAOk&PYTfh<8`Ws4CK_)mzIx@Em7NDcMY#XZryJ?iD`YhSGCC4J}krQPO7>OD;0em*vts9P!28Y%LAeC!uA73z80=?V0eta8Puv@jS|LXqVS-wF%vU1_UQz9YC*R|pCd?PQJO)Uv0`AaSQJS|5 zf^t=DOl(LjNkviQ(189cOx&Z&zGx>V2WU(Uj#gd=jr1?Ig_YSr8p$e%r?mRIo!>OV z4ubZV=4L2ASiwHQ>iRmg87sj7#=)I?qmJv(`-oo>otnhif`7eA2y= zBNUv;QG%4pA-7E577~z}?NMP#3Y8Y5=`5&N+Mi9!=mA9Y&iC%Bs!(_-P?Cw9P5|%H z0cG&5(SpW$7!i)fAbhoGG`QfX3$_C&z1syI4iR;LT1)MAyy4?>zJLZ8hEMRIFYTv`lw!bcjhaT znx!YF!UGjDRH9||+3;5r7CuoA%@CXjR&ZBzZOB`5fyqm)OuOgePP_uqS;y5XZ7)Is zX@BqYGd-*myfOX);vJUB^ZAI&#~w!^3-h=A7QWiAbTfYQYx+oA%}jmbxxM#5g`Mst zN0f#NcMexQRgie!dWE@RWi!C?S-!!ydbR1m$)6ALz6qro}7T| z!PubFdB6aBxT}N<(8NanG}1)iyTfe6uz#%_Me`q5o(3v||Eyf|vofqn6H5*BBpP_l zHXs#Se3b{3u#co1_LR0^Sio_@*#%o!#9;tJac{U`Mev{owo?Z1o(7<{LkE6uO$`?S z{i7?QC+rGr=qqSs2U`_T0dR+0iQ@;{A<<{Rgl~rz%1tfFZ@o8CT$ZHFqNLpj*BbKl^(1-%yS1Z<>QfiTr1n|7%}= z$Nb;6_MapEuO0n`_^%QF#p=K2_%Gr1ORE2ask5Jo{HICrpR2Qf)#HHH{41+~_P?aZ z|1>#}IAK+=4b@xlfe>sl2~gV%h>nXq|JHV*!o06?z+&OEY`90~pLUPip#CyERYv?w z&9OTV-s*Bv*=tNizY)rGCqCyJX&%AchQ9Xggk2Xk*a>Xn_eAwoT&Y>>U0IQ*pXz_B z$yaeF4I|5*2|8td7tVUC`0IyY_07?-ElZqb2O1I(LLL~)M=r+6V(3~|lUYz<%a*cW z^W|l}XmXZ8-huF*yrV@P1^)|t5B^k&1ia9aP_$H$A=BXia&0QX|Y3?P>Og^uo|#vKrq42 zX@-Rp7_bbd;C7t%YHBdLZ~r3?A{T5C@dkGdGmqGahlB|1*xef9Pm33};Ls|v^hv)L zD?vbiKv)l~dIMotVB0dK7l{YGp+jOr3>)947e@3KQ(ba4?xSFSNvjk&GZs_s3ckUE z0KhSBfSry!y#%;PFx0ky?Oh6MLgFa#pngjz9`ro~ic`otEse0n-AMMr76By@?HBnD z@gN+qgxPh#y#u^hIATMAPR`e$nL@zxZ6C7IVj(VG zBRq$R4k$9vV_&Z<{gj^^e)<3p`UqKL+w5^#nYYOQ&56CLQupY>KDGnV(|LaNJbrO< zx0L`7(uA;eL-a^dVEJ8%Ny#`2=SlOa30GcG&-X$fi#*4kaK15c;pS@Z-Cp%rO!wwx$mx>3U4Zu~rMKfSbC$oRLZdwr^3DgA0V zu1{7>O51Fe$MR?FOZEI{Z@qVJD|R(8Y&N<}5dQgPmCp;Wk7y6InsNSWC7i+z*DX?ckCb{Q(&MfE4Q7+ znpqz`pkJg!3SVfwhrI*K@QzrBo6VbDG!tT5fr=(-Xk|_T)|?-&P;HrnCN(Axa$Sz& zhgNFC1Lxty<6PEj!-K=qFP@srKxf#?*PF)*y+i!oIDg(hjc@G#dZq&|&(NhjQ z6B_0F=-tzq7oMc(bIzAOaLE4JZp*u_-qxX-P|ZXkX%a zBGtOs8Sm9aT-CT;>dDeA>4P0Ks?BZqoX8it>#QGN?w_TQIYWPJ@$6$ng4C91kF{as znT@dr%>7#EX+(z2di;h@vb%TW{-dS`bq|ja>Yc*s(L9BLK^_(NFDevYLl!=i=3C>V zw1wPlWtUZcI6%J1*K%i*B&ASob9^gt1&zokRj0rGa2XHkx`_pAdM$0|QK=bLs(66gBjeAYehBi&Z#zBO9-Iw5=Ok*Spr zBI!q~#=ZVih=SW?^F_<#sOv8ToE%tTXvqJ#HH98|kUj zvA;dfab~lSaGf!bn*&smXpzj4xcN z&QrathTbgh`P5U^%*E!ub08c`|G{C=T+LK+f)zKhR&wWAYczr)$QKts&lIx2I&xM;bp4JHWMgY zh>9C|_}z%NW_-QqaCNV8I83UZO+TDMsPa4WIclPouQi!V=4!cQ@T5M4yNw^WUxIF3lw!+X}SfwGT_{OSC zf~=N1)1Wa`^ds>aAPF}hw-=;&C)hb6N$z|sx$N9d9DHDB1i3#t^ex1-8%Fma!UlZ~ zYbmrJr7Dh?Q6zCY=wKiGR#z^kUA02zSzj-Pn>)gTV%2x_L}#fCF(E2OUaN~-+EGH8 z4=F>F+o}q4TcmohD0*Dk@(epBhC4uJfOl!h~lV_y1ZQPfyv9ArF5E7Bp?}q1%g`-XP{V@xF;2{KUQ;-owoWW z$lK?#1~zGH5!nz&T7a(cNa->8S(NU8Eu z!wES`^!R!s5GJy5wJ_|t>i~z`52Ba}#UJ4k+@tlPyLiwx_ou^7vqRb~Q;M-48+5Vt zvW;n>+XwQKv@NmqDgth1xn*1f$F1;e{+_IOb48BG2Ksz{R|A&$y#Opx;-ynP()XEX zijhoa_9lTFCL%z5OX6~UvG&o&QiGLxt8HH|cU* z*yPTk?re#<7VDSw9)oAEigU!9w_6`7Cg%IZ&1q7*7j7sFY&qBkKf3d5rNd(JQRU*8 zG@8qreaI!Dqh3kPNyD9pOmeJx)zhl1Uu1M&%&y$;b=xkL=LrPW7s90Zu7y##>@wXD z(=lpCu3*Ue@DS0DKH%1*=#YTv$$-h#jw_&}S-sIff2^nl4t>%XG?MG`wxI+v@fF8R zSr5;`x52e#Cl&K7=2JhqaT-%95p~H;6ZO-(+9wqyf{*3#ppuKb;!f$-9fj4tSv8*( zHNG6iuTzI8gwH(nCh#NUz$sSMNiB zDblU@oyV1JEVfHjEE@{AF{56A1Fq~O-7xkRE^BhMDZ5{953F`DQc@AC9d)T9kIJybMLPE;OoCMU2js$qH6^nW ziR1jUt;d}aRB;grg+5e+NYhYl84`QQ@ajtCRnvw0dY7IOUUoL38&n~C9%{*A8$a+{ zbBBWqlI^}@gkoX6eiif3jT)&ZpIbFyhWoH9*0D}LIa43$Llfs*Xh&@f-<119yQX8? zj9o=rY{xPVx`N8h9%u;T`g4Ox4#Xc6h&|eu`eJ#}*o;Jhlr-4>%kKW8p?G-<=EZiB z&GPLpZ4RFM&@1bi(w@^&wTnu#=V@C|^;N`5X%myH|M5nAPm@mU*h{e=H_OYzl+B-B zOmkzopkkTbM%(S0Y(q@S9GUEKSeW-@P+{v^XjjR{@m5%YhbuCJl6l&Of;-;syItjE zGO4evVwB>(HEUB=hn>i8dJ4ZWzxqC=Ttgr(x*riJWwT~?_lzD4L|DWo&u))ZWh^t) zLq|)rL;5Qr;jL@-;-ch{5iDU^Y6Wdfj+mi%pIi+|EuQp)`zt17+fSJFSlX{~wXI&Z z)iA0>Ev@b7Uo}|)3&IQs)`OO%kS4n$1+96GuhE_IcR4~)DMwk>{hTv@B~JXJ<>rg)+a(PM`*H zwjcxv!tOjp$FkWVP67{7e0Rc#=EXTJIGoT!@Stnhz6~4|;AJyq!ct@329NdMxjEu8 zEv8m^r^wW}i6Z94T?6LZ>^DN^&<%)Tz`OUH>OJ(}`WWmFUqp_D1PME1`YLDUu}=%! zQSTc6t8h-F=TUKe+hPyxX%8NB>E><-?(ON$AMQO-w%tZHGUTdYAs#fRfWU*;AqZUR zOP079cQ!PQ5MntFN9g-_0GdDhw{XCg!QVEu&h>oU2Z&CfmgmJ;!{)gEi*MBW)ir|}t3HQY3s^cc?Wf;*ruo+MJ(fI?-^Y`wLQEN= zgx?(%Am6dyKBx3h@PsuuM+Eya$;8=W!(LfHcb+1fks_X77Vw}K2pqZZ<8x<&^+t7l z3QC%K=q|`Yte+Sv3ZZxyNE4HGt)HaZ0iu=1bw3YX2C}}HchM$_ zu{6`~4kAOjKIUTlCzlk`p*B?u2`hqOEd>zGuje101~B~S-*wNPpg=uK%+L;W_b~hO zVGb`{3YqZ;MD#&?&drpI#iEYMq7In!GUO>5h+-R1H+h{|iDaz6YJ1L}y8|J+tC?RD zfAr1u4d9gKfZz^O37iw#SNN``<_U8xu%T~y1-3eIu8-)1VYsuf;ifb6nYqOabfic>F)QauK z?g^GUloqX6%7P~Vfmt6`vb&a;H)VM5rr){}vnS)ujsDUE{@KcLBwOr8A3=Wt$h?i@#JaYeHe?5(Mj?b}i7)P<|aujWr*^|Q|#N=cscX>h9$qOL` z8DPEnc5uI^;)Y~?w`>P}!>vkbleD4Iny=)0*v~)0F(~Pcw+0HlE-Yr(Kg8re-!QCC zvUO&*@)r~E)djhTT9vRe1YvC>zs2_Hg-$`g#SE0ci>hRS1+tb+DR3pdYL=aE-*{`A zD(=t6;jcltZf~@%7kh->)~R|*D9hq_n4dR^EHgeq3wyYO-nD7e>K=c)I(gx(P_OKF zV&LeAo%Jc<{pdRtbZxy)7MYWq!UJh%J5R~gjD-6Rsc6>>$+TW`u)gB(u%VhhT=X0p zNoE?cL#UFuS$0siw=sLgW1JW6$9Wqz8+~tE`gu6RZ|s{pMVp|lH$;0_niw=18Bj+_~1`SkH;( zFZ?t*9LgBi#5O72P9LR(pHRScr4AJyoa8(F##6FQfD^B}E61b923DOw=QeVY@3 zNhRIAKhjVS9I>n;b1M?1)=g1@(2edVS-0z`>(H0^CbvXg+x3}J=7d@Op&J`Y?`ofC zVKiFX&QvmcLWGf@#9o;sghu`tqvR@BP**!$_U4Du$ZWM}o%M4|R93)F;XpwqlH!{nFR29hU}&ZaMr#XU z9Gv8jY5v{_bv7h@is@mHFDMifv49 z3ol=m=9}(B^QQCD6?z=bbMjMfq3P3!bHk-`r^d7%r6lgJnHpIxSl`sUe*HPQlsVOI zp)&+swi1PWiPWB~YD7F{(?vih6>Qq=Pg%nXOq-96FA7ITEc10E_^}%?305~{dp&oZ zjeWl^Ag^fBq%svO6@}h%2F~TZth7^Vt``aV5dMk8CCgzi$Z%7;M2*&qJc2MBuCykE|>Q$WK9;D1;#qM5^0a1z&cg8hNgDZ)I zim$4+OrGAoms5jPjd3Xvhml%O>;#B>C^$@rW^Z6uJ>#^z;9L`mD)XeR2--XZwUWdw zwz^S4TdOIx`BcNWQur>F&AL~d5VbE6>Dh2^_Wd1Gd0 zkaj~X*XhEtHKbd}u465cVC?|WSD9KN`lt@9v*~n4I@QDO$Uz`V-xEYC1DI4BW8bj+ zBY}Yqq6`i*k2RTX>J9pM_4nHk3$=%oi>YJdnXk-brL@&Ktl>f9tG&8@?JPYWpH+Jk zw55ZG%{UhN%C9YOe!{Vpyq)pNaJ(ebw3BY^NNxf19paJDId}@uXy)xiko$;`dyBn=5X&JAy&8>gL%Bn@DtpNT&leaNPq>YOzKw(3S3de|jCf0+$kM!QJzgdy zWoivab`fkqd6Wvz;g>3Ix}IGIs~&>`daUyeC!of6zqI@?X^N=sS_?jMs%^C|h=6y0 z+YOOHywkoeu{Vx0b~(n{Hp=i$8$A44n<`ysKTM|!N(}51-7Nmh|H|%z`nlJ^=~xGp z&Sf$!qwm@jWuE8VyAK?W5Lvw@t2N0QbTP~M?u9wN#p~MsFDEA5!P~({$bjn`ByS|` zpC@EV!75|+eWP8g1B3YsxE(p1#V>eTF80~mZczo`=m!(zKA*>d*95WZ=goMKO+A}n zgQ^SS+8R1$*eZS2Y~6>-q^K_5LUzT9beU*u!^E(l#&1Ud#(uwBp+gfRXS&W5HHlDK z)#2%wdA|bT(wL1P3yk;NC{Aa7MSHo-e>#z_gW)l4p8{){W1(5q@}00cXXE!{T7^q* zS{>U)^AvotTgjBzWtye|Yl&5{qwUs7qqQrbQ z_UiO)4BKy?E^H-BVEN!9nTpukW zB>JuUgBYR2@gYKPK#;RP{^hW_+qDZ6rLkvKz0MJa!$VRxQ`X8`nI3@o-RN46rEqtA z_`=b(lHFS4c#wH(b2Qg^irBH^7&^60I$rw4tbV}!Q+6;f6&P3j?r9lruqc^q#9-N& za(1l1y)Dos%$IL2J~88_e)~y`Qun<#-w{@Wh%N3C|KumVK!g2ZGDd02XI(nWvmXp) z3es)pBISAfq!c~`b-vrD?gbSde@_;SFsZExW>|so-fipVW-b>$33D+O9xN@;-kvea zexDy1L+X5qa(q+4dvdNv!+~f!5 z*{oeS{+!Zw<3?zxC*(1&B9uasU8HOKeM@;|m{YG_r6T?JhuEIinOgxwNZJH1(ex~V zxkR%%n~FQY)l^@Ri)p~-g=s&G-IT=?btX^3c9r|mUEdzkJ<<6Q>{v5=?=AwzeFFa> zbrU!$SSyBMKVjj12U~RL2Sm~7B(Tp?0F$T1?qkGoisw`~WC8NPX6)b|)MdV{!Dz%c zYnk;f>>AaYU_WrH^!XjE`9Ip_(tNwfqcGqYDhkDRigP%>V0$cwqyRQrXHNuV?g54v z5V_K?hm=8IJ0FE(kcz%V^K55Z&|O?MTJiw-E|BdSRvfmr28%?9Ae1q3j%dNAHe7Pt z2_qKCiwAjT;5bjxj=J~HJCPWR6X8P%uDr-D{o9--;i|YuaCi?Uqkt#9(DOE*_Vtv@ zZ*TKyT+b&nEcCyf;Qjx382uizM z$k!T(yDja2t1W~b=s+xP7jpkBP`CZP!01WuZ|!Q0&qo0Q1$oyJFm+U z7<+99rzA*+2aUaFJH3z8;~Q+klHmq-ZE=rie1HST8FvwYcVi*~W!chq5!cVkySo#B zHOgMAw)k4MXE&EL$T4e6Q9eRSbf+2a3@xR_h@Dv|R2&7NUw?Np?S?HTimq4+*+i9W z;4HToq-b(QIft+jyy%K8LG*T8lV{inJi^7@B6w6$&!~dPc_my*IOoRdXcQjAdxC8G7N<)lBnJe+?eXX)WKQ?F#?X7IM*@d2 zLSQzq2~`p6*T7k-_(k5Lh$=TO;hJIA%0dUl^Fpw-5~qMK`$o7fIQU4T>U7j-?N$97 zPiE$ApsH)FH2>6wSUEcML&whKtumKIlzGo{Ik@PF79icvzrn2C?J{fygFLsY2=3~g zJ`Amp-k9{I_Z^0nk`$#sa=^m!jn$5<$lYop!MQkoU=(4+L!&6q~L}nWMRexqqWfsLb>bJPgv+X;Ad;L;VzkR z&A5GLQ69)wwA2xNx7elcwi&VDB*%Ag)A7P`!HJNgAi6>2S!&TJxkXr} z%Gx0!?yy^7VIz(0_&Of+&~Q9l@BX$l1m0O~qX*}ZUVg2UL1oLa5aI1Gk#H1BmnV)1 zDlktp+lqpP=@y+Az^uHy7fEd_#g z2+O;>(4_j)Z~7)?v9@!4R2ZSN-l%0|N1C9$$Xk0(4_?+4&T~|#hu>@UvGpEA+<}zD zEqI)mvPecGgWnv{BDUDj1^Xd-U2?+l3+&r*~s_;D&WC$^kAT}@8r;&1@a^_H1 zwe)TWBq+STJ>{=3e*+Vtp`rv z22G`AJSrVfot$Bul($8|bt~-i906$owy_QMp?Ya=^H@%6DQM0;_PVaqbK7p0*CZ2~ z&(Ce71%&Lz`3N&0GmpLYY{d7J|`-xj7Meu(0z@cM{gY4A&Itr5id5K2N!vIslGv33yV~P!-Kmx8Aa|MTO(;Cy}i>HN``#sqTe0(Xy@_2m@rq4B(>hI5;x?fUzy->yE+J9s% zBZ;+wo+KWEv1BlGt^;}pr5$m?4E&cIhAw1u|8$t|k;dUTShMz0lywj0u{+oxf1^JVb}9exlU``$HB2z zDY1OpVufHUhw%38v^L3#PEBlW2Tt??eCs-T9C05UUiQ+Mm2wDhntHe_KIV$+r^^p{ zaRCovn?_tT@VB>hM%E=qjNIk{no-&e+*CpBFk;r(t|LQ5^9qcCMTw;c*!p9~(#czt z@6^F`nezedv5&6M2+XtBPGATLDou*6Xt|)eD-h zv`j?ch{2e#df$n5&4^Mrwx*0MTtsZW;N|fGRM8%0qFC2FTDp)b42ZM?&F~I0;UUBg=JnN_^*v`}kl|qk&vVw}LC77P zs6AT<5OftWgXTvAjtG809(Ue!L5&uqeRCli)=KfRKDI0}#2R3tn(fgnYa64MKTjHL z)$?E^lc2T6V+iL3;Kt7ucgr7Zbv4}t**yab29 zuk@Mt0wJ%HKf8;*e_2DP&SBhc58Jj*o@Fj*3tqM?hzN#W8GnigIm}$bgNA;>75nnK zpPQ|i)byRZ!q5oOOUojH#}A$;$9$`!q+=_G?+Xv6VZNd-yx}&j0f-0m@C92?JvbU? zHU>LB0YqADuL%4Sh(2#L?-tLL#PYZNeC-06%>xf#4$P@I1G2)tV-kLdz=;CdPjf-8 z+i`(uy4G<8j2V5J-+GHG!+GC9=(&%AWhb{Dth;YCNn%D_W>kgT(*0wUXVg>kkGqj5iNj)a9)XAd+8mYx*-jR$B^%UGn?*|8aTcp47&$;i)r5l zYopq1hBefp8^WiDOdpY|BmQ|8FD){Tec0oDANGT>%uGs;gL1HJ zL4tKzvg4ZOWovQjMP#O>Ac7P8@B5$31^@msTwbz$Y%x)Q_q&gRT`DKiGOD z#wBs8n`$WZVyy6_QR##YU9iEgx3Hid8Z2YHONALFHWV54n7&nx2g!Opt#+AcavIlu zmF}Bbx@y*AX@>l8@-5q5c+DWL=vG$)jxl~>-k9A_(WYRJ;pNn28Ds4alj3p1+w@eC zFm{}1V||II!*XczS9zqkecOg<0!ic@gYIDG!+alkjYxJk0kY|Hn*tvK@^n+>(IRD8 z%IG%fu-9x3iNFng*f7VV(3_RP!X1~uNv`EvH6n=~55MHOKYy`e)yF)#4Hq(2x$+YF znKf{tR;ktWR>jaJd#tiS!)E~kwrR6-zL>#!cI#2+CgJhU2I22~Q`E+Tg`-Oyt3F=T z0krJy%qvlR*Nn#Avuccf*q$Cfm)=+^@Iu`WuL!b#J-*NJ%+kq?1HzVv37$CX948X! znRc9In>Q8}mVD(I%$R(5<;?D&0dvWu9(s&CJ?z5dsVEJVT@g!co(4~!>cYt|Q4