add Harmonization

[candemircan]

Adding a custom Harmonization class. The model is explained in this paper and is available under this repo. It is a type of model that gets deep neural networks to rely on similar aspects of images as humans do while doing object recognition.

[LukasMut]

@candemircan Thanks for this PR! I think this will definitely be a useful addition to the current models in thingsvision. I know Drew's work quite well. Could you add tests for at least two of the models, and update the README.md similarly to how you've already changed the docs? I think you need to rebase before you do any of the above (due to recent changes in the codebase).

[candemircan]

hey Lukas :) thanks for the quick response. Just made the changes as you requested. Hope it works out fine!

Best,
Can

[LukasMut]

@candemircan, thanks a lot for the changes! Could you rebase one more time and submit the PR again? I had to accept a different PR before I could accept yours.

[LukasMut]

@candemircan Also, could you do me a favor and remove the folder build/lib after you've rebased? This folder was accidentally added during the previous PR but shouldn't part of the the repo.

[LukasMut]

@candemircan I see that the unittests are not passing. Did you rebase correctly?

[candemircan]

@LukasMut I believe I did, but a comma was missing. Now I fixed it. Let's see if it passes now

[LukasMut]

@candemircan Unfortunately, the unittests are still failing. I think there's a module missing for one of the harmonization models,

ModuleNotFoundError: No module named 'keras_cv_attention_models.maxvit'

Please see yourself when and why the tests are failing.

[LukasMut]

@candemircan I see that you've added the missing module to the requirements.txt file, but I am wondering whether it has to be added to the setup.py and environment.yml files too? Unfortunately, the tests are still failing for some reason. Could you test locally what's going on?

[candemircan]

Hi Lukas, Apologies for the repeated fails :) Yes, I’ll make sure all works locally and push a new commit sometime this week. 20 Feb 2023 at 16:00 by ***@***.***:

…

@candemircan <https://github.com/candemircan>> I see that you've added the module to the > requirements.txt> file, but I am wondering whether it has to be added to the > setup.py> and > environment.yml> files too? Unfortunately, the tests are still failing for some reason. Could you test locally what's going on? — Reply to this email directly, > view it on GitHub <#130 (comment)>> , or > unsubscribe <https://github.com/notifications/unsubscribe-auth/APEIP2XODEEAT4BSNNKHBNTWYOBH7ANCNFSM6AAAAAAVATEIGE>> . You are receiving this because you were mentioned.> Message ID: > <ViCCo-Group/thingsvision/pull/130/c1437153575> @> github> .> com>

[candemircan]

Hi again Lukas,

I have added the keras_cv_attention_models to requirements.txt and setup.py. I see that the dependency is also installed under GitHub Actions, however the mac-os test still fails with the same ModuleNotFoundError: No module named 'keras_cv_attention_models.maxvit' message. Do you have any idea why this may still be the case?

Best,
Can

[LukasMut]

@Alxmrphi, do you have any idea what's going on with codecov here?

[LukasMut]

@Alxmrphi, do you have any idea what's going on with codecov here?

My gut feeling tells me that it has something to do with the TensorFlow version, but I am not 100% sure. One could also try to pip install git+https://github.com/leondgarse/keras_cv_attention_models rather than keras-cv-attention-models.

@candemircan do you use keras-cv-attention-models or keras_cv_attention_models for installing the module via pip? I think it needs to be the former (although the Harmonization repo uses the latter).

What's interesting is that the unittests are both passing but codecov is failing 🤔

[Alxmrphi]

I'm scratching my head at this one, too. How is it passing the unit tests but not the code coverage? I downloaded and ran the unit tests locally and they fail on the harmonization model tests. Quick fixes to install libraries, more failures in different places. The first hurdle it fails at is not having access to keras_applications.

[candemircan]

@candemircan do you use keras-cv-attention-models or keras_cv_attention_models for installing the module via pip? I think it needs to be the former (although the Harmonization repo uses the latter).

Hi Lukas,

I was using the latter, but you are right that the package is named as the former in pypi. Should I change it and make another commit?

[LukasMut]

@candemircan do you use keras-cv-attention-models or keras_cv_attention_models for installing the module via pip? I think it needs to be the former (although the Harmonization repo uses the latter).

Hi Lukas,

I was using the latter, but you are right that the package is named as the former in pypi. Should I change it and make another commit?

@candemircan Let me see whether I can figure this out. I feel like that error is not happening on our end.

[LukasMut]

I'm scratching my head at this one, too. How is it passing the unit tests but not the code coverage? I downloaded and ran the unit tests locally and they fail on the harmonization model tests. Quick fixes to install libraries, more failures in different places. The first hurdle it fails at is not having access to keras_applications.

@Alxmrphi, I see that this error is happening within the harmonization module and not on our end. See the issue that references this PR.

[LukasMut]

I approve this PR but I'd like to wait until we've fixed the dependency issues before I merge it into the main branch.

[andropar]

Same as Lukas, we should wait until we figured out what causes codecov to fail, but the rest of the PR looks fine to me!