Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Android sample (JNI binding + Java wrapper class) #101

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

shubham0204
Copy link

@shubham0204 shubham0204 commented Sep 14, 2024

This PR adds examples/clip.android, an Android sample that interfaces with the shared libraries of clip.cpp through JNI. The structure of the project is highly inspired from llama.cpp/examples/llama.android.

Summary of changes

  • examples/clip.android is an Android project with two modules app and clip.
  • The app module contains the sources for a demo application which allows the user to select an image and enter a text description to compare them (uses Kotlin/Compose)
  • The clip module contains clip/src/main/cpp/clip-android that contains the JNI interfaces which allow communication with the underlying functions from clip.h.
  • Also, the clip module contains clip/src/main/java/android/clip/cpp/CLIPAndroid.java which is wrapper class containing native methods.
  • The clip module also houses an instrumented test clip/src/androidTest/java/android/example/clip/CLIPInstrumentedTest.kt written in Kotlin
  • A README.md is included in the examples/clip.android directory which describes how to run the sample app and the instrumented test
  • clip.cpp is modified to address issue (Possible) Incorrect assignment of image mean/std values in clip_model_load() #99
  • CMakeLists.txt is modified to add -mcpu=native when ${CMAKE_SYSTEM_PROCESSOR} MATCHES "aarch64" only if -DCLIP_NATIVE is enabled
  • added a new method clip_image_preprocess_no_resize in clip.h and clip.cpp which does works exactly like clip_image_preprocess but does not perform linear interpolation (no image resizing). Android's Bitmap class and Python's PIL can handle bilinear interpolation easily, so it would be great to have a method that skips resizing.

Note

Though the clip module here in built as an Android library module, it can be built into a 'Java library' module (packaged as a JAR) and used in other Java-only projects as a port to clip.cpp

It was fun working on this sample, and I would like @monatis to review it and share feedback.

…size, update tests to remove magnitude comparison
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant