Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to @vladmandic/human #1218

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Switch to @vladmandic/human #1218

wants to merge 4 commits into from

Conversation

ghisch
Copy link

@ghisch ghisch commented Dec 2, 2024

This PR updates the face detection classifier from @vladmandic/face-api to @vladmandic/human.
Fix #1171

Key Changes

  1. Improved Performance

    • @vladmandic/human offers significantly better performance compared to face-api, with faster processing and more reliable face detection.
    • It detects more faces per image
    • Unexpectedly identifies pet faces (tested with my dog 🐶).
  2. Configuration Adjustments

    • Updated maxDetected to 100 (default is 1) to support multiple faces in a single image.
    • Face vector: previously called descriptor (128 dimensions in face-api), now called embedding (1024 dimensions in human).
  3. Face Clustering

    • Face clustering does not work properly right now. It's limited to handling duplicate images. I don't know why. I feel it's linked to the dimensions size difference.
    • Clustering parameters (MIN_CLUSTER_SEPARATION, MAX_CLUSTER_EDGE_LENGTH) might need optimization for the new 1024-dimensional embeddings. Suggestions and assistance on tuning these are welcome! 🙏
  4. Compatibility Note

    • Breaking Change: Existing detections made with face-api are incompatible with human embeddings. This requires a reset of existing data to align with the new system.

Testing

  • Tested on my LSIO Nextcloud instance in PURE_JS (WASM) mode only.
  • Needs testing in GPU, CPU, and Node.js environments.

Request for Feedback

  • I'm not a developer and relied on ChatGPT for much of the implementation. Feedback and suggestions for improvement are highly appreciated.

Recap (Help Needed 🙏)

  • Optimize face clustering to accommodate the higher-dimensional embeddings.
  • Verify functionality across GPU, CPU, and Node.js environments.

Let me know if there's anything I can improve or clarify further!

Adapt dimensions to 1024 as outputed by @vladmandic/human

Signed-off-by: ghisch <[email protected]>
@ghisch ghisch marked this pull request as ready for review December 2, 2024 02:07
@Felitendo
Copy link

Thank you sooo much for this PR, I think this is strongly needed. The face-api Repo hasn't gotten a meaningful commit since January and is also Archived now. Hopefully @marcelklehr can look at this PR when he has time :)

@marcelklehr
Copy link
Member

Hello @ghisch
Thank you for taking the time to compose this PR. Indeed using a new embedding model needs careful calibration of the hyperparameters and extensive testing. I can currently not provide either in my capacity as Nextcloud GmbH engineer, sadly. I can try to look into this in my free time, but can't be sure when I'll get to this.

@ghisch
Copy link
Author

ghisch commented Dec 2, 2024

Hello @marcelklehr 👋 I totally understand no problem.

Also, FYI, human lib repo also provide some kind of clustering in JS, I feel like inspiration can be taken from here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

switch face-api for human
3 participants