Recognize results are lost when transferring files to other user

[andreaskoelsch]

Which version of recognize are you using?

6.1.1

Enabled Modes

Object recognition, Face recognition

TensorFlow mode

Normal mode

Downstream App

Files App

Which Nextcloud version do you have installed?

28.0.4

Which Operating system do you have installed?

Debian 12

Which database are you running Nextcloud on?

MariaDB 10.11.6

Which Docker container are you using to run Nextcloud? (if applicable)

No response

How much RAM does your server have?

16GB

What processor Architecture does your CPU have?

x86_64

Describe the Bug

When files are transferred from one user to another user via transfer ownership, only the classification results are transferred, face detections are lost. Also, the transferred files are not added to the face detection queue for detecting the faces again.

Expected Behavior

Either keep the face recognition and run the clustering for the new user or re-run both face recognition and clustering after moving the files.

To Reproduce

• Upload images to one user
• let recognize do its thing
• transfer ownership of these images to another user
• see that face detections are lost and files are not added to face detection queue

Debug log

No response

[github-actions]

Hello 👋

Thank you for taking the time to open this issue with recognize. I know it's frustrating when software
causes problems. You have made the right choice to come here and open an issue to make sure your problem gets looked at
and if possible solved.
I try to answer all issues and if possible fix all bugs here, but it sometimes takes a while until I get to it.
Until then, please be patient.
Note also that GitHub is a place where people meet to make software better together. Nobody here is under any obligation
to help you, solve your problems or deliver on any expectations or demands you may have, but if enough people come together we can
collaborate to make this software better. For everyone.
Thus, if you can, you could also look at other issues to see whether you can help other people with your knowledge
and experience. If you have coding experience it would also be awesome if you could step up to dive into the code and
try to fix the odd bug yourself. Everyone will be thankful for extra helping hands!
One last word: If you feel, at any point, like you need to vent, this is not the place for it; you can go to the forum,
to twitter or somewhere else. But this is a technical issue tracker, so please make sure to
focus on the tech and keep your opinions to yourself. (Also see our Code of Conduct. Really.)

I look forward to working with you on this issue
Cheers 💙

[andreaskoelsch]

PS: When moving files to a Group Folder, both face detection and object classification are re-triggered. Is this intended or worth a separate issue?

[andreaskoelsch]

It gets even more odd. I just realized, when I move separate files to Group Folders, they are triggered. If I move entire folders, on the other hand, the contained files are not triggered. Pretty sure this is not what's intended

[andreaskoelsch]

PPS: Even though the files which are moved separately to Group Folders are reprocessed by recognize, the faces do not show up for the other users, but only for the one who moved them there.

Current workaround: move the files/folders via the filesystem and re-scan the directories. That way, they are reprocessed and the results show up for all users in the Group Folder.

[marcelklehr]

Hello @andreaskoelsch

Thank you for this detailed analysis! File transfer to other users is a relatively new feature in nextcloud, so we haven't integrated with this feature yet. I'll also look into the other issues you found.

[andreaskoelsch]

Great, looking forward to testing it 😊

[andreaskoelsch]

I just found the time to test this today. Seems, it's still not fixed, as the face clustering is not triggered. When I move a file from my personal space to a group folder, it gets added to the classify and face recognitiion queues. However, after the predictions, the detected faces are not added to the clustering. In my own profile, the face recognition works, because it was already detected before moving the file to the group folder. However, other users of the group folder do not get a face recognition

[marcelklehr]

Mh, that may be because group folders are not technically shares, while I've been using the share manager to enumerate users with access. I'll change that to use the mount cache instead...

[marcelklehr]

Hey @andreaskoelsch
I've prepared a beta version with a potential fix. Could you try this https://github.com/nextcloud/recognize/releases/tag/v7.0.2-beta.1 and report back here?

[andreaskoelsch]

Yes, we're almost there ;-) Thanks for the beta, I just tested it by uploading a picture of myself and moving it to a group folder. What happened after moving the file:

• re-triggered face and imagenet classification
• found face
• 1 face left for clustering, planned tasks: 0

So, what's left is that the clustering needs to trigger a new task after the face detection.

Also, what I found not ideal: After moving the file to the group folder, the classification labels are already available for both users before the imagenet executes again. It seems, they are already transferred with the file and another imagenet execution could be avoided for energy efficiency. Also, this makes me wonder if it would maybe be possible to transfer the face detection in a similar way, so the other group users would only need to run the clustering of the newly available faces instead of having to re-run the face detection. However, face detection is cheap, so if this is hard to implement, I won't complain

[marcelklehr]

No new classification should be run when moving files :/ Are you sure that another face classification is run on the file? The face detection should just be copied.

[andreaskoelsch]

Yes, absolutely. That was already the case in the comment above (#1118 (comment))

[marcelklehr]

Can you try moving a folder with a file in it, to check if the face detections get copied to the groupfolder users? Then all i have to fix is the re-classification.

[andreaskoelsch]

I just tested this, i.e.moving a folder containing a file with a face detection to a group folder. After moving the entire folder, neither face detection nor clustering nor imagenet classification are triggered for this file.

The original user of the file still sees the face detection and classification labels of the file. Other users only see the classification labels, but no face detection. I also checked the unassigned faces tab, but nothing there.

[marcelklehr]

And just as a sanity check, when moving the folder to a shared folder, does the face detection show up for the sharees of that share?

[andreaskoelsch]

Is that question answered with the second paragraph of my last comment? otherwise, I don't understand it.. To repeat, only the original user sees the face detection of that file.

Btw, thanks for the quick replies ❤️

[marcelklehr]

Sorry, I will explain. There's groupfolders and there is normal shared folders (ie. a folder that a user shared with some other users). Could you repeat your experiment but instead of moving the folder with face pictures into a group folder, could you move it into a normal folder that is simply shared with some users? The question is again, does this copy the face detections to the other users. (From a technical perspective, groupfolders and shared folders are vastly different beasts)

[andreaskoelsch]

Ohh, gotcha. So, I tried sharing this folder with one image inside with another user. Interestingly, recognize then showed 2 faces left for clustering, 0 planned tasks. Not sure, why two faces.. anyways, I ran occ recognize:cluster-faces manually and the second user also shows the face.

ClusterDebug: Retrieving face detections for user testuser
ClusterDebug: Found 2 fresh detections. Adding 0 old detections and 2078 sampled detections from already existing clusters. Calculating clusters on 2080 detections.
ClusterDebug: Clustering complete. Total num of clustered detections: 2

[marcelklehr]

Ok. So, to summarize:

• groupfolders still don't work
• moving files triggers new classifications

[andreaskoelsch]

Not quite. Let me give a more detailed summary. Given a selfie with one face detection which I deleted and re-uploaded for each test scenario. All tests done with the most recent beta.

• Shared Folder:

  • Sharing a folder with a selfie inside says 2 new faces in the recognize admin panel, even though there's just one detection there. Also, this requires a manual call of clustering, then it works for all sharees
  • Moving the selfie to an already existing shared folder works (only one new face waiting for clustering), but requires manual call of clustering. Afterwards, both sharees see the detection
  • Moving a folder with the selfie inside to an already exising shared folder does not work. No new detections for the sharees.

• Group folders:

  • Moving the selfie to the group folder retriggers face and imagenet classifications. Afterwards, it says 1 new detections, 0 planned tasks. After manual clustering call, all group folder users see the detection.
  • Moving the folder with the selfie inside to a group folder neither triggers a classification nor are there new detections. Very much like in the third case above.

So moving individual files works for both sharing and group folders, but requires manual clustering call and causes unnecessary classifications in case of group folders. Moving folders with files inside does not work.

PS: Classification labels work in all cases!

[marcelklehr]

So, I believe I've found the culprit(s). I'll make available a new beta later today, so you can test :)

[andreaskoelsch]

Was this last commit supposed to be the new beta, or will you publish a beta release? I don't want to stress you, just want to know whether I should already test or still wait.

[marcelklehr]

Nope, I still haven't published the beta, sorry.

[andreaskoelsch]

no worries, it's not urgent

[marcelklehr]

here you go :) https://github.com/nextcloud/recognize/releases/tag/v7.0.2-beta.2

[andreaskoelsch]

Thanks! So far, I've only tested one use case: Moving files to a group folder. Unfortunately, the moved files are still added to both face recognition and imagenet queues. However, Face Clustering for the group folder users is now triggered and working.

[marcelklehr]

Unfortunately, the moved files are still added to both face recognition and imagenet queues

I think I've fixed that now, too.

[andreaskoelsch]

Locally? At least not in the current beta

[marcelklehr]

Locally? At least not in the current beta

Yes, not in the beta, sorry, in the repo :)

[andreaskoelsch]

Today, I had a case of a moved folder. Face detections were not copied to other users. So, this is still not working. I can do some more testing with the next version

[marcelklehr]

I've published a new release just now with the fix

[andreaskoelsch]

Thanks for the effort. I just tested it and it seems the classification is now broken completely. I created a new ticket for this, as it's probably more urgent (#1154)

[andreaskoelsch]

Thanks for the quick fix to above issue. I tested again now. Moving files works as expected now, moving folders with files inside them still not.

[marcelklehr]

I'll look into this soon.

[marcelklehr]

Hi @andreaskoelsch
the priority of this issue was reduced to "normal". This means Nextcloud GmbH will not investigate this actively anymore. We will however, happily accept pull requests with fixes.

[andreaskoelsch]

Thanks for the message, it's alright for me. At least it works now by moving files instead of folders, so there is a simple workaround for the issue