Does anyone succeed on imagenet?

[HuaZheLei]

I tried some small Ps, some small Ks and many learning rates. But I always get a loss of 0.693. Anyone can share his/her experience on imagenet?

[lucasb-eyer]

Hi, it's a great question and while I have ImageNet experience, I have never tried triplet there.

For the hardest dataset I had, I needed to use P=2,K=2 until the network got past those 0.693 and then gradually increase them. Are you able to converge with P=2, K=2?

Alternatively, you can try the batch_sample version of the loss that I implemented in the sampling branch (but did not get the time to merge yet, see #33.) I have very good experience with it on very large datasets, but haven't tried it on ImageNet either.

I would love to hear about your experiences trying these!

[ergysr]

From my experience (not on ImageNet) the loss doesn't get past the margin when either sample difficulty or label noise is high. Few alternatives are the batch_sample version that Lucas suggests, the weighted triplet version, or a multi-task loss (e.g. hard triplet combined with categorical cross-entropy). I have tried the last two and they work well with hard/noisy examples. So you have at least three options to try.

[willard-yuan]

I have trained with P=2,K=2 on Tiny-Imagenet-200, and the loss doesn't converge. Using the batch_sample first and then batch_hard works well.

[lucasb-eyer]

Thanks for the feedback @willard-yuan, that's useful feedback.

[mazatov]

I just came across this discussion. What's the significance of 0.693 number? @HuaZheLei @ergysr

I was trying to train on Market1501 but with fc1024_normalize. The model converged for fc1024 but got stuck at 0.693 for the normalized version. I had it previously stuck at 0.693 for other datasets too. 🤷‍♂