why get same output with different input?

[maozezhong]

@lsdefine Thanks for your sharing, I use transformer to do seq2seq task. Like, input a article and predict the abstract. When I finish training, I get almost same output with different input. Code are same as your example, data should be right, because with same data, and use lstm block as seq2seq, I got the proper output.
Hope for your answer, Thanks.

[mellahysf]

Hi.
I have the same question. I used the same code with the same dataset (en2de).
@maozezhong did you found the response ?
Thank you .

[lsdefine]

The learning rate and the optimization strategy should be carefully tuned. I suggest using the same optimization strategy as BERT. MELLAH Youssef <[email protected]> 于2020年6月28日周日 上午6:38写道：

…

Hi. I have the same question. I used the same code with the same dataset (en2de). @maozezhong <https://github.com/maozezhong> did you found the response ? Thank you . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZYVG4RLBT5TNAJK43LUQDRYZYFNANCNFSM4IXKBDHA> .

[mellahysf]

Thank you very much for response.
I think my question was not clear .
I'm using your code to train in the same dataSet (en2de) and the accuracy is good . 70% in accuracy it means that the model is capable of correctly responding to 70% of dataset inputs, but when i test the pretrained modele (eve if with inputs from the same dataset), it gives me about the same response !! it is normal ?
Can you share with us the pretrained model ore some uses cases of testing it ? (from dataset inputs or other inputs)

Thank you very much.

[lsdefine]

Did you mean that the ``eval'' mode in the code does not work? Is the model loading failed? Check the line 34-35. By the way, a newer version of tf and keras has the similar problem. keras-team/keras#7177 You can try to set the learning_phrase manually. MELLAH Youssef <[email protected]> 于2020年6月28日周日 下午7:55写道：

…

Thank you very much for response. I think my question was not clear . I'm using your code to train in the same dataSet (en2de) and the accuracy is good . 70% in accuracy it means that the model is capable of correctly responding to 70% of dataset inputs, but when i test the pretrained modele (eve if with inputs from the same dataset), it gives me about the same response !! it is normal ? Can you share with us the pretrained model ore some uses cases of testing it ? (from dataset inputs or other inputs) Thank you very much. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZYVG4ZJRD64L74TTECSYLRY4VR5ANCNFSM4IXKBDHA> .