Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is the test result the average value of multiple models? #20

Open
yelusaleng opened this issue Jul 16, 2021 · 2 comments
Open

Is the test result the average value of multiple models? #20

yelusaleng opened this issue Jul 16, 2021 · 2 comments

Comments

@yelusaleng
Copy link

hi authors,
i‘ve only tested the performance of cutblur once by using python main.py --model CARN --augs cutblur --alpha 0.7 --dataset RealSR --scale 4 --camera all --dataset_root ./input/RealSR/ --ckpt_root ./pt/RealSR/cutblur/ --save_result --save_root ./output/RealSR/cutblur/.
the obtained result is 28.89 which is lower than the result 29.00 in the paper.

therefore, i would like to know if 29.00 is the average of multiple models tested.

@nmhkahn
Copy link
Collaborator

nmhkahn commented Jul 19, 2021

In the RealSR dataset, we have observed that early stopping is crucial for the best performance. The overfitting seems to worsen when the network is bigger (e.g. EDSR, RCAN) and when not using data augmentation, obviously.
Is the 28.89 PSNR of the "best"? or "the last" result? By the way, all the performance reports are measured a single time.

@yelusaleng
Copy link
Author

thank you for your response.
the 28.89 is the best, not the last.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants