Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to run without cache #19

Open
Randl opened this issue Apr 27, 2020 · 1 comment
Open

Allow to run without cache #19

Randl opened this issue Apr 27, 2020 · 1 comment

Comments

@Randl
Copy link

Randl commented Apr 27, 2020

As discussed in Slack, having option to ignore cache can be useful sometimes.

In particular, if I run model someone already run, the result is returned from cache. However, the runtime isn't fetched (or cached?). Then, it is still shown in leaderboard. In case the original run is in leaderboard it is duplicated, otherwise it appears only once but without runtime. Both seem to me undesirable.

Probably, cache should be attached to a repository, and there should be a way of either clearing it or ignoring it for a run, for example with "Re-run without cache" button.

Randl added a commit to Randl/TResNet that referenced this issue May 4, 2020
@Randl
Copy link
Author

Randl commented May 18, 2020

Same problem if build fails half-way, see for example https://sotabench.com/user/EvgeniiZh/repos/Randl/ECANet#latest-results
The models which were measured in first build do not have runtime associated with them because it was measured in build which later failed.
A simple band-aid would be caching runtime too (that probably would require to invalidate existing caches tho)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant