Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache/store sync status #109

Open
mazay opened this issue Jul 21, 2021 · 0 comments
Open

Cache/store sync status #109

mazay opened this issue Jul 21, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@mazay
Copy link
Owner

mazay commented Jul 21, 2021

Consider adding an optional cache for storing the sync status - list of files/s3 keys with checksums (maybe mtime) to speedup the initial sync which could take a lot of time depending on the data size. With cache restarts/reloads should be relatively fast.

Cache may also help with improving retries for failed files, e.g. save failed file in redid and retry in X s/m/h.

@mazay mazay added the enhancement New feature or request label Jul 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant