Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Concept - Unpopular cache items don't forward propagate #53

Open
Turnerj opened this issue May 12, 2020 · 1 comment
Open

Concept - Unpopular cache items don't forward propagate #53

Turnerj opened this issue May 12, 2020 · 1 comment
Labels
enhancement New feature or request
Milestone

Comments

@Turnerj
Copy link
Member

Turnerj commented May 12, 2020

https://blog.cloudflare.com/why-we-started-putting-unpopular-assets-in-memory/

For Cloudflare, they wanted to prevent unnecessary writes to SSDs to keep them around longer so they didn't want to have unpopular cache items in their SSD cache. On top of the lifetime benefits, it also increased overall performance as writes-while-reading slows down individual chips on an SSD.

While CacheTower is unaware of the storage medium, it would be interesting to implement a similar approach. If we track how often an item in the cache is hit, we can track whether we should even back populate it from in-memory to whatever other cache layers are configured.

This can prevent unnecessary writing and storage for "one-hit-wonders" in deeper layers of the cache stack. If you are using Redis, it would take pressure off the Redis server. If you are using Disk, it would take pressure off the disk.

At minimum we would need:

  • To track every cache access (As a uint perhaps?)
  • Update back populate logic to check if cache access is often or not

Considerations:

  • Does this only work for in-memory cache?
  • If not, does this work just for the first cache layer?
@Turnerj Turnerj added the enhancement New feature or request label May 12, 2020
@Turnerj Turnerj added this to the v1.0 milestone May 12, 2020
@Turnerj
Copy link
Member Author

Turnerj commented May 14, 2020

As an aside to this, potentially establishing an Least Recently Used (LRU) eviction strategy. While I can't measure the memory used in the cache (I don't serialize it, I just reference it), maybe there is some backend API I can use to gauge memory usage? Or just do it dumbly by the number of items?

@Turnerj Turnerj changed the title Concept - Unpopular cache items don't back populate Concept - Unpopular cache items don't forward propagate May 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant