Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Improve docs #2

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions IMPLEMENTATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
There are two ways we update a cached count:
* When we try to fetch the count, if it isn't already cached, we calculate it from the database and cache the result.
* When a record is created/deleted/updated, an after_commit hook updates the count using Memcached's `incr` and `decr` commands.

It's possible for the count to get out of sync with what's in the database. At Academia.edu, we use this method of counting only when we're okay with some inaccuracy:
* We create models much more often than we destroy them. Thus, the count is roughly monotonic over time. This means that only large values are likely to get out of sync, for two reasons:
* Small values are only a few unreliable operations away from a database recalculation, whereas large values are many unreliable operations away.
* Small values have expiry-resetting operations (i.e., `incr`/`decr`) applied to them infrequently, so they expire and are recalculated often.
* It's often okay for big values to be slightly off:
* In paginated lists where the total item count is displayed to the user.
* In calculations that depend on only the order of magnitude of the count, not the exact value.
4 changes: 4 additions & 0 deletions README.rdoc
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ There are {many alternatives}[https://rubygems.org/search?utf8=%E2%9C%93&query=c

{Documentation}[http://academia-edu.github.io/cached_counts/]

In your Gemfile:

gem 'incrdecr_cached_counts', require: 'cached_counts'

Basic usage:

class User < ActiveRecord::Base
Expand Down
136 changes: 97 additions & 39 deletions lib/cached_counts.rb
Original file line number Diff line number Diff line change
@@ -1,50 +1,73 @@
require 'cached_counts/dalli_check'
require 'cached_counts/connection_for'

# A mixin that provides two class methods: +caches_count_where+ and +caches_count_of+.
module CachedCounts
extend ActiveSupport::Concern

module ClassMethods
# Cache the count for a scope in memcached.
# Cache the count for a scope in Memcached.
#
# e.g.
# User.caches_count_where :confirmed
# > User.confirmed_count # User.confirmed.count, but cached
# For example, if you have a model:
# class Post < ActiveRecord::Base
# include CachedCounts
# scope :sponsored, -> { where(sponsored: true) }
# caches_count_where :sponsored, if: :sponsored?
# end
#
# Automatically adds after_commit hooks which increment/decrement the value
# in memcached when needed. Queries the db on cache miss.
# Then you can call
# Post.sponsored_count
# to fetch the number of sponsored posts from Memcached.
#
# Defines a few class methods:
# [+#{attribute_name}_count+]
# The number of models in the scope.
# [+#{attribute_name}_count_key+]
# The key in Memcached that stores the count.
# [+expire_#{attribute_name}_count+]
# Deletes the key from Memcached.
# [+#{attribute_name}_count=(value)+]
# Sets the value of the count. (Should be necessary only for testing.)
#
# And a few instance methods:
# [+increment_#{attribute_name}_count+]
# Increments the count if it already exists. (Should be necessary only for testing.)
# [+decrement_#{attribute_name}_count+]
# Decrements the count if it already exists. (Should be necessary only for testing.)
#
# @param [String] attribute_name
# The name that appears in generated method names.
#
# @param [Hash] options
#
# @option options [String] :scope
# Name of the scope to count. Defaults to the +attribute_name+
# (the required argument to +caches_count_where+).
# Name of the scope to count. Defaults to the +attribute_name+.
#
# @option options [Proc, Symbol] :if
# Proc that decides whether an object is included in the count. (This must
# be equivalent to +:scope+.)
#
# @option options [String, Array<String>] :alias
# Alias(es) for the count attribute.
# e.g.
# caches_count_where :confirmed, alias: 'sitemap'
# > User.sitemap_count
# Alias(es) for the count attribute. For example:
# class Post < ActiveRecord::Base
# # ...
# caches_count_where :sponsored, alias: 'paid'
# end
#
# Post.paid_count
#
# @option options [Integer] :expires_in
# Expiry for the cached value.
#
# @option options [Proc] :if
# proc passed through to the after_commit hooks;
# decides whether an object counts towards the association total.
#
# @option options [Integer, #to_s] :version
# Cache version - bump if you change the definition of a count.
#
# @option options [Proc] :race_condition_fallback
# Fallback to the result of this proc if the cache is empty, while
# loading the actual value from the db. Works similarly to
# +race_condition_ttl+ but for empty caches rather than expired values.
# Meant to prevent a thundering-herd scenario, if for example a
# memcached instance goes away. Can be nil; defaults to using a value
# grabbed from the cache or DB at startup.
# On cache miss, first write the result of this proc to the cache (to
# prevent a thundering herd), and then perform the count and update the
# cache.
#
# Defaults to a value grabbed from the cache or the database at startup.
#
def caches_count_where(attribute_name, options = {})
# Delay actual run to work around circular dependencies
Expand All @@ -54,39 +77,74 @@ def caches_count_where(attribute_name, options = {})
end
end

# Cache the count for an association in memcached.
# Cache the count for an association in Memcached.
#
# e.g.
# User.caches_count_of :friends
# > User.first.friends_count # Users.first.friends.count, but cached
# For example, if you have two associated models:
# class Post < ActiveRecord::Base
# include CachedCounts
# has_many :comments
# caches_count_of :comments
# end
#
# Automatically adds after_commit hooks to the associated class which
# increment/decrement the value in memcached when needed. Queries the db
# on cache miss.
# class Comment < ActiveRecord::Base
# include CachedCounts
# belongs_to :post
# end
#
# Then you can call
# Post.first.comments_count
# to fetch the number of comments on the first post from Memcached. Note
# that the associated class must +include CachedCounts+ also.
#
# Defines a few instance methods:
# [+#{attribute_name}_count+]
# The number of models in the association.
# [+#{attribute_name}_count_key+]
# The key in Memcached that stores the count.
# [+expire_#{attribute_name}_count+]
# Deletes the key from Memcached.
# [+#{attribute_name}_count=(value)+]
# Sets the value of the count. (Should be necessary only for testing.)
#
# And a few class methods:
# [+#{attribute_name}_count_for(id)+]
# The number of models associated with the given +id+.
# [+#{attribute_name}_count_key(id)+]
# The key in Memcached that stores the count for the given +id+.
#
# And a few instance methods on the associated class:
# [+increment_#{klass.name.demodulize.underscore}_#{attribute_name}_count+]
# Increments the count if it already exists. (Should be necessary only for testing.)
# [+decrement_#{klass.name.demodulize.underscore}_#{attribute_name}_count+]
# Decrements the count if it already exists. (Should be necessary only for testing.)
#
# @param [String] attribute_name
# The name that appears in the generated method names.
#
# @param [Hash] options
#
# @option options [Symbol] :association
# Name of the association to count. Defaults to the +attribute_name+
# (the required argument to +caches_count_of+).
# Name of the association to count. Defaults to the +attribute_name+.
#
# @option options [String, Array<String>] :alias
# Alias(es) for the count attribute. Useful with join tables.
# e.g.
# caches_count_of :user_departments, alias: 'users'
# > Department.first.users_count
# Alias(es) for the count attribute. For example:
# class Post < ActiveRecord::Base
# # ...
# caches_count_of :comments, alias: 'replies'
# end
#
# Post.first.replies_count
#
# @option options [Integer] :expires_in
# Expiry for the cached value.
#
# @option options [Proc] :if
# proc passed through to the after_commit hooks on the counted class;
# decides whether an object counts towards the association total.
# @option options [Proc, Symbol] :if
# Proc that decides whether an object is included in the count. (Must be
# equivalent to +:scope+.)
#
# @option options [Proc] :scope
# proc used like an ActiveRecord scope on the counted class on cache misses.
# Proc used like an ActiveRecord scope to decide which objects are
# included in the count. (Must be equivalent to +:if+.)
#
# @option options [Integer, #to_s] :version
# Cache version - bump if you change the definition of a count.
Expand Down