Skip to content

Commit

Permalink
Ensure records are fetched early to avoid multiple DB queries
Browse files Browse the repository at this point in the history
One of the optimizations that cursor pagination allows is fetching one
more record than required from the database to then be able to
efficiently determine if there is another page (which would contain at
the least this extra item). However, the way this was implemented it
actually triggered individual queries, one to load the records and
another `COUNT` query to determine the amount of records with the
additional one. This was due to how ActiveRecord tries to delay fetching
records of a relation to the latest possible point.

To prevent this, we can call `#fetch` manually to request the records to
be loaded earlier.
  • Loading branch information
nicolas-fricke committed Apr 19, 2021
1 parent dfc7e96 commit f790973
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 6 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ These are the latest changes on the project's `master` branch that have not yet
### Changed
- **Breaking change:** The way records are retrieved from a given cursor has been changed to no longer use `CONCAT` but instead simply use a compound `WHERE` clause in case of a custom order and having both the custom field as well as the `id` field in the `ORDER BY` query. This is a breaking change since it now changes the internal order of how records with the same value of the `order_by` field are returned.
- Remove `ORDER BY` clause from `COUNT` queries

### Fixed
- Only trigger one SQL query to load the records from the database and use it to determine if there was a previous / is a next page

## [0.1.3] - 2021-03-17

Expand Down
14 changes: 8 additions & 6 deletions lib/rails_cursor_pagination/paginator.rb
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ def previous_page?
else
# When paginating backwards, if we managed to load one more record than
# requested, this record will be available on the previous page.
@page_size < limited_relation_plus_one.reorder('').size
records_plus_one.size > @page_size
end
end

Expand All @@ -202,7 +202,7 @@ def next_page?
if paginate_forward?
# When paginating forward, if we managed to load one more record than
# requested, this record will be available on the next page.
@page_size < limited_relation_plus_one.reorder('').size
records_plus_one.size > @page_size
else
# When paginating backward, if applying our cursor reduced the number
# records returned, we know that the missing records will be on
Expand All @@ -215,19 +215,21 @@ def next_page?
#
# @return [Array<ActiveRecord>]
def records
records = limited_relation_plus_one.first(@page_size)
records = records_plus_one.first(@page_size)

paginate_forward? ? records : records.reverse
end

# Apply limit to filtered and sorted relation that contains one item more
# than the user-requested page size. This is useful for determining if there
# is an additional page available without having to do a separate DB query.
# Then, fetch the records from the database to prevent multiple queries to
# load the records and count them.
#
# @return [ActiveRecord::Relation]
def limited_relation_plus_one
memoize :limited_relation_plus_one do
filtered_and_sorted_relation.limit(@page_size + 1)
def records_plus_one
memoize :records_plus_one do
filtered_and_sorted_relation.limit(@page_size + 1).load
end
end

Expand Down

0 comments on commit f790973

Please sign in to comment.