From f79097373720530683857452b543e20dd6f25060 Mon Sep 17 00:00:00 2001 From: Nicolas Fricke Date: Tue, 13 Apr 2021 19:38:57 +0200 Subject: [PATCH] Ensure records are fetched early to avoid multiple DB queries One of the optimizations that cursor pagination allows is fetching one more record than required from the database to then be able to efficiently determine if there is another page (which would contain at the least this extra item). However, the way this was implemented it actually triggered individual queries, one to load the records and another `COUNT` query to determine the amount of records with the additional one. This was due to how ActiveRecord tries to delay fetching records of a relation to the latest possible point. To prevent this, we can call `#fetch` manually to request the records to be loaded earlier. --- CHANGELOG.md | 3 +++ lib/rails_cursor_pagination/paginator.rb | 14 ++++++++------ 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 61228c6..5ea49a4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,9 @@ These are the latest changes on the project's `master` branch that have not yet ### Changed - **Breaking change:** The way records are retrieved from a given cursor has been changed to no longer use `CONCAT` but instead simply use a compound `WHERE` clause in case of a custom order and having both the custom field as well as the `id` field in the `ORDER BY` query. This is a breaking change since it now changes the internal order of how records with the same value of the `order_by` field are returned. - Remove `ORDER BY` clause from `COUNT` queries + +### Fixed +- Only trigger one SQL query to load the records from the database and use it to determine if there was a previous / is a next page ## [0.1.3] - 2021-03-17 diff --git a/lib/rails_cursor_pagination/paginator.rb b/lib/rails_cursor_pagination/paginator.rb index 9a93cbe..fda0bba 100644 --- a/lib/rails_cursor_pagination/paginator.rb +++ b/lib/rails_cursor_pagination/paginator.rb @@ -191,7 +191,7 @@ def previous_page? else # When paginating backwards, if we managed to load one more record than # requested, this record will be available on the previous page. - @page_size < limited_relation_plus_one.reorder('').size + records_plus_one.size > @page_size end end @@ -202,7 +202,7 @@ def next_page? if paginate_forward? # When paginating forward, if we managed to load one more record than # requested, this record will be available on the next page. - @page_size < limited_relation_plus_one.reorder('').size + records_plus_one.size > @page_size else # When paginating backward, if applying our cursor reduced the number # records returned, we know that the missing records will be on @@ -215,7 +215,7 @@ def next_page? # # @return [Array] def records - records = limited_relation_plus_one.first(@page_size) + records = records_plus_one.first(@page_size) paginate_forward? ? records : records.reverse end @@ -223,11 +223,13 @@ def records # Apply limit to filtered and sorted relation that contains one item more # than the user-requested page size. This is useful for determining if there # is an additional page available without having to do a separate DB query. + # Then, fetch the records from the database to prevent multiple queries to + # load the records and count them. # # @return [ActiveRecord::Relation] - def limited_relation_plus_one - memoize :limited_relation_plus_one do - filtered_and_sorted_relation.limit(@page_size + 1) + def records_plus_one + memoize :records_plus_one do + filtered_and_sorted_relation.limit(@page_size + 1).load end end