You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current resumability logic uses binary search to find the first missing ledger in a given range. While this approach is very efficient and independent of the storage backend, it can be unintuitive for users. It also requires that there are no multiple gaps within the range being exported meets the following precondition:
This feature requires ledgers to be present on the remote data store for some (possibly empty) prefix of the requested range and then absent for the (possibly empty) remainder.
What Would You Like to See?
Reimplement resumability by replacing binary search with a method that lists ledger files within the given range to identify the most recent ledger. The object naming convention we have chosen will ensure that the objects can be lexicographically sorted. This approach is more intuitive as well as deterministic. For bounded mode, the new logic would be efficient since we can list all objects in sorted order with the latest ledger first. However, for unbounded mode, we’ll need to find the most recent ledger that is less than or equal to the end ledger which requires further consideration.
We can also use a hybrid approach, using the listing method for storage backends like GCS that support sorted listing and defaulting to binary search for backends that do not.
What Alternatives Are There?
keep the current binary search implementation
The text was updated successfully, but these errors were encountered:
What Problem Does Your Feature Solve?
The current resumability logic uses binary search to find the first missing ledger in a given range. While this approach is very efficient and independent of the storage backend, it can be unintuitive for users. It also requires that there are no multiple gaps within the range being exported meets the following precondition:
This feature requires ledgers to be present on the remote data store for some (possibly empty) prefix of the requested range and then absent for the (possibly empty) remainder.
What Would You Like to See?
Reimplement resumability by replacing binary search with a method that lists ledger files within the given range to identify the most recent ledger. The object naming convention we have chosen will ensure that the objects can be lexicographically sorted. This approach is more intuitive as well as deterministic. For bounded mode, the new logic would be efficient since we can list all objects in sorted order with the latest ledger first. However, for unbounded mode, we’ll need to find the most recent ledger that is less than or equal to the end ledger which requires further consideration.
We can also use a hybrid approach, using the listing method for storage backends like GCS that support sorted listing and defaulting to binary search for backends that do not.
What Alternatives Are There?
keep the current binary search implementation
The text was updated successfully, but these errors were encountered: