Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: CRUD Upload Queue Monitoring #311

Merged
merged 3 commits into from
Sep 20, 2024
Merged

Conversation

stevensJourney
Copy link
Collaborator

Overview

#254 Added monitoring of the ps_crud CRUD upload operations.

The monitoring relies on ps_crud rows being consumed and removed by the BackendConnector uploadData method. An upload delay is added if the CRUD entry at the head of the queue has not been changed between calls to uploadData.

The ps_crud table contains id and data columns. id Should be a unique incrementing integer while the data column contains JSON of the CRUD data change operation.
image

The raw ps_crud entry is parsed into a CrudEntry instance which exposes the data above. The mapping in the parsing maps ps_crud:id => CrudEntry.clientId and ps_crud:data:id to CrudEntry.id.

The previous implementation incorrectly assumed CrudEntry.id was the ps_crud:id column value. This causes the monitoring check to think CRUD operations have not been consumed if there are multiple ps_crud operations for the same row (with the same data id). This causes a delay in uploads if a single row has been mutated multiple times.

This PR updates the CRUD monitoring to use the correct id value. Unit tests are also added to ensure the delay functions as planned.

Copy link

changeset-bot bot commented Sep 20, 2024

🦋 Changeset detected

Latest commit: e70477b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@powersync/common Patch
@powersync/web Patch
@powersync/react-native Patch
@powersync/diagnostics-app Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

rkistner
rkistner previously approved these changes Sep 20, 2024
@stevensJourney stevensJourney merged commit 944ee93 into main Sep 20, 2024
5 checks passed
@stevensJourney stevensJourney deleted the fix/crud-upload-delay branch September 20, 2024 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants