(feat): Added manageDestinationExternally option to diff trigger creation. #881
Closed
Chriztiaan wants to merge 14 commits intomainfrom
Closed
(feat): Added manageDestinationExternally option to diff trigger creation. #881Chriztiaan wants to merge 14 commits intomainfrom
manageDestinationExternally option to diff trigger creation. #881Chriztiaan wants to merge 14 commits intomainfrom
Conversation
…e destination table to be persisted beyond cleanup of a trigger, also allows the creation to complete even if the destination table already exists.
🦋 Changeset detectedLatest commit: 3ccc8e3 The changes in this PR will be included in the next version bump. This PR includes changesets to release 8 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Chriztiaan
changed the title
persistDestination option to diff trigger creation. manageDestinationExternally option to diff trigger creation.
Chriztiaan
force-pushed
the
feature/persisted-diff-trigger-destination
branch
from
aff4d8b to
49f564f
Compare
…l internal management of the destination table.
Chriztiaan
force-pushed
the
feature/persisted-diff-trigger-destination
branch
from
49f564f to
8e501da
Compare
…ce cleanup regex won't automatically clear externally managed triggers.
stevensJourney
approved these changes
Mar 10, 2026
Contributor
Author
|
Closing this PR in favour of #888. |
Chriztiaan
changed the title
manageDestinationExternally option to diff trigger creation. manageDestinationExternally option to diff trigger creation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The first version of the TanstackDB
on-demandsync mode (temp-tanstack-db#6) repeatedly recreates diff triggers on the same destination table as the targetWHEREclause changes. Each cycle (dispose old trigger and create new trigger) drops and recreates the destination table, this raises a potential issues with pending mutations and creates gaps that would need to be covered with locks.Solution
Add a
manageDestinationExternallyoption tocreateDiffTrigger(). When enabled:The SDK no longer creates or drops the destination table and the caller owns the full lifecycle (create before first trigger, drop when done). Since the SDK isn't managing the table, it shouldn't be part of the the
TriggerClaimManagerclaim/release flow. Lastly, triggers created with this option use a different naming patternso that the periodic
cleanupResourcesregex(/^_ps_temp_trigger…/)skips them. This avoids the cleanup process from dropping externally-managed tables it doesn't own.A
createDiffDestinationTablehelper is also exposed so callers don't need to replicate the create table destination table query themselves.