Skip to content

Conversation

@samwillis
Copy link
Collaborator

@samwillis samwillis commented Jun 24, 2025

Implements and closes #195

Adds automatic lifecycle management for collections to optimize resource usage.

New Features:

  • Added startSync option (defaults to true, set to false for lazy loading)
  • Automatic garbage collection after gcTime (default 5 minutes) of inactivity
  • Collection status tracking: "idle" | "loading" | "ready" | "error" | "cleaned-up"
  • Manual preload() and cleanup() methods for lifecycle control

Usage:

const collection = createCollection({
  startSync: false, // set to true to start sync on creation
  gcTime: 300000, // Cleanup timeout (default: 5 minutes)
})

console.log(collection.status) // Current state
await collection.preload() // Ensure ready
await collection.cleanup() // Manual cleanup

@changeset-bot
Copy link

changeset-bot bot commented Jun 24, 2025

🦋 Changeset detected

Latest commit: 2708e73

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages
Name Type
@tanstack/db Patch
@tanstack/vue-db Patch
@tanstack/react-db Patch
@tanstack/db-collections Patch
@tanstack/db-example-react-todo Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Jun 24, 2025

@tanstack/db-example-react-todo

npm i https://pkg.pr.new/@tanstack/db@198
npm i https://pkg.pr.new/@tanstack/db-collections@198
npm i https://pkg.pr.new/@tanstack/react-db@198
npm i https://pkg.pr.new/@tanstack/vue-db@198

commit: 2708e73

@github-actions
Copy link
Contributor

github-actions bot commented Jun 24, 2025

Size Change: +1.49 kB (+5.48%) 🔍

Total Size: 28.7 kB

Filename Size Change
./packages/db/dist/esm/collection.js 7.87 kB +1.48 kB (+23.1%) 🚨
./packages/db/dist/esm/query/compiled-query.js 1.49 kB +14 B (+0.95%)
ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/deferred.js 230 B
./packages/db/dist/esm/errors.js 150 B
./packages/db/dist/esm/index.js 409 B
./packages/db/dist/esm/proxy.js 3.75 kB
./packages/db/dist/esm/query/evaluators.js 1.06 kB
./packages/db/dist/esm/query/extractors.js 870 B
./packages/db/dist/esm/query/functions.js 1.28 kB
./packages/db/dist/esm/query/group-by.js 976 B
./packages/db/dist/esm/query/joins.js 1.14 kB
./packages/db/dist/esm/query/order-by.js 1.42 kB
./packages/db/dist/esm/query/pipeline-compiler.js 878 B
./packages/db/dist/esm/query/query-builder.js 2.14 kB
./packages/db/dist/esm/query/select.js 1.1 kB
./packages/db/dist/esm/query/utils.js 1.13 kB
./packages/db/dist/esm/SortedMap.js 1.24 kB
./packages/db/dist/esm/transactions.js 1.33 kB
./packages/db/dist/esm/utils.js 219 B

compressed-size-action::db-package-size

@github-actions
Copy link
Contributor

github-actions bot commented Jun 24, 2025

Size Change: 0 B

Total Size: 822 B

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 173 B
./packages/react-db/dist/esm/useLiveQuery.js 409 B
./packages/react-db/dist/esm/useOptimisticMutation.js 240 B

compressed-size-action::react-db-package-size

@samwillis samwillis changed the title Implement PRD proposals and run tests Implement collection lifecycle PRD proposals Jun 24, 2025
- Resolved merge conflicts in collection.ts, compiled-query.ts, and types.ts
- Combined lifecycle management features with compare functionality
- Added startSyncImmediate method for compiled queries to override lazy loading
- Preserved all lifecycle features: lazy loading, GC timers, status tracking
- Maintained compatibility with new compare option for sorted collections
- All 384 tests passing
@samwillis samwillis marked this pull request as ready for review June 24, 2025 13:04
@samwillis samwillis requested a review from KyleAMathews June 24, 2025 13:04
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Stale Promise Cache and Memory Leak

The preloadPromise is not reset after it resolves or rejects, leading to stale promise caching. This prevents subsequent preload() calls from retrying or initiating a fresh preload, even after cleanup or error recovery. Additionally, collections are added to a global collectionsStore in the constructor but are never removed during cleanup(), causing a memory leak.

packages/db/src/collection.ts#L338-L407

*/
public preload(): Promise<void> {
if (this.preloadPromise) {
return this.preloadPromise
}
this.preloadPromise = new Promise<void>((resolve, reject) => {
if (this._status === `ready`) {
resolve()
return
}
if (this._status === `error`) {
reject(new Error(`Collection is in error state`))
return
}
// Start sync if collection was cleaned up
if (this._status === `cleaned-up`) {
try {
this.startSync()
} catch (error) {
reject(error)
return
}
}
// Wait for first commit
this.onFirstCommit(() => {
resolve()
})
})
return this.preloadPromise
}
/**
* Clean up the collection by stopping sync and clearing data
* This can be called manually or automatically by garbage collection
*/
public async cleanup(): Promise<void> {
// Clear GC timeout
if (this.gcTimeoutId) {
clearTimeout(this.gcTimeoutId)
this.gcTimeoutId = null
}
// Stop sync
if (this.syncCleanupFn) {
this.syncCleanupFn()
this.syncCleanupFn = null
}
// Clear data
this.syncedData.clear()
this.syncedMetadata.clear()
this.derivedUpserts.clear()
this.derivedDeletes.clear()
this._size = 0
this.pendingSyncedTransactions = []
this.syncedKeys.clear()
this.hasReceivedFirstCommit = false
this.onFirstCommitCallbacks = []
this.preloadPromise = null
// Update status
this._status = `cleaned-up`
return Promise.resolve()
}

Fix in Cursor


Bug: Sorting Fails for Valid Index Zero

The toArray getter's sorting logic incorrectly checks for the truthiness of _orderByIndex on the first array item. This prevents sorting when _orderByIndex is 0 (a valid index) or when the first item lacks the property but other items in the array have it and require sorting.

packages/db/src/collection.ts#L1373-L1378

// should be done by the query engine.
if (array[0] && (array[0] as { _orderByIndex?: number })._orderByIndex) {
return (array as Array<{ _orderByIndex: number }>).sort(
(a, b) => a._orderByIndex - b._orderByIndex
) as Array<T>
}

Fix in Cursor


Bug: Collection Methods Fail to Remove Subscribers

Memory leak in Collection's asStoreMap() and asStoreArray() methods. Both methods call addSubscriber() but lack a corresponding removeSubscriber() call. This keeps activeSubscribersCount above zero, preventing the Collection instance from being garbage collected and rendering its lifecycle management (GC timer) ineffective.

packages/db/src/collection.ts#L1510-L1517

// Start sync and track subscriber - this subscription will never be removed
// as the store is kept alive for the lifetime of the collection
this.addSubscriber()
this.changeListeners.add(() => {
this._storeMap!.setState(() => new Map(this.entries()))
})

packages/db/src/collection.ts#L1535-L1542

// as the store is kept alive for the lifetime of the collection
this.addSubscriber()
this.changeListeners.add(() => {
this._storeArray!.setState(() => this.toArray)
})
}
return this._storeArray

Fix in Cursor


BugBot free trial expires on July 22, 2025
You have used $0.00 of your $50.00 spend limit so far. Manage your spend limit in the Cursor dashboard.

Was this report helpful? Give feedback by reacting with 👍 or 👎

Copy link
Collaborator

@KyleAMathews KyleAMathews left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

We need some tests & implementing cleanup in electric/query.

@samwillis
Copy link
Collaborator Author

@KyleAMathews this is ready for another review, I have added back the tests that were accidentally deleted.

The electric sync now returns the unsubscribe function for cleanup:

// Return the unsubscribe function
return stream.subscribe((messages: Array<Message<Row>>) => {

And the query sync already did:

return actualUnsubscribeFn

@samwillis samwillis requested a review from KyleAMathews June 25, 2025 13:27
@samwillis samwillis requested a review from KyleAMathews June 25, 2025 18:20
Copy link
Collaborator

@KyleAMathews KyleAMathews left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

nice work!

@samwillis samwillis merged commit 945868e into main Jun 26, 2025
4 checks passed
@samwillis samwillis deleted the cursor/implement-prd-proposals-and-run-tests-9ff9 branch June 26, 2025 13:12
@github-actions github-actions bot mentioned this pull request Jun 25, 2025
cursor bot pushed a commit that referenced this pull request Jul 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[PRD]: Collection Lifecycle for TanStack DB

4 participants