fix: enforce all where conditions when index optimization is partial

.changeset/index-optimization-partial-and-or.md

@@ -0,0 +1,17 @@
+---
+'@tanstack/db': patch
+---
+
+Fix incorrect results from index-optimized `where` clauses that combine indexed and non-indexed conditions.
+
+- `OR` expressions are now only served from indexes when every disjunct can use an index; otherwise the query falls back to a full scan. Previously, rows matched only by a non-indexed disjunct were missing from the result.
+- `AND` expressions still use indexes for the conditions that have them, but the remaining conditions are now enforced by re-checking each candidate row against the full expression. Previously, non-indexed conditions were silently dropped, returning rows that did not match the query.
+- Compound range conditions (e.g. `age > 5 AND age < 10`) combined with conditions on other fields no longer ignore those other conditions.
+- Compound range conditions sharing the same boundary value (e.g. `age >= 5 AND age > 5`) now apply the strictest bound regardless of the order the conditions appear in, using the same value comparison semantics as the indexes (dates, locale strings, ...).
+- Compound range conditions that only bound one side (e.g. `age > 5 AND age >= 8`) no longer return an empty result.
+- Strict range comparisons (`gt`/`lt`) on BTree-indexed fields holding normalized values such as dates now correctly exclude the boundary value.
+- Compound range conditions with a `null`/`undefined` bound (e.g. `gt(score, undefined)`) now re-filter against the full expression instead of returning index-ordered rows, matching the semantics of a full scan (a comparison against `null`/`undefined` is never true).
+- Index-optimized `eq`, `IN`, and range queries on a field that has rows with `null`/`undefined` values no longer leak those rows into results. BTree indexes store and return such rows (they sort as the smallest key), but a comparison against `null`/`undefined` is never true, so these results are now re-filtered against the full expression to stay equivalent to a full scan.
+- String range conditions (`gt`/`gte`/`lt`/`lte`) on a collection using locale string collation (the default) are no longer served by the index. The index orders strings with `localeCompare` while the `where` evaluator compares them with standard relational operators, so an index range lookup could omit matching rows; these conditions now fall back to a full scan.
+- Range conditions whose operand is not ordered the same way by the index and the `where` evaluator (arrays, plain objects, Temporal values) now fall back to a full scan instead of using the index, which could otherwise omit matching rows.
+- Range conditions on an index created with a custom comparator now fall back to a full scan, since the comparator's ordering may not match the `where` evaluator's relational operators.

.changeset/nan-postgres-semantics.md

@@ -0,0 +1,15 @@
+---
+'@tanstack/db': patch
+---
+
+Adopt PostgreSQL float semantics for `NaN` in `where` clauses and ordering.
+
+`NaN` (and invalid `Date` values, whose timestamp is `NaN`) previously had no consistent order — `NaN === NaN` is `false` in JavaScript, so `NaN` compared unequal to everything and could not be sorted or indexed deterministically. Following PostgreSQL, `NaN` is now treated as **equal to itself** and **greater than every other non-null value**:
+
+- `eq(row.value, NaN)` matches rows whose value is `NaN`; `inArray(row.value, [NaN, ...])` matches them too.
+- Range comparisons treat `NaN` as the greatest value: `gt`/`gte` include it, `lt`/`lte` exclude it.
+- Ordering by a field containing `NaN` is now deterministic, with `NaN` sorting last (and `null` still ordered by `NULLS FIRST`/`NULLS LAST`).
+
+`null`/`undefined` are unaffected: they continue to use three-valued logic (a comparison with `null` yields `UNKNOWN`).
+
+This makes results independent of whether a query is served from an index or a full scan.

docs/guides/live-queries.md

@@ -728,6 +728,13 @@ not(condition)
 
 For a complete reference of all available functions, see the [Expression Functions Reference](#expression-functions-reference) section.
 
+### Comparison semantics
+
+Comparisons follow SQL/PostgreSQL conventions rather than raw JavaScript:
+
+- **`null` / `undefined` use three-valued logic.** Any comparison involving `null` or `undefined` evaluates to `UNKNOWN`, so the row is not matched. For example `eq(user.score, null)` matches nothing — use a dedicated null check (e.g. `isUndefined`) to match missing values.
+- **`NaN` follows PostgreSQL float semantics.** `NaN` is treated as equal to itself and greater than every other (non-null) value. So `eq(row.value, NaN)` matches `NaN` rows, `gt(row.value, x)` includes `NaN`, and ordering by such a field places `NaN` last. (Invalid `Date` values, whose timestamp is `NaN`, behave the same way.) This differs from JavaScript, where `NaN === NaN` is `false`, and matches how PostgreSQL orders and indexes floating-point values.
+
 ## Select
 
 Use `select` to specify which fields to include in your results and transform your data. Without `select`, you get the full schema.

packages/db/src/collection/change-events.ts

@@ -138,11 +138,16 @@ export function currentStateAsChanges<
     )
 
     if (optimizationResult.canOptimize) {
-      // Use index optimization
+      // Use index optimization. When the index lookup is inexact, the keys
+      // are a superset of the true result (some conditions could not be
+      // served by an index), so re-check each row against the full expression.
+      const filterFn = optimizationResult.isExact
+        ? undefined
+        : createFilterFunctionFromExpression(expression)
       const result: Array<ChangeMessage<WithVirtualProps<T, TKey>, TKey>> = []
       for (const key of optimizationResult.matchingKeys) {
         const value = collection.get(key)
-        if (value !== undefined) {
+        if (value !== undefined && (filterFn?.(value) ?? true)) {
           result.push({
             type: `insert`,
             key,

packages/db/src/indexes/base-index.ts

@@ -68,6 +68,15 @@ export interface IndexInterface<
 
   supports: (operation: IndexOperation) => boolean
 
+  /**
+   * Whether range lookups (gt/gte/lt/lte) on this index can be trusted to
+   * return every matching key. Range traversal relies on the index ordering, so
+   * it is unsafe when the index uses a custom comparator, whose order may not
+   * match the WHERE evaluator's relational operators. Callers must fall back to
+   * a full scan when this is `false`.
+   */
+  get supportsRangeOptimization(): boolean
+
   matchesField: (fieldPath: Array<string>) => boolean
   matchesCompareOptions: (compareOptions: CompareOptions) => boolean
   matchesDirection: (direction: OrderByDirection) => boolean
@@ -90,6 +99,11 @@ export abstract class BaseIndex<
   protected totalLookupTime = 0
   protected lastUpdated = new Date()
   protected compareOptions: CompareOptions
+  /**
+   * Set by subclasses when constructed with a user-supplied comparator, whose
+   * ordering may not match the WHERE evaluator's relational operators.
+   */
+  protected hasCustomComparator = false
 
   constructor(
     id: number,
@@ -144,6 +158,10 @@ export abstract class BaseIndex<
     return this.supportedOperations.has(operation)
   }
 
+  get supportsRangeOptimization(): boolean {
+    return !this.hasCustomComparator
+  }
+
   matchesField(fieldPath: Array<string>): boolean {
     return (
       this.expression.type === `ref` &&

packages/db/src/indexes/basic-index.ts

@@ -65,6 +65,7 @@ export class BasicIndex<
   ) {
     super(id, expression, name, options)
     this.compareFn = options?.compareFn ?? defaultComparator
+    this.hasCustomComparator = options?.compareFn != null
     if (options?.compareOptions) {
       this.compareOptions = options!.compareOptions
     }

packages/db/src/indexes/btree-index.ts

@@ -62,6 +62,7 @@ export class BTreeIndex<
 
     // Get the base compare function
     const baseCompareFn = options?.compareFn ?? defaultComparator
+    this.hasCustomComparator = options?.compareFn != null
 
     // Wrap it to denormalize sentinels before comparison
     // This ensures UNDEFINED_SENTINEL is converted back to undefined
@@ -247,7 +248,16 @@ export class BTreeIndex<
       toKey,
       toInclusive,
       (indexedValue, _) => {
-        if (!fromInclusive && this.compareFn(indexedValue, from) === 0) {
+        // Only exclude the boundary when an exclusive lower bound was
+        // actually provided. Without a `from` bound, `fromKey` defaults to
+        // the minimum key and must not be dropped. Compare against the
+        // normalized key since indexed values are stored normalized
+        // (e.g. dates as timestamps), so the raw `from` would never match.
+        if (
+          hasFrom &&
+          !fromInclusive &&
+          this.compareFn(indexedValue, fromKey) === 0
+        ) {
           // the B+ tree `forRange` method does not support exclusive lower bounds
           // so we need to exclude it manually
           return

packages/db/src/indexes/reverse-index.ts

@@ -73,6 +73,10 @@ export class ReverseIndex<
     return this.originalIndex.supports(operation)
   }
 
+  get supportsRangeOptimization(): boolean {
+    return this.originalIndex.supportsRangeOptimization
+  }
+
   matchesField(fieldPath: Array<string>): boolean {
     return this.originalIndex.matchesField(fieldPath)
   }

packages/db/src/query/compiler/evaluators.ts

@@ -3,7 +3,11 @@ import {
   UnknownExpressionTypeError,
   UnknownFunctionError,
 } from '../../errors.js'
-import { areValuesEqual, normalizeValue } from '../../utils/comparison.js'
+import {
+  areValuesEqual,
+  isUnorderable,
+  normalizeValue,
+} from '../../utils/comparison.js'
 import type { BasicExpression, Func, PropRef } from '../ir.js'
 import type { NamespacedRow } from '../../types.js'
 
@@ -14,6 +18,19 @@ function isUnknown(value: any): boolean {
   return value === null || value === undefined
 }
 
+/**
+ * Equality that follows PostgreSQL float semantics for `NaN`/invalid Dates:
+ * such values are equal to one another and unequal to anything else. For all
+ * other values it defers to {@link areValuesEqual}. Operands must not be
+ * null/undefined (callers handle UNKNOWN first).
+ */
+function valuesEqual(a: any, b: any): boolean {
+  if (isUnorderable(a) || isUnorderable(b)) {
+    return isUnorderable(a) && isUnorderable(b)
+  }
+  return areValuesEqual(a, b)
+}
+
 function toDateValue(value: any): Date | null {
   if (value instanceof Date) {
     return Number.isNaN(value.getTime()) ? null : value
@@ -233,8 +250,9 @@ function compileFunction(func: Func, isSingleRow: boolean): (data: any) => any {
         if (isUnknown(a) || isUnknown(b)) {
           return null
         }
-        // Use areValuesEqual for proper Uint8Array/Buffer comparison
-        return areValuesEqual(a, b)
+        // NaN/invalid Dates are equal to one another (PostgreSQL semantics);
+        // otherwise use areValuesEqual for proper Uint8Array/Buffer comparison
+        return valuesEqual(a, b)
       }
     }
     case `gt`: {
@@ -247,6 +265,11 @@ function compileFunction(func: Func, isSingleRow: boolean): (data: any) => any {
         if (isUnknown(a) || isUnknown(b)) {
           return null
         }
+        // NaN/invalid Dates sort greater than every other value, and are equal
+        // to one another (PostgreSQL semantics)
+        if (isUnorderable(a) || isUnorderable(b)) {
+          return isUnorderable(a) && !isUnorderable(b)
+        }
         return a > b
       }
     }
@@ -260,6 +283,9 @@ function compileFunction(func: Func, isSingleRow: boolean): (data: any) => any {
         if (isUnknown(a) || isUnknown(b)) {
           return null
         }
+        if (isUnorderable(a) || isUnorderable(b)) {
+          return isUnorderable(a)
+        }
         return a >= b
       }
     }
@@ -273,6 +299,9 @@ function compileFunction(func: Func, isSingleRow: boolean): (data: any) => any {
         if (isUnknown(a) || isUnknown(b)) {
           return null
         }
+        if (isUnorderable(a) || isUnorderable(b)) {
+          return isUnorderable(b) && !isUnorderable(a)
+        }
         return a < b
       }
     }
@@ -286,6 +315,9 @@ function compileFunction(func: Func, isSingleRow: boolean): (data: any) => any {
         if (isUnknown(a) || isUnknown(b)) {
           return null
         }
+        if (isUnorderable(a) || isUnorderable(b)) {
+          return isUnorderable(b)
+        }
         return a <= b
       }
     }
@@ -370,7 +402,7 @@ function compileFunction(func: Func, isSingleRow: boolean): (data: any) => any {
         if (!Array.isArray(array)) {
           return false
         }
-        return array.some((item) => normalizeValue(item) === value)
+        return array.some((item) => valuesEqual(normalizeValue(item), value))
       }
     }
 

packages/db/src/utils/comparison.ts

@@ -17,6 +17,21 @@ function getObjectId(obj: object): number {
   return id
 }
 
+/**
+ * Whether a value has no IEEE-754 natural order: `NaN`, or an invalid Date
+ * (whose timestamp is `NaN`). The query engine follows PostgreSQL float
+ * semantics for these values — they are all equal to one another and greater
+ * than every other (non-null) value — so the comparator and the WHERE
+ * evaluator treat them explicitly instead of letting `NaN` compare unequal to
+ * everything (which has no consistent order and cannot be indexed or sorted).
+ */
+export function isUnorderable(value: any): boolean {
+  return (
+    (typeof value === `number` && Number.isNaN(value)) ||
+    (value instanceof Date && Number.isNaN(value.getTime()))
+  )
+}
+
 /**
  * Universal comparison function for all data types
  * Handles null/undefined, strings, arrays, dates, objects, and primitives
@@ -30,6 +45,16 @@ export const ascComparator = (a: any, b: any, opts: CompareOptions): number => {
   if (a == null) return nulls === `first` ? -1 : 1
   if (b == null) return nulls === `first` ? 1 : -1
 
+  // Handle NaN / invalid Dates. Following PostgreSQL float semantics, they are
+  // all equal and sort greater than every other non-null value. This keeps the
+  // order total (NaN would otherwise compare equal to everything), so such
+  // values can be sorted and stored in tree-based indexes.
+  const aUnordered = isUnorderable(a)
+  const bUnordered = isUnorderable(b)
+  if (aUnordered && bUnordered) return 0
+  if (aUnordered) return 1
+  if (bUnordered) return -1
+
   // if a and b are both strings, compare them based on locale
   if (typeof a === `string` && typeof b === `string`) {
     if (opts.stringSort === `locale`) {