Skip to content

fix(datagrid): render query results before loading table metadata (#1574)#1582

Merged
datlechin merged 3 commits into
mainfrom
fix/1574-query-render-latency
Jun 4, 2026
Merged

fix(datagrid): render query results before loading table metadata (#1574)#1582
datlechin merged 3 commits into
mainfrom
fix/1574-query-render-latency

Conversation

@datlechin
Copy link
Copy Markdown
Member

Closes #1574.

Problem

Against a remote MySQL (Alibaba Cloud RDS, ping ~50ms), a small query (~500 rows) shows a query "execution time" of ~500ms, but the grid takes 3-5 seconds to appear with a spinner. Local databases are fast, and DataGrip/Navicat against the same remote DB respond in ~1s. Disabling TLS does not help.

Root cause

The displayed execution time measured only the SELECT on the primary connection. The grid did not render until a separate, sequential metadata fetch finished:

  • On the first query to an editable table, the schema fetch was awaited inside QueryExecutor.executeQuery before the rows were applied.
  • That fetch opened a second connection (full TLS + auth handshake) and ran three sequential round-trips, two against information_schema (foreign keys, row count), which is slow on Alibaba RDS.

On localhost the round-trip cost is ~0ms, so the work was invisible. On a 50ms remote link with slow system tables it is the 3-5s gap. libmariadb already sets TCP_NODELAY, so Nagle was not the cause.

Fix

Render rows before metadata (all drivers). executeQuery now returns only the rows. The coordinator starts the schema fetch alongside the SELECT, renders the grid as soon as rows arrive, then applies the editing metadata in the background: column defaults, foreign keys, nullability, primary keys, enum values, and row count, bumping metadataVersion so editability lights up a moment later. Covers the editor-query and parameterized paths.

MySQL reads primary key and nullability from the result set. MariaDBPluginConnection now reads PRI_KEY_FLAG, NOT_NULL_FLAG, and AUTO_INCREMENT_FLAG from MYSQL_FIELD and carries them through PluginQueryResult.columnMeta. A simple SELECT * FROM table is fully editable at first paint with no extra round-trip.

Safety

  • Phase 2 uses a new non-destructive DataChangeManager.setPrimaryKeyColumns, so it cannot wipe edits made in the brief window before metadata lands.
  • SQLStatementGenerator already falls back to a full-row WHERE clause when no primary key is known, so that window stays correct for other drivers.
  • Every background metadata apply is guarded by queryGeneration, and the schema task is cancelled on query cancel or error.

Notes

  • PluginQueryResult gains an additive columnMeta field (defaults to nil). This is ABI-safe per the PluginKit rules, so no currentPluginKitVersion bump. The PluginKit ABI gate will report an additive diff; that is expected. The bundled MySQL plugin and the app rebuild together.

Tests

  • Unit tests for QueryExecutor.inlineMetadata (primary keys, nullability, composite keys, absent/empty, count mismatch).
  • A flag-parsing test for the MySQL makeColumnMeta mapping.
  • swiftlint --strict clean on all changed files.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 60179ab190

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1121 to +1122
metadata: inlineMeta,
hasSchema: false,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve primary-key default sorting while metadata loads

When needsMetadataFetch is true but the driver does not populate resultColumnMeta (currently every plugin except MySQL), this passes metadata: nil and hasSchema: false into applyPhase1Result. That method immediately calls applyDefaultSortIfPending; with the “Default row sort: Primary key” setting it resolves no PKs, marks didEvaluateDefaultSort = true, and later launchPhase2Work only applies the fetched PK metadata without re-running default-sort resolution. On first loading a table through PostgreSQL/SQLite/etc., the table therefore stays unsorted by primary key, whereas before this commit the schema was awaited and the PKs were available for phase 1.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9225af0374

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +357 to +362
parent.mutateActiveTableRows(for: tabId) { rows in
rows.updateDisplayMetadata(
columnDefaults: parsed.columnDefaults,
columnForeignKeys: parsed.columnForeignKeys,
columnNullable: parsed.columnNullable
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply phase 2 metadata to the originating result set

When the schema fetch finishes after the user has switched the active result set (for example to a pinned result set), this mutates the tab's currently active TableRows, not necessarily the result set created by the query; switchActiveResultSet swaps the session rows on selection. That lets defaults, foreign keys, and nullability from the newly executed table be written onto a different result set, producing incorrect edit metadata until the rows are reloaded. Carry the originating result-set id through phase 2 or skip the update if the active result set changed.

Useful? React with 👍 / 👎.

@datlechin datlechin merged commit 94ec469 into main Jun 4, 2026
3 of 5 checks passed
@datlechin datlechin deleted the fix/1574-query-render-latency branch June 4, 2026 16:37
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e2d4ae316a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

public let executionTime: TimeInterval
public let isTruncated: Bool
public let statusMessage: String?
public let columnMeta: [PluginColumnInfo]?
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Bump PluginKit for the PluginQueryResult ABI change

For user-installed drivers built against PluginKit 18 before this change, the version gate still accepts them (currentPluginKitVersion and minimumCompatiblePluginKitVersion remain 18), but their binary was compiled against the old PluginQueryResult layout/initializer. Adding this stored property changes the Swift ABI, so those plugins can fail when returning query results instead of being rejected with an update message; please bump the PluginKit version or add a compatibility wrapper with this ABI change.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Slow query response when connecting to Alibaba Cloud RDS (high perceived latency), other clients work fine

1 participant