Skip to content

Add statement-level query tag support#341

Open
joohoyeo-dev wants to merge 3 commits intodatabricks:mainfrom
joohoyeo-dev:statement-level-query-tags
Open

Add statement-level query tag support#341
joohoyeo-dev wants to merge 3 commits intodatabricks:mainfrom
joohoyeo-dev:statement-level-query-tags

Conversation

@joohoyeo-dev
Copy link
Copy Markdown

Summary

  • Adds per-statement query tag support via driverctx.NewContextWithQueryTags, allowing users to attach query tags to individual SQL statements through context
  • Tags are serialized into TExecuteStatementReq.ConfOverlay["query_tags"], consistent with the Python (#736) and NodeJS (#339) connector implementations
  • Previously only session-level query tags were supported (set once via WithSessionParams at connection time)

Usage

ctx := driverctx.NewContextWithQueryTags(context.Background(), map[string]string{
    "team": "data-eng",
    "app":  "etl-pipeline",
})
rows, err := db.QueryContext(ctx, "SELECT * FROM table")

Changes

File Description
driverctx/ctx.go NewContextWithQueryTags, QueryTagsFromContext, propagation in NewContextFromBackground
query_tags.go (new) SerializeQueryTags — map to wire format with escaping
connection.go Read tags from context → serialize → set ConfOverlay["query_tags"]
driverctx/ctx_test.go 5 tests for context helpers
query_tags_test.go (new) 13 tests for serialization (escaping, edge cases)
connection_test.go 6 integration tests verifying ConfOverlay behavior
examples/query_tags/main.go Updated with session + statement-level examples

Test plan

  • Unit tests for SerializeQueryTags covering nil, empty, single/multi tags, escaping of \, :, , in values and keys
  • Unit tests for NewContextWithQueryTags / QueryTagsFromContext including nil context, missing key, timeout preservation, background propagation
  • Integration tests verifying ConfOverlay["query_tags"] is correctly set (or absent) in captured TExecuteStatementReq
  • Verify existing tests still pass (CI)

This pull request was AI-assisted by Isaac.

@joohoyeo-dev
Copy link
Copy Markdown
Author

joohoyeo-dev commented Apr 2, 2026

⏺ Test 1 — Session-level only:
// Expected in Query History: {"team": "platform", "test": "go-connector-pr"}
db.QueryRowContext(context.Background(), "SELECT 1")

Test 2 — Statement-level tags:
// Expected in Query History: {"team": "query-tags-test", "application": "go-connector-e2e", "env": "staging"}
ctx := driverctx.NewContextWithQueryTags(context.Background(), map[string]string{
"team": "query-tags-test",
"application": "go-connector-e2e",
"env": "staging",
})
db.QueryRowContext(ctx, "SELECT 2")

Test 3 — Different statement-level tags:
// Expected in Query History: {"team": "analytics", "job": "daily-report"}
ctx2 := driverctx.NewContextWithQueryTags(context.Background(), map[string]string{
"team": "analytics",
"job": "daily-report",
})
db.QueryRowContext(ctx2, "SELECT 3")

Test 4 — No statement-level tags (session-level still applies):
// Expected in Query History: {"team": "platform", "test": "go-connector-pr"}
db.QueryRowContext(context.Background(), "SELECT 4")

@joohoyeo-dev
Copy link
Copy Markdown
Author

Screenshot 2026-04-01 at 9 05 50 PM Screenshot 2026-04-01 at 9 06 06 PM Screenshot 2026-04-01 at 9 06 11 PM Screenshot 2026-04-01 at 9 06 14 PM

Copy link
Copy Markdown

@jiabin-hu jiabin-hu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were you able to run the sample code successfully?

EDIT: nvm saw your run results

// NewContextWithQueryTags creates a new context with per-statement query tags.
// These tags are serialized and passed via confOverlay as "query_tags" in TExecuteStatementReq.
// They apply only to the statement executed with this context and do not persist across queries.
func NewContextWithQueryTags(ctx context.Context, queryTags map[string]string) context.Context {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is the best way to create a context object with an optional param in Go. I'm not familiar w/ Go. Perhaps ask Claude to double check.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question! This is actually the idiomatic Go pattern for per-request metadata in database/sql drivers. Since the QueryContext(ctx, query, args) interface is fixed by the standard library, context values are the standard way to pass per-request options.

This driver already uses the same pattern for other per-request data:

  • driverctx.NewContextWithConnId
  • driverctx.NewContextWithCorrelationId
  • driverctx.NewContextWithQueryId
  • driverctx.NewContextWithStagingInfo

So NewContextWithQueryTags follows the established convention.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either in this PR or as follow up, we should allow user to pass in a map as query tags so it's consistent with the statement level. I did it in Python and plan to do in NodeJS, too.

databricks/databricks-sql-python@e916f71

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great point — addressed in 606f44e. Added WithQueryTags(map[string]string) as a connector option that accepts a structured map and handles serialization internally, consistent with the statement-level API and your Python approach (databricks/databricks-sql-python@e916f71).

// Before: pre-serialized string
dbsql.WithSessionParams(map[string]string{
    "QUERY_TAGS": "team:engineering,app:etl",
})

// After: structured map (preferred)
dbsql.WithQueryTags(map[string]string{
    "team": "engineering",
    "app":  "etl",
})

WithSessionParams still works for backward compatibility. If both are used, WithQueryTags takes precedence (applied after in the options chain). Tests added in connector_test.go.

@joohoyeo-dev
Copy link
Copy Markdown
Author

checked the tags still work as expected in Logfood

Copy link
Copy Markdown
Collaborator

@vikrantpuppala vikrantpuppala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please fix failing checks, lgtm otherwise, thanks!

joohoyeo added 3 commits April 2, 2026 08:41
Previously, query tags could only be set at the session level via
WithSessionParams during connection creation. This adds per-statement
query tag support, allowing different tags for each query execution.

Users pass query tags through context using the new
driverctx.NewContextWithQueryTags function. The tags are serialized
into the TExecuteStatementReq.ConfOverlay["query_tags"] field,
consistent with the Python and NodeJS connector implementations.

Co-authored-by: Isaac
Signed-off-by: Jooho Yeo <jooho.yeo@databricks.com>
Verifies that session-level tags (TOpenSessionReq.Configuration) and
statement-level tags (TExecuteStatementReq.ConfOverlay) are independent:
session params don't leak into ConfOverlay, and statement-level tags
are correctly set even when session-level tags exist.

Co-authored-by: Isaac
Signed-off-by: Jooho Yeo <jooho.yeo@databricks.com>
Addresses review feedback from jiabin-hu:

1. Add WithQueryTags(map[string]string) as a connector option that
   accepts a structured map and serializes it internally, consistent
   with the statement-level API and the Python connector approach
   (databricks/databricks-sql-python@e916f71).

2. Context values pattern is the idiomatic Go approach for per-request
   metadata in database/sql drivers (same pattern used by ConnId,
   CorrelationId, QueryId, and StagingInfo in this driver).

Co-authored-by: Isaac
Signed-off-by: Jooho Yeo <jooho.yeo@databricks.com>
@joohoyeo-dev joohoyeo-dev force-pushed the statement-level-query-tags branch from 606f44e to 04704e1 Compare April 2, 2026 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants